Compare commits

...
30 Commits
Author SHA1 Message Date
christianmanivong 4e40f95bc1 Merge pull request 'feat(scheduled-jobs): ScheduledJobsMixin and its POSIX implementation (#86), release 4.27.0' (#87) from feature/scheduled-jobs into main
driver / types (push) Successful in 59s
driver / lint (push) Successful in 14s
driver / test (3.13) (push) Successful in 1m0s
driver / lowest (push) Successful in 35s
driver / build (push) Successful in 30s
driver / security (push) Successful in 54s
driver / publish (push) Successful in 41s
driver / standard (push) Successful in 15s
driver / test (3.12) (push) Successful in 1m6s
CI / driver (push) Successful in 3m7s
2026-11-03 07:37:25 +00:00
christianmanivong 70e1c9b218 chore(release): 4.27.0
driver / types (pull_request) Successful in 1m0s
driver / test (3.13) (pull_request) Successful in 1m11s
driver / test (3.12) (pull_request) Successful in 1m20s
driver / lowest (pull_request) Successful in 40s
driver / build (pull_request) Successful in 34s
driver / security (pull_request) Successful in 1m5s
CI / driver (pull_request) Successful in 2m33s
driver / lint (pull_request) Successful in 14s
driver / standard (pull_request) Successful in 17s
driver / publish (pull_request) Skipped
Scheduled jobs (NetOrk/netork#1933): ScheduledJobsMixin, its POSIX
implementation and napalm_device_types.cron. 4.26.0 went out meanwhile
with the sshd drop-in change, so this is 4.27.0.
2026-11-03 08:34:14 +01:00
christianmanivong a70501ff60 Merge remote-tracking branch 'origin/main' into feature/scheduled-jobs
# Conflicts:
#	CHANGELOG.md
2026-11-03 08:33:52 +01:00
christianmanivong c6d119eb83 fix(scheduled-jobs): a monotonic timer as its unit file says it, OnUnitActiveSec
driver / types (pull_request) Successful in 57s
driver / test (3.12) (pull_request) Successful in 1m12s
driver / lowest (pull_request) Successful in 43s
driver / test (3.13) (pull_request) Successful in 1m14s
driver / build (pull_request) Successful in 27s
driver / security (pull_request) Successful in 1m8s
CI / driver (pull_request) Successful in 2m47s
driver / standard (pull_request) Successful in 14s
driver / lint (pull_request) Successful in 13s
driver / publish (pull_request) Skipped
systemctl show names a timer's monotonic settings by their properties,
OnUnitActiveUSec=3h; the schedule now reads as the unit file writes it.
Found on a live Ubuntu host.

Refs #86
2026-11-03 08:11:48 +01:00
christianmanivong 8e9d21c368 Merge pull request 'feat(ssh): SSHD_CONFIG_GENERATED keeps the hardening to the drop-in (#88), release 4.26.0' (#89) from fix/ssh-hardening-generated-config into main
driver / lint (push) Successful in 23s
driver / types (push) Successful in 2m7s
driver / test (3.13) (push) Successful in 1m41s
driver / lowest (push) Successful in 1m30s
driver / build (push) Successful in 1m5s
driver / security (push) Successful in 1m41s
driver / publish (push) Successful in 1m11s
driver / test (3.12) (push) Successful in 1m35s
driver / standard (push) Successful in 17s
CI / driver (push) Successful in 8m25s
2026-11-03 07:00:40 +00:00
christianmanivong 25f28297e4 chore(release): 4.26.0
driver / standard (pull_request) Successful in 17s
driver / types (pull_request) Successful in 2m9s
driver / test (3.13) (pull_request) Successful in 2m2s
driver / test (3.12) (pull_request) Successful in 2m10s
driver / lowest (pull_request) Successful in 36s
driver / build (pull_request) Successful in 35s
driver / security (pull_request) Successful in 1m9s
CI / driver (pull_request) Successful in 3m28s
driver / lint (pull_request) Successful in 13s
driver / publish (pull_request) Skipped
2026-11-03 07:56:46 +01:00
christianmanivong 847c902193 feat(ssh): SSHD_CONFIG_GENERATED keeps the hardening to the drop-in (#88)
A platform that writes sshd_config from its own database (OpenMediaVault)
had to opt out of harden_ssh_server altogether. Where that generated file
includes /etc/ssh/sshd_config.d, the drop-in is safe; only the fallback
that rewrites sshd_config itself would be undone on the next apply. With
SSHD_CONFIG_GENERATED set, a sshd_config without the include is left alone
and the result says why.
2026-11-03 07:56:46 +01:00
christianmanivong ffbb194300 fix(scheduled-jobs): a file without a last newline no longer swallows the next header
driver / types (pull_request) Successful in 1m4s
driver / test (3.12) (pull_request) Successful in 1m6s
driver / test (3.13) (pull_request) Successful in 1m4s
driver / build (pull_request) Successful in 32s
driver / lowest (pull_request) Successful in 41s
driver / security (pull_request) Successful in 1m13s
CI / driver (pull_request) Successful in 2m43s
driver / standard (pull_request) Successful in 15s
driver / lint (pull_request) Successful in 14s
driver / publish (pull_request) Skipped
GNU sed keeps a missing final newline, so the next section's header ran
on into the file's last line and its jobs were lost. The report ends
every such file with a newline of its own.

Refs #86
2026-11-03 05:50:44 +01:00
christianmanivong edef6cf07c fix(scheduled-jobs): write a linked crontab through its link, with the target's mode
driver / types (pull_request) Successful in 1m11s
driver / test (3.12) (pull_request) Successful in 1m25s
driver / test (3.13) (pull_request) Successful in 1m18s
driver / lowest (pull_request) Successful in 40s
driver / build (pull_request) Successful in 36s
driver / security (pull_request) Successful in 1m8s
CI / driver (pull_request) Successful in 2m54s
driver / standard (pull_request) Successful in 16s
driver / lint (pull_request) Successful in 20s
driver / publish (pull_request) Skipped
stat without -L gave the link's own mode, 777, and install replaced the
link with a plain file of that mode: cron ignores a file anyone may write,
so every job in it stopped (/etc/cron.d/vzdump on older PVE links to
/etc/pve/vzdump.cron). The file the link names is written instead.

Refs #86
2026-11-03 05:39:57 +01:00
christianmanivong df476a12f2 feat(scheduled-jobs): the install command, time zone, owners and platform sources are the driver's
driver / types (pull_request) Successful in 1m7s
driver / test (3.13) (pull_request) Successful in 1m4s
driver / test (3.12) (pull_request) Successful in 1m12s
driver / lowest (pull_request) Successful in 39s
driver / build (pull_request) Successful in 30s
driver / security (pull_request) Successful in 1m6s
CI / driver (pull_request) Successful in 2m48s
driver / standard (pull_request) Successful in 17s
driver / lint (pull_request) Successful in 13s
driver / publish (pull_request) Skipped
A BSD install takes -D as a destdir, OpenWrt keeps its time zone in uci
and its package database in opkg, and a platform's generated files are its
own; each is a class attribute a driver sets.

Refs #86
2026-11-03 05:33:14 +01:00
christianmanivong 834cb2e0ea fix(scheduled-jobs): an own job goes to /etc/cron.d only where the host has it
driver / types (pull_request) Successful in 1m12s
driver / test (3.12) (pull_request) Successful in 1m21s
driver / lowest (pull_request) Successful in 39s
driver / test (3.13) (pull_request) Successful in 1m21s
driver / build (pull_request) Successful in 32s
driver / security (pull_request) Successful in 1m7s
CI / driver (pull_request) Successful in 3m3s
driver / standard (pull_request) Successful in 19s
driver / lint (pull_request) Successful in 15s
driver / publish (pull_request) Skipped
BusyBox crond reads no /etc/cron.d, so an Alpine host without systemd
takes the job in the user's crontab instead of a file it would never run.

Refs #86
2026-11-03 05:30:19 +01:00
christianmanivong f573fa53d2 feat(scheduled-jobs): ScheduledJobsMixin and its POSIX implementation (#86)
driver / types (pull_request) Successful in 1m16s
driver / test (3.13) (pull_request) Successful in 1m18s
driver / test (3.12) (pull_request) Successful in 1m27s
driver / lowest (pull_request) Successful in 46s
driver / build (pull_request) Successful in 43s
driver / security (pull_request) Successful in 1m27s
CI / driver (pull_request) Successful in 3m11s
driver / standard (pull_request) Successful in 20s
driver / lint (pull_request) Successful in 23s
driver / publish (pull_request) Skipped
What a host runs on its own schedule as one shape: cron lines, run-parts
scripts, anacron, systemd timers, periodic, a platform's own jobs, Windows
tasks. Every job says who owns its definition and what the driver can do
to it; every write names the digest the caller saw and is refused when the
host's definition changed since.

PosixScheduledJobsMixin reads everything in one framed report as root,
switches a cron line behind #napalm-disabled:, a timer by systemctl and a
run-parts script by its execute bit, runs a job now bounded by timeout, and
writes an application's job as a timer where systemd runs, else in
/etc/cron.d or the user's crontab. napalm_device_types.cron parses cron
expressions, gives their next runs and writes them as OnCalendar lines.

get_cron_jobs and CronJobDict go: nothing called them.

Version 4.26.0, so drivers can require it before the release is tagged.

Closes #86
2026-11-03 05:29:27 +01:00
christianmanivong fdf2f89115 Merge pull request 'chore(release): 4.25.0' (#84) from chore/release-4.25.0 into main
driver / types (push) Successful in 1m4s
driver / lint (push) Successful in 14s
driver / test (3.13) (push) Successful in 1m3s
driver / lowest (push) Successful in 34s
driver / build (push) Successful in 29s
driver / security (push) Successful in 1m2s
driver / publish (push) Successful in 29s
driver / standard (push) Successful in 17s
driver / test (3.12) (push) Successful in 1m10s
CI / driver (push) Successful in 3m6s
2026-11-02 20:01:39 +00:00
christianmanivong d63abff467 chore(release): 4.25.0
driver / types (pull_request) Successful in 1m9s
driver / test (3.12) (pull_request) Successful in 1m20s
driver / lowest (pull_request) Successful in 36s
driver / test (3.13) (pull_request) Successful in 1m6s
driver / build (pull_request) Successful in 33s
driver / security (pull_request) Successful in 1m8s
CI / driver (pull_request) Successful in 3m4s
driver / standard (pull_request) Successful in 23s
driver / lint (pull_request) Successful in 13s
driver / publish (pull_request) Skipped
2026-11-02 20:55:27 +01:00
christianmanivong 9d6a1a65ed Merge pull request 'feat(backup): BackupServerDriver, a role base for backup servers (#82)' (#83) from feat/backup-server-role into main
driver / lint (push) Successful in 15s
driver / types (push) Successful in 57s
driver / test (3.13) (push) Successful in 1m8s
driver / lowest (push) Successful in 42s
driver / build (push) Successful in 39s
driver / security (push) Successful in 1m27s
driver / publish (push) Skipped
driver / standard (push) Successful in 16s
driver / test (3.12) (push) Successful in 1m7s
CI / driver (push) Successful in 2m49s
2026-11-02 19:55:05 +00:00
christianmanivong 95e3aa1e26 feat(backup): BackupServerDriver, a role base for backup servers (#82)
driver / types (pull_request) Successful in 1m6s
driver / test (3.13) (pull_request) Successful in 1m7s
driver / test (3.12) (pull_request) Successful in 1m15s
driver / lowest (pull_request) Successful in 30s
driver / build (pull_request) Successful in 29s
driver / security (pull_request) Successful in 58s
CI / driver (pull_request) Successful in 2m22s
driver / lint (pull_request) Successful in 15s
driver / standard (pull_request) Successful in 16s
driver / publish (pull_request) Skipped
A backup server keeps datastores, one backup group per source with its
snapshots, and the jobs that keep them healthy: garbage collection,
verify, prune, sync and tape. None of that is a pool, a volume or a
replication job, so it is a role of its own instead of being bent into
StorageDriver. A server that manages its own disks lists StorageDriver
second.

The role declares the reads (datastores, groups with their newest
snapshot, a group's snapshots, jobs, remotes) and the actions (garbage
collection, running a job, verifying, protecting and deleting a
snapshot), which start a task and return its id for get_backup_task.
The shapes live in napalm_device_types.backups.

ClusterStorageDict gains an optional backup_server: which server,
datastore and namespace a hypervisor's backup storage writes to, with
the fingerprint it pins. Matching that to a server is the application's.
2026-11-02 20:46:40 +01:00
christianmanivong a0901e3277 Merge pull request 'chore(release): 4.24.0' (#81) from chore/release-4.24.0 into main
driver / lint (push) Successful in 17s
driver / types (push) Successful in 1m14s
driver / test (3.13) (push) Successful in 1m8s
driver / lowest (push) Successful in 36s
driver / build (push) Successful in 31s
driver / security (push) Successful in 1m16s
driver / publish (push) Successful in 32s
driver / standard (push) Successful in 19s
driver / test (3.12) (push) Successful in 1m15s
CI / driver (push) Successful in 4m45s
2026-11-02 09:45:41 +00:00
christianmanivong 060e210130 chore(release): 4.24.0
driver / standard (pull_request) Successful in 19s
driver / types (pull_request) Successful in 1m12s
driver / test (3.12) (pull_request) Successful in 1m16s
driver / lowest (pull_request) Successful in 35s
driver / test (3.13) (pull_request) Successful in 1m18s
driver / build (pull_request) Successful in 34s
driver / security (pull_request) Successful in 1m20s
CI / driver (pull_request) Successful in 3m17s
driver / lint (pull_request) Successful in 22s
driver / publish (pull_request) Skipped
2026-11-02 10:38:38 +01:00
christianmanivong 31973d2d5f Merge pull request 'feat(lldp): one lldpctl keyvalue parser and the LLDP projections' (#78) from feat/lldpctl-keyvalue-parser into main
driver / lint (push) Successful in 17s
driver / types (push) Successful in 1m24s
driver / test (3.13) (push) Successful in 1m13s
driver / lowest (push) Successful in 36s
driver / build (push) Successful in 39s
driver / security (push) Successful in 1m28s
driver / publish (push) Skipped
driver / standard (push) Successful in 18s
driver / test (3.12) (push) Successful in 1m26s
CI / driver (push) Successful in 3m30s
2026-11-02 09:38:06 +00:00
christianmanivong abda0973e7 Merge remote-tracking branch 'origin/main' into feat/lldpctl-keyvalue-parser
driver / types (pull_request) Successful in 1m7s
driver / test (3.12) (pull_request) Successful in 1m14s
driver / test (3.13) (pull_request) Successful in 1m12s
driver / lowest (pull_request) Successful in 38s
driver / build (pull_request) Successful in 35s
driver / security (pull_request) Successful in 1m13s
CI / driver (pull_request) Successful in 2m55s
driver / standard (pull_request) Successful in 18s
driver / lint (pull_request) Successful in 17s
driver / publish (pull_request) Skipped
2026-11-02 10:24:03 +01:00
christianmanivong 4eeaaef236 Merge pull request 'docs(structure): format the code examples as current ruff does' (#80) from fix/ruff-markdown-format into main
driver / lint (push) Successful in 17s
driver / types (push) Successful in 1m15s
driver / test (3.13) (push) Successful in 1m7s
driver / lowest (push) Successful in 44s
driver / build (push) Successful in 36s
driver / security (push) Successful in 1m22s
driver / publish (push) Skipped
driver / standard (push) Successful in 20s
driver / test (3.12) (push) Successful in 1m20s
CI / driver (push) Successful in 3m2s
2026-11-02 09:23:57 +00:00
christianmanivong 3490d8b62a docs(structure): format the code examples as current ruff does
driver / standard (pull_request) Successful in 21s
driver / types (pull_request) Successful in 1m5s
driver / test (3.12) (pull_request) Successful in 1m14s
driver / test (3.13) (pull_request) Successful in 1m7s
driver / lowest (pull_request) Successful in 34s
driver / build (pull_request) Successful in 32s
driver / security (pull_request) Successful in 1m29s
CI / driver (pull_request) Successful in 3m21s
driver / lint (pull_request) Successful in 17s
driver / publish (pull_request) Skipped
ruff 0.17 formats Python blocks in Markdown too, and CI installs ruff
unpinned, so the lint job on main failed on docs/STRUCTURE.md.

Closes #79
2026-11-02 09:18:42 +01:00
christianmanivong 2542081c6f feat(lldp): report the neighbour's chassis MAC in lldp_neighbors
driver / standard (pull_request) Successful in 20s
driver / types (pull_request) Successful in 1m13s
driver / test (3.12) (pull_request) Successful in 1m19s
driver / test (3.13) (pull_request) Successful in 1m12s
driver / lowest (pull_request) Successful in 38s
driver / build (pull_request) Successful in 31s
driver / security (pull_request) Successful in 1m13s
CI / driver (pull_request) Failing after 3m1s
driver / lint (pull_request) Failing after 15s
driver / publish (pull_request) Skipped
An application matches an LLDP neighbour by its chassis MAC first, and
napalm-linux and napalm-netgear report it as mac; the projection did
not, so every driver built on it would have reported neighbours by name
only (NAPALM/napalm-openwrt#31). lldp_neighbors now carries mac, lower
case, when the chassis id is a MAC address (LldpNeighborDict).
2026-11-02 08:50:50 +01:00
christianmanivong 4427be35cc feat(lldp): parse lldpctl keyvalue into one row per neighbour
driver / types (pull_request) Successful in 1m8s
driver / test (3.12) (pull_request) Successful in 1m18s
driver / test (3.13) (pull_request) Successful in 1m3s
driver / lowest (pull_request) Successful in 50s
driver / build (pull_request) Successful in 37s
driver / security (pull_request) Successful in 1m18s
CI / driver (pull_request) Failing after 3m7s
driver / standard (pull_request) Successful in 16s
driver / lint (pull_request) Failing after 16s
driver / publish (pull_request) Skipped
napalm-linux, napalm-openwrt and napalm-proxmox each parsed lldpd's
neighbours on their own, and the copies grouped lines by interface (two
neighbours on one port merged), cut the interface name at its first dot
and looked for chassis.id.value, a key lldpd never prints. The parse
lives here once now: neighbours are split at their via line, the
interface name runs up to the neighbour's first field, ids are read
under the type lldpd prints them with, and a failed lldpctl raises.

The rows project into NAPALM's two LLDP shapes with lldp_neighbors and
lldp_neighbors_detail, the projection part of #10.

Refs #10
2026-11-02 08:42:10 +01:00
christianmanivong ad9ec0fb4d Merge pull request 'docs: a structure that keeps device-types vendor- and product-agnostic' (#37) from docs/structure-plan into main
driver / lint (push) Failing after 13s
driver / types (push) Successful in 1m5s
driver / test (3.13) (push) Successful in 1m2s
driver / lowest (push) Successful in 36s
driver / build (push) Successful in 42s
driver / security (push) Successful in 1m19s
driver / publish (push) Skipped
driver / standard (push) Successful in 14s
driver / test (3.12) (push) Successful in 1m0s
CI / driver (push) Failing after 2m48s
Reviewed-on: #37
2026-11-02 07:19:02 +00:00
christianmanivong 8dd7e8ab9a Merge branch 'main' into docs/structure-plan
driver / types (pull_request) Successful in 1m0s
driver / test (3.12) (pull_request) Successful in 58s
driver / test (3.13) (pull_request) Successful in 58s
driver / lowest (pull_request) Successful in 30s
driver / build (pull_request) Successful in 27s
driver / security (pull_request) Successful in 58s
CI / driver (pull_request) Failing after 2m22s
driver / standard (pull_request) Successful in 14s
driver / lint (pull_request) Failing after 13s
driver / publish (pull_request) Skipped
2026-11-02 07:18:47 +00:00
christianmanivong 935fad2a2d Merge pull request 'fix(updates): read dnf's security advisories from the cached metadata (#76)' (#77) from fix/dnf-security-cache-only into main
driver / lint (push) Successful in 14s
driver / types (push) Successful in 1m1s
driver / test (3.13) (push) Successful in 55s
driver / lowest (push) Successful in 31s
driver / build (push) Successful in 31s
driver / security (push) Successful in 1m3s
driver / publish (push) Skipped
driver / standard (push) Failing after 2s
driver / test (3.12) (push) Successful in 1m1s
CI / driver (push) Failing after 2m6s
Reviewed-on: #77
2026-11-02 07:18:33 +00:00
christianmanivong f337ae7fb8 fix(updates): read dnf's security advisories from the cached metadata
driver / types (pull_request) Successful in 55s
driver / test (3.12) (pull_request) Successful in 1m2s
driver / lowest (pull_request) Successful in 32s
driver / test (3.13) (pull_request) Successful in 1m4s
driver / build (pull_request) Successful in 25s
driver / security (pull_request) Successful in 57s
CI / driver (pull_request) Successful in 2m27s
driver / standard (pull_request) Successful in 15s
driver / lint (pull_request) Successful in 13s
driver / publish (pull_request) Skipped
DNF_SECURITY_COMMAND ran `dnf updateinfo list --security`, which downloads
the repositories' metadata first whenever the cached copy has expired. On an
Oracle Linux host on OCI that took longer than netOrk's whole 120 s poll.
It now runs with --cacheonly, as APT_UPGRADABLE_COMMAND reads apt's local
index; refreshing stays `dnf makecache`. Without a cache dnf fails at once,
and the __DNF_RC marker already turns that into "security unknown".

Closes #76
2026-11-02 08:07:48 +01:00
christianmanivong c7ea15dbb4 docs: what the drivers need from the SNMP profile, opt-outs, and how getters behave
CI / test (3.10) (push) Successful in 26s
CI / test (3.11) (push) Successful in 24s
CI / test (3.12) (push) Successful in 26s
CI / test (3.13) (push) Successful in 25s
CI / test (3.10) (pull_request) Successful in 25s
CI / test (3.12) (pull_request) Successful in 26s
CI / test (3.13) (pull_request) Successful in 25s
CI / test (3.11) (pull_request) Successful in 24s
The SNMP profile has to express fallback OIDs, tables folded by first,
average or maximum, numbers inside text values, memory as used, free or
percent, and the uptime source. TX errors counting drops becomes
OPNsense's entry instead of every firewall's.

A driver opts out of an inherited template with None, so callers ask
callable(). USES_SSH, FULL_UPGRADE and platform stay as driver facts.
Getters do not write, a failed read raises, a write reports its real
result, and the application uses public driver methods only.
2026-10-14 10:56:17 +02:00
christianmanivong 01fe680456 docs: a structure that keeps device-types vendor- and product-agnostic
CI / test (3.10) (push) Successful in 28s
CI / test (3.11) (push) Successful in 26s
CI / test (3.12) (push) Successful in 27s
CI / test (3.13) (push) Successful in 25s
CI / test (3.10) (pull_request) Successful in 25s
CI / test (3.12) (pull_request) Successful in 25s
CI / test (3.13) (pull_request) Successful in 24s
CI / test (3.11) (pull_request) Successful in 24s
Roles and capabilities declare, templates run declared steps, reconcilers
stay free of I/O. POSIX host readers, the SSH exec transport and the
privilege layer move to a new package, napalm-unix. SNMP polling, ping
sweeps, discovery scoring, guest profiles, display labels and warnings
belong to the application; drivers declare SNMP OIDs as data and report
facts. Fixes become templates whose values come from the caller.

Says in which order this happens and where the open proposals #8-#14 and
#22 go instead.
2026-10-14 10:20:55 +02:00
33 changed files with 4295 additions and 50 deletions

No files matched your search

+73
View File
@@ -7,6 +7,79 @@ recorded here; their history is the git log.
## [Unreleased]
## [4.27.0] - 2026-11-03
### Added
- `ScheduledJobsMixin`: what a host runs on its own schedule as one shape,
`ScheduledJobDict` — cron lines, run-parts scripts, anacron, systemd timers, `periodic`,
a platform's own jobs, Windows tasks — with who owns each definition (`managed_by`) and
what the driver can do to it (`actions`). `get_scheduled_jobs` reads them with the host's
time zone; `manage_scheduled_job` enables, disables or runs one now; `put_scheduled_job`
and `delete_scheduled_job` create, change and delete a job an application writes. Every
write names the `digest` the caller saw (`job_digest`) and raises `ScheduledJobChanged`
when the host's definition differs; `ScheduledJobRefused` for a job that cannot take a
verb (NetOrk/netork#1933, #86).
- `PosixScheduledJobsMixin`: the implementation over the command channel. One framed report
as root reads `/etc/crontab`, `/etc/cron.d`, the user crontabs, the run-parts directories,
`/etc/anacrontab` and every systemd timer with the unit it starts, and which package owns
each definition. A cron line is switched off behind `#napalm-disabled: `; an
application's job is a timer where systemd runs, else a file in `/etc/cron.d` or marked
lines in the user's crontab.
- `napalm_device_types.cron`: cron expressions as Vixie cron reads them, their next runs in
a time zone, and as systemd `OnCalendar` lines that fire on exactly the same minutes.
### Removed
- `OSDriver.get_cron_jobs` and `CronJobDict`. No application called them; the readers in
napalm-linux and napalm-bsd go with them. `get_scheduled_jobs` replaces them.
## [4.26.0] - 2026-11-03
### Added
- `OpenSshServerMixin.SSHD_CONFIG_GENERATED`: a driver whose platform writes sshd_config
from its own database, but includes `/etc/ssh/sshd_config.d` (OpenMediaVault), sets it
and keeps `harden_ssh_server`. The hardening then goes only into the drop-in; a
sshd_config without that include is left alone and the result says why, since the
platform would write over a change to it on its next apply (#88, NetOrk/netork#1950).
## [4.25.0] - 2026-11-02
### Added
- `BackupServerDriver` (`ROLE = "backup_server"`), a role base for backup servers such as
Proxmox Backup Server (#82). It declares `get_backup_datastores`, `get_backup_groups`
(each group with its newest snapshot), `get_backup_snapshots`, `get_backup_jobs`
(garbage collection, verify, prune, sync and tape in one list) and `get_backup_remotes`,
and the actions `start_garbage_collection`, `run_backup_job`, `verify_backup_snapshot`,
`set_backup_snapshot_protected` and `delete_backup_snapshot`, which return a task to
follow with `get_backup_task`. The shapes live in `napalm_device_types.backups`. A
server that manages its own disks lists `StorageDriver` as its second role.
- `ClusterStorageDict.backup_server` (optional): which backup server, datastore and
namespace a hypervisor's backup storage writes to, with the certificate fingerprint it
pins, so an application can tie a server's backup groups to a cluster's guests.
## [4.24.0] - 2026-11-02
### Added
- `parse_lldpctl_keyvalue(output)` and `LLDPCTL_KEYVALUE_COMMAND`: lldpd's neighbours
(`lldpctl -f keyvalue`) as one `LldpNeighborRow` per neighbour, for every driver of a
host that runs lldpd. Several neighbours on one port (a phone and the PC behind it, an
unmanaged switch) stay apart, an interface name with dots (`eth0.1`) stays whole, the
chassis and port ids are read whatever their type (MAC, address, interface name, local
id), with management addresses and capabilities. A failed lldpctl (lldpd not running,
lldpctl missing) raises instead of reading as no neighbours.
- `lldp_neighbors(rows)` and `lldp_neighbors_detail(rows, interface="")`: NAPALM's
`get_lldp_neighbors` and `get_lldp_neighbors_detail` from those rows, so a driver no
longer builds the two shapes itself. A neighbour that names its chassis by a MAC
address carries it in `get_lldp_neighbors` as `mac`, in lower case
(`LldpNeighborDict`), so an application can match it by MAC.
### Fixed
- `DNF_SECURITY_COMMAND` reads the cached metadata only (`dnf --cacheonly`), as
`APT_UPGRADABLE_COMMAND` reads apt's local index (#76). With an expired cache dnf
downloaded the repositories' metadata first, and on an Oracle Linux host on OCI that
outlasted netOrk's whole poll budget. Refreshing the index stays a step of its own
(`dnf makecache`). Without a cache dnf fails at once, which already reads as
"security unknown".
## [4.23.0] - 2026-11-02
### Changed
+37 -4
View File
@@ -47,6 +47,18 @@ UUID and NIC MACs -- what ties the BMC to the host it manages. Its sensors come
NAPALM's `get_environment`, with `cpu` and `memory` empty, since a BMC does not know the
host OS's load; it has no `HealthMetricsMixin` (SNMP) and no command channel.
A backup server (Proxmox Backup Server) is a role of its own too, `BackupServerDriver`
(`backup_server`). Its datastores, the backup groups in them (each with its newest
snapshot), its jobs (garbage collection, verify, prune, sync, tape) and its remotes have
no counterpart in `StorageDriver`, so they are not bent into pools, volumes and
replication jobs. A server that also manages its own disks and pools lists
`StorageDriver` second: `class ProxmoxBackupDriver(BackupServerDriver, StorageDriver, ...)`.
Its actions (garbage collection, running a job, verifying, protecting or deleting a
snapshot) start a task and return its id at once; `get_backup_task` follows it. On the
hypervisor side, a cluster storage that writes to a backup server says which one
(`ClusterStorageDict.backup_server`: server, port, datastore, namespace, fingerprint), so
an application can tie the server's groups to the cluster's guests.
### Role bases declare; they never implement
**A role base must not contain a single runtime method — not even a
@@ -179,6 +191,16 @@ without `ss` is read with `netstat -lntup` (OpenWrt's busybox, old net-tools); o
cgroup names the procd service (`/services/<name>/<instance>`). A host with neither raises
`ListeningSocketsUnavailable`.
LLDP neighbours are rows: `LldpNeighborRow` is one neighbour as its local port heard it
(chassis and port id with their type, name, description, management addresses,
capabilities in NAPALM's names), and `lldp_neighbors(rows)` / `lldp_neighbors_detail(rows,
interface)` are NAPALM's two getters projected from them, a port with two neighbours
keeping both; a neighbour named by its chassis MAC carries it as `mac` (lower case). A driver for a host that runs lldpd sends `LLDPCTL_KEYVALUE_COMMAND` and
hands its output to `parse_lldpctl_keyvalue`: one row per neighbour block, an interface
name with dots (`eth0.1`) kept whole, ids read under the type lldpd prints them with
(`chassis.mac`, `port.local`, ...). When lldpctl fails (lldpd not running, no `lldpctl`)
the parse raises instead of reading as no neighbours.
`HostFirewallMixin` (`get_host_firewall`) reads a Linux host's own netfilter rules the same
way: `nft -j list ruleset`, with the tables iptables-nft manages taken from `iptables-save`
instead, because nft shows their extensions (`-m conntrack`, `-m set`) only as an opaque `xt`.
@@ -279,6 +301,20 @@ trip (`list-unit-files` plus one `systemctl show` over every loaded unit) instea
`is-enabled` and a `show` per unit. A host without systemd raises `SystemdUnavailable`,
a `NotImplementedError`, so a driver can fall back to another init system.
`ScheduledJobsMixin` (`get_scheduled_jobs`, `manage_scheduled_job`, `put_scheduled_job`,
`delete_scheduled_job`) is what a host runs on its own schedule — cron lines, run-parts
scripts, anacron, systemd timers, `periodic`, a platform's own jobs, Windows tasks — as one
shape, `ScheduledJobDict`. Every job says who owns its definition (`managed_by`: package,
admin, platform, application) and what the driver can do to it (`actions`): every job a
driver can switch takes `enable`, `disable` and `run`; only a job an application wrote
through `put_scheduled_job` takes `edit` and `delete`. Every write names the `digest` the
caller saw and raises `ScheduledJobChanged` when the host's definition is no longer that.
`PosixScheduledJobsMixin` implements it over `run_command`: one framed report as root,
a cron line switched off behind `#napalm-disabled: `, a timer by `systemctl`, an
application's job as a timer where systemd runs, else in `/etc/cron.d` or the user's
crontab. A driver sets the paths its host keeps jobs in. `napalm_device_types.cron` parses
cron expressions, gives their next runs and writes them as `OnCalendar` lines.
### Access channels: why there is no container model here
`CommandChannelMixin` declares a driver's public channel to its host:
@@ -451,6 +487,7 @@ Requires Python ≥ 3.12 (tested up to 3.13) and NAPALM ≥ 4.0.
| `PhoneDriver` | IP telephony endpoints | Yealink |
| `MediaDriver` | Speakers, streamers, media renderers | Sonos, Chromecast |
| `ManagementControllerDriver` | Server baseboard management controllers | Supermicro BMC, Dell iDRAC, HPE iLO, Lenovo XCC |
| `BackupServerDriver` | Backup servers: datastores, backup groups, GC/verify/prune/sync/tape jobs | Proxmox Backup Server |
Mixins that come with the classes above: `PingSweepMixin` (subnet sweeps, in
every driver through `DeviceTypeDriver`) and `DhcpServerMixin` (static DHCP
@@ -598,10 +635,6 @@ class LinuxDriver(OSDriver):
def get_processes(self):
# return List[ProcessDict]
...
def get_cron_jobs(self):
# return List[CronJobDict]
...
```
### Storage / NAS
+244
View File
@@ -0,0 +1,244 @@
# Structure
Status: agreed plan. Nothing here is implemented yet; the migration at the end says in
which order it will be.
## Principle
**napalm-device-types is vendor-, OS-, protocol- and product-agnostic.** It holds the
contracts every driver implements and the generic logic that works unchanged for every
driver of a role. It knows no shell, no paramiko, no package manager, no container CLI,
no SNMP OID, no cloud-init and no application that uses the drivers.
That rules out three kinds of code that are here today:
- **How a kind of host is read.** Shell commands and their parsers for POSIX hosts move to
a new package, **napalm-unix**, which napalm-linux and napalm-bsd depend on.
- **What an application does with a device.** Polling a device over SNMP, sweeping a range
with pings, scoring discovery evidence, choosing what a provisioned guest installs, and
deciding what counts as a problem belong to the application, not to the drivers.
- **Product naming.** Files, markers and messages a driver leaves on a device carry the
driver's name or none, never the name of the application that called it.
```
napalm-device-types roles, capabilities, templates, models, reconcilers <- every driver
^
napalm-unix POSIX host readers, SSH exec transport, privilege <- napalm-linux, napalm-bsd
^
napalm-linux, napalm-bsd connection, platform getters, platform steps
application SNMP polling, ping sweeps, discovery scoring, guest
profiles, warnings, and the values every fix is given
```
## What stays in napalm-device-types
| Kind | Rule | Modules |
|---|---|---|
| **Role** | Declares only, under `TYPE_CHECKING`. No runtime method. | `access_point`, `switch`, `firewall`, `hypervisor`, `os`, `storage`, `phone`, `media`, `residential_gateway`, `management_controller`, `backup_server` |
| **Capability** | Declares only. | `packages`, `services`, `updates`, `nat_vpn`, `mac_acl`, `config_lifecycle`, `interface_filter`, `host_reboot` |
| **Template** | A concrete method that runs a fixed sequence of abstract steps. A driver implements the steps, or overrides the whole method when its device does it in one go. All values come from the caller. | the fix routines below |
| **Contract types** | Interfaces and data shapes other packages implement or fill. | `CommandChannelMixin`, `CommandResult`, `ByteStream`; the declaration of `ContainerEngineMixin`; the reader mixins' public method declarations; the SNMP profile type; `models` |
| **Reconciler** | Concrete and vendor-neutral, without I/O of its own. | `firewall_rules` and `dhcp` diff/apply, `_matching`, `lag` |
| **Introspection** | Concrete, without I/O. | `roles`; `base` (`FingerprintRule`, `PortSpec`, the fingerprint attributes) |
| **Driver facts** | Class attributes that describe the device or how it is reached, not what an application does with it. | `USES_SSH`, `FULL_UPGRADE`, `platform`, `SNMP_HEALTH_PROFILE` |
### Fix routines are templates
A change the application wants made on a device, for example "run an SNMP agent that
answers us", is one template per purpose. The template fixes the order of the steps; the
driver knows how each step is done on its device; the caller supplies every value.
```python
class SnmpAgentMixin:
def configure_snmp_agent(self, settings: SnmpAgentSettingsDict) -> FixResultDict:
steps = [
self._install_snmp_agent,
self._write_snmp_agent_config,
self._enable_snmp_agent,
self._verify_snmp_agent,
]
... # runs them in order, stops at the first failure
# declared only; napalm-unix implements them for net-snmp
def _install_snmp_agent(self) -> FixStepDict: ...
def _write_snmp_agent_config(self, settings: SnmpAgentSettingsDict) -> FixStepDict: ...
def _enable_snmp_agent(self) -> FixStepDict: ...
def _verify_snmp_agent(self, settings: SnmpAgentSettingsDict) -> FixStepDict: ...
```
napalm-unix implements the four steps for net-snmp, for Linux and the BSDs alike. A REST
driver such as OPNsense overrides `configure_snmp_agent` as a whole, because its device
takes the whole configuration in one call. The community, the allowed sources, contact
and location are the caller's.
The same shape replaces every action in today's `run_device_action`:
| Today | Template |
|---|---|
| `fix_snmp` | `configure_snmp_agent(settings)` |
| `fix_apt_proxy` | `configure_package_proxy(url)` |
| `fix_docker_permissions` | `grant_container_engine_access(user)` |
| `install_lldpd`, `install_auc` | `ensure_package(name)` |
The mixin is mixed in by the driver, or by the platform package, not by a role base. A
driver that inherits a template it cannot run on its device, as a NAS driver built on the
Linux driver inherits the package proxy, opts out by setting the method to `None`. So a
caller asks `callable(getattr(driver, "configure_snmp_agent", None))`, never `hasattr`.
The caller verifies the result from its own side where it can; the SNMP agent is asked
by the application's SNMP client, not by a probe the driver hand-builds.
### SNMP: drivers declare data, the application polls
The drivers do not collect over SNMP. A driver whose device answers the standard UCD-MIB
and IF-MIB declares nothing. A driver whose device publishes the same values under
proprietary OIDs declares them as data, in a class attribute of a type defined here:
```python
class ProcurveDriver(SwitchDriver):
SNMP_HEALTH_PROFILE: ClassVar[SnmpHealthProfile] = {
"uptime": {"oids": ["1.3.6.1.4.1.11.2.14.11.1.2.4.1.5.1"], "format": "ticks"},
"cpu": {"oids": ["1.3.6.1.4.1.11.2.14.11.5.1.9.6.1.0", "1.3.6.1.4.1.11.2.3.7.8.6.1.0"]},
"memory": {"total": "...", "free": "...", "unit": "bytes"},
}
```
One OID per value is not enough for the drivers there are. The profile type has to say:
- **Fallbacks:** an ordered list of OIDs; the first that answers wins (ProCurve's CPU).
- **Tables:** an OID to walk and how to fold its rows: first, average or maximum (TP-Link
reports CPU and memory per stack unit).
- **Text values:** a pattern that takes the number out of a string (Netgear reports CPU
as `"5 Secs ( 3.2%)"`).
- **Memory:** total with used or free, or a percentage only, and the unit.
- **Uptime:** which OID, standard ones included (`sysUpTime` where a device has no
`hrSystemUptime`), and its format.
- **Interfaces:** a pattern of names to skip, and whether the TX-error counter counts
drops. Today the second is `True` on every firewall, because OPNsense counts drops
there; it becomes OPNsense's profile entry.
Polling, the standard OIDs, the counter arithmetic (64-bit counters first, 32-bit as the
fallback) and the defaults are the application's.
### Warnings are facts
A driver reports facts: the proxy a host is configured with, a DSL line's SNR, the
firmware version it runs and the one it is offered. It does not compare them with a
setting of the application or with a threshold, and it does not name a warning.
The same goes for lookups a device does not make itself: what firmware a vendor
currently publishes is the application's catalogue, not a getter.
### Getters read, and say when they cannot
- A getter changes nothing on the device. It does not start a daemon, rewrite its
configuration or install a package on the way to an answer.
- A read that fails raises. An empty list means "there is none", never "could not ask";
a sync that deletes and re-adds on top of an empty answer would otherwise duplicate
everything.
- A write reports failure from the device's answer or exit status, never `success: True`
by default.
### The application uses public methods only
The application never calls a driver's private methods (`_get`, `_post`,
`_send_command`, `_action_*`). What it needs from a device becomes a declared method of
the driver.
## What moves to napalm-unix
| Module | Knows |
|---|---|
| `host_status` | apt, dnf, dpkg, systemd timers, `df` |
| `kernel` | `/proc`, `/lib/modules`, gzip, base64 |
| `listening` | `ss`, `netstat`, `/proc/<pid>/cgroup` |
| `systemd` | `systemctl` |
| `virtual_ips` | vip-manager, keepalived, awk |
| `system_identity` | SMBIOS via sysfs and `dmidecode` |
| `package_updates` | apt and dnf output |
| `ssh_server` | OpenSSH (`sshd -T`, `sshd -t`, `sshd_config`); which algorithms to drop stays the caller's |
| `terminal` | ANSI escape codes in shell output |
| `channel`: `run_on_transport`, `open_stream_on_transport`, `ParamikoExecStream` | paramiko exec channels |
| `container_engine`: `ContainerEngineConnection`, engine probing | the Docker CLI, `system dial-stdio` |
It also takes what napalm-linux and napalm-bsd each carry today:
- **Privilege:** one `sudo -S` / `sudo -n` / root layer, including the rule that a
privileged command never gets stdin while a sudo password is set, and one way to write a
file as root (stage with `mktemp`, then `install`). Two copies of this have already
drifted apart.
- **Parsers:** `/etc/passwd` with `/etc/group`, crontab lines, `ps` with `lstart`.
- **Fix steps:** net-snmp, the package proxy, container engine group membership.
The FreeBSD part of `host_status` (`freebsd-version -k` against `-r`) moves to napalm-bsd.
In napalm-unix every reader follows one pattern:
- **Shape:** `FOO_COMMAND`, `parse_foo(output)`, and `FooMixin.get_foo()`.
- **Reaching the host:** the mixin uses `run_command(command, privileged=...)` from
`CommandChannelMixin`. The six per-reader `_run_*` hooks, with four different signatures,
go away.
- **Framing:** one frame helper writes and parses the `BEGIN`/`END` markers for every reader.
- **Errors:** one exception for "cannot read" and one for "not on this host". A read that
fails or is cut short raises; it never reads as "nothing there".
- **Names on the host:** temporary files, markers and drop-ins carry `napalm` or the
driver's name. A drop-in written under an older name is still recognised and replaced.
Layout: `napalm_unix/{posix,linux,freebsd,openssh,transport}/`.
## What moves to the application
| Today | Why it is not a driver's |
|---|---|
| `health_metrics`, `_ucd_metrics` | Polling over SNMP is something the application does to a device; the driver is not even connected while it runs. |
| `ping_sweep` | Looping over addresses, capping the targets and reporting progress is a scan. A device with a faster batch keeps it as a capability of its own. |
| `rules_max_weight` | Scoring is how one application ranks drivers. The rules themselves stay with the driver. |
| `provisioning` (`QEMU_GUEST_AGENTS`, cloud-init network-config, decompressors) | Which agent a guest gets and how its network is configured is a guest profile. The hypervisor driver is handed the packages and commands. |
| `TYPE_LABEL`, `REBOOT_SETTLE_SECONDS` | A display name and a polling delay. |
| firmware catalogues (napalm-netgear's `netgear_plus`) | What a vendor publishes is not something the device reports. |
| `run_device_action` | Replaced by the templates above. |
`OSDriver.ROLE` stops defaulting to `"linux"`. Each OS driver states its own role.
## Rules, enforced by tests in this package
1. Roles and capabilities own no runtime method; a template runs only declared steps.
2. No module imports `paramiko`, `socket`, `subprocess` or `asyncio`, and none defines a
`*_COMMAND`.
3. No module, string or default names an application.
4. Role methods return TypedDicts, not `Any`.
Each rule is added with the step that makes it pass.
## Migration
This package cannot keep re-exports pointing at napalm-unix: the contract package would
then depend on its implementation. So a moved name is first deprecated here and then
removed, and nothing breaks in between.
| Step | Change |
|---|---|
| 1 | The application moves to 4.9 and the drivers that need it; napalm-bsd stops passing stdin to a privileged command while a sudo password is set. |
| 2 | This document; the application's architecture docs and this README follow it. |
| 3 | `SnmpHealthProfile`. The application polls on its own; the four switch and phone drivers declare their OIDs as data. `HealthMetricsMixin` is deprecated. |
| 4 | The fix templates. napalm-linux and napalm-bsd implement the steps; the application calls the templates with its values. `run_device_action` is deprecated. The other drivers follow, one issue each. |
| 5 | Ping sweep, scoring, provisioning, `TYPE_LABEL` and `REBOOT_SETTLE_SECONDS` move to the application and are deprecated here. Drivers report facts instead of warnings. |
| 6 | napalm-unix, with the moved code and its tests, the unified hooks and the frame helper. napalm-linux and napalm-bsd import from it; the moved names are deprecated here. |
| 7 | 5.0 removes everything deprecated. It also splits `models.py` by area, types the remaining `Dict[str, Any]` role returns, declares what drivers already offer undeclared (`get_networks`, `search_packages`, `first_boot_pending`, the base-system update entry), and makes `install_package` declare the result drivers return. |
| 8 | The README shrinks to the concepts; the reader details move with the readers. |
## Proposals this replaces
Several open issues proposed adding shared driver code here. Under this structure:
| Issue | Goes to |
|---|---|
| #8 netmiko session | not here: a CLI-driver helper package, or left in the drivers |
| #9 config mode for CLI switches | not here: a CLI-driver helper package, or left in the drivers |
| #10 LLDP rows | the projection of rows into NAPALM's two shapes stays here; the `lldpctl` parser goes to napalm-unix |
| #11 CLI text helpers | not here; the natural port sort, which `lag` already has, may become public |
| #12 ping | the reply builder stays here; the BusyBox and iputils parser goes to napalm-unix |
| #13 Linux userland | napalm-unix; the snmpd part becomes the SNMP template's steps |
| #14 `run_device_action` | replaced by the fix templates |
| #22 hooks onto `run_command` | step 6 |
+66
View File
@@ -22,6 +22,7 @@ driver do X".
Role bases -- what a device *is*:
* :class:`~napalm_device_types.access_point.AccessPointDriver`
* :class:`~napalm_device_types.backup_server.BackupServerDriver`
* :class:`~napalm_device_types.switch.SwitchDriver`
* :class:`~napalm_device_types.firewall.FirewallDriver`
* :class:`~napalm_device_types.hypervisor.HypervisorDriver`
@@ -57,6 +58,8 @@ instead of being restated on every role that happens to need it:
* :class:`~napalm_device_types.service_clusters.ServiceClustersMixin`
* :class:`~napalm_device_types.service_packages.ServicePackagesMixin`
* :class:`~napalm_device_types.sudo_history.SudoHistoryMixin`
* :class:`~napalm_device_types.scheduled_jobs.ScheduledJobsMixin`
* :class:`~napalm_device_types.scheduled_jobs_posix.PosixScheduledJobsMixin`
* :class:`~napalm_device_types.services.ServiceControlMixin`
* :class:`~napalm_device_types.systemd.SystemdServicesMixin`
* :class:`~napalm_device_types.updates.UpdateMixin`
@@ -78,6 +81,22 @@ from napalm_device_types.apt_proxy import (
same_proxy_url,
)
from napalm_device_types.arp import ARP_AGE_UNKNOWN
from napalm_device_types.backup_server import BackupServerDriver
from napalm_device_types.backups import (
BACKUP_JOB_KINDS,
BackupDatastoreDict,
BackupGroupDict,
BackupJobKind,
BackupRemoteDict,
BackupRunState,
BackupServerJobDict,
BackupServerRefDict,
BackupSnapshotDict,
BackupSnapshotRefDict,
BackupTaskDict,
BackupTaskStatusDict,
BackupVerifyState,
)
from napalm_device_types.base import DeviceTypeDriver, FingerprintRule, PortSpec, rules_max_weight
from napalm_device_types.channel import (
ByteStream,
@@ -135,6 +154,13 @@ from napalm_device_types.listening import (
ListeningSocketsUnavailable,
parse_listening_sockets,
)
from napalm_device_types.lldp import (
LldpNeighborDict,
LldpNeighborRow,
lldp_neighbors,
lldp_neighbors_detail,
)
from napalm_device_types.lldpctl import LLDPCTL_KEYVALUE_COMMAND, parse_lldpctl_keyvalue
from napalm_device_types.local_accounts import (
LocalAccountChangeDict,
LocalAccountDict,
@@ -181,6 +207,17 @@ from napalm_device_types.ping_sweep import PingSweepMixin, driver_supports_ping
from napalm_device_types.provisioning import VMProvisionIncomplete
from napalm_device_types.residential_gateway import ResidentialGatewayDriver
from napalm_device_types.roles import primary_role_of, role_keys_of, roles_of
from napalm_device_types.scheduled_jobs import (
ScheduledJobChanged,
ScheduledJobDict,
ScheduledJobRefused,
ScheduledJobResultDict,
ScheduledJobsMixin,
ScheduledJobSpecDict,
ScheduledJobsReadingDict,
job_digest,
)
from napalm_device_types.scheduled_jobs_posix import PosixScheduledJobsMixin
from napalm_device_types.service_clusters import (
SERVICE_CLUSTERS_COMMAND,
ServiceClustersMixin,
@@ -249,6 +286,19 @@ from napalm_device_types.vm_migration import MigrationPreflightDict
from napalm_device_types.vm_stop_request import VmStopRequestMixin
__all__ = [
"BACKUP_JOB_KINDS",
"BackupDatastoreDict",
"BackupGroupDict",
"BackupJobKind",
"BackupRemoteDict",
"BackupRunState",
"BackupServerJobDict",
"BackupServerRefDict",
"BackupSnapshotDict",
"BackupSnapshotRefDict",
"BackupTaskDict",
"BackupTaskStatusDict",
"BackupVerifyState",
"BackupDict",
"BackupJobDict",
"ClusterDict",
@@ -288,6 +338,7 @@ __all__ = [
"SnmpMemorySpec",
"SnmpValueSpec",
"AccessPointDriver",
"BackupServerDriver",
"ByteStream",
"CommandChannelMixin",
"CommandResult",
@@ -348,6 +399,12 @@ __all__ = [
"ListeningSocketsMixin",
"ListeningSocketsUnavailable",
"parse_listening_sockets",
"LLDPCTL_KEYVALUE_COMMAND",
"LldpNeighborDict",
"LldpNeighborRow",
"lldp_neighbors",
"lldp_neighbors_detail",
"parse_lldpctl_keyvalue",
"LocalAccountsMixin",
"LocalAccountDict",
"LocalAccountChangeDict",
@@ -356,6 +413,15 @@ __all__ = [
"PortSpec",
"ResidentialGatewayDriver",
"ServiceControlMixin",
"PosixScheduledJobsMixin",
"ScheduledJobChanged",
"ScheduledJobDict",
"ScheduledJobRefused",
"ScheduledJobResultDict",
"ScheduledJobSpecDict",
"ScheduledJobsMixin",
"ScheduledJobsReadingDict",
"job_digest",
"StorageDriver",
"SwitchDriver",
"SYSTEMD_SERVICES_COMMAND",
+222
View File
@@ -0,0 +1,222 @@
"""Crontabs and anacrontabs as text: the jobs in them, and changing one line of them.
Pure: no I/O. :mod:`napalm_device_types.scheduled_jobs_posix` reads the files and
writes them back; this module says what is in them and what the new text is.
**A line's key survives switching it off.** A job switched off is the same line
behind the marker :data:`DISABLED_MARKER`; the key is built from the line without
it, so the job keeps its key and the marker is all that changes. Two equal lines
in one file get ``:2``, ``:3`` after the first.
**A job an application wrote** is the line after ``# napalm-scheduled-job: <name>``
and is keyed ``job:<name>``, whatever its schedule.
**Everything else is left as it was**: comments, blank lines, variables, lines
cron would not take.
"""
from __future__ import annotations
import hashlib
import re
from dataclasses import dataclass, field
from shlex import quote
DISABLED_MARKER = "#napalm-disabled: "
JOB_MARKER = "# napalm-scheduled-job: "
_VARIABLE = re.compile(r"^([A-Za-z_][A-Za-z0-9_]*)\s*=\s*(.*)$")
_ANACRON_PERIOD = re.compile(r"^(\d+|@(daily|weekly|monthly|yearly|annually))$")
@dataclass(frozen=True)
class CronLine:
"""One job in a crontab: where it is, what it says, and what was set before it."""
index: int # the line's number in the text, from 0
key: str
schedule: str
run_as: str
command: str
enabled: bool
env: dict[str, str] = field(default_factory=dict)
name: str = "" # the name an application gave it
description: str = ""
def _unquote(value: str) -> str:
value = value.strip()
if len(value) >= 2 and value[0] == value[-1] and value[0] in "'\"":
return value[1:-1]
return value
def _digest(text: str) -> str:
return hashlib.sha256(" ".join(text.split()).encode()).hexdigest()[:8]
def _split(content: str, *, system: bool, user: str) -> tuple[str, str, str] | None:
"""(schedule, run_as, command) of a cron line; None for one cron would not take."""
fields = 1 if content.startswith("@") else 5
owner = 1 if system else 0
parts = content.split(None, fields + owner)
if len(parts) < fields + owner + 1:
return None
schedule = " ".join(parts[:fields])
run_as = parts[fields] if system else user
return schedule, run_as, parts[-1]
def _anacron_split(content: str) -> tuple[str, str, str] | None:
"""(period, job identifier, command) of an anacrontab line."""
parts = content.split(None, 3)
if len(parts) < 4 or not _ANACRON_PERIOD.match(parts[0]) or not parts[1].isdigit():
return None
return parts[0], parts[2], parts[3]
@dataclass
class _Scan:
"""What is set while a text is read line by line."""
env: dict[str, str] = field(default_factory=dict)
seen: dict[str, int] = field(default_factory=dict)
name: str = ""
description: str = ""
def key(self, base: str) -> str:
n = self.seen.get(base, 0) + 1
self.seen[base] = n
return base if n == 1 else f"{base}:{n}"
def take_name(self) -> tuple[str, str]:
named = self.name, self.description
self.name, self.description = "", ""
return named
def _content(raw: str) -> tuple[str, bool] | None:
"""The job text of *raw* and whether it is on; None for a comment or a blank."""
line = raw.strip()
if line.startswith(DISABLED_MARKER.strip()):
return line[len(DISABLED_MARKER.strip()) :].strip(), False
if not line or line.startswith("#"):
return None
return line, True
def _remember(scan: _Scan, raw: str) -> bool:
"""Take a variable or an application's marker from *raw*; whether it was one."""
line = raw.strip()
if line.startswith(JOB_MARKER.strip()):
name, _, description = line[len(JOB_MARKER.strip()) :].strip().partition(" ")
scan.name, scan.description = name, description.strip()
return True
variable = _VARIABLE.match(line)
if variable:
scan.env[variable[1]] = _unquote(variable[2])
return True
return False
def parse_crontab(path: str, text: str, *, system: bool, user: str = "") -> list[CronLine]:
"""The jobs of the crontab *text* at *path*.
*system*: a sixth field names the user (``/etc/crontab``, ``/etc/cron.d``);
otherwise every job runs as *user*.
"""
scan, jobs = _Scan(), []
for index, raw in enumerate(text.splitlines()):
if _remember(scan, raw):
continue
found = _content(raw)
split = _split(found[0], system=system, user=user) if found else None
if found is None or split is None:
continue
content, enabled = found
name, description = scan.take_name()
key = f"job:{name}" if name else scan.key(f"cron:{path}:{_digest(content)}")
schedule, run_as, command = split
jobs.append(
CronLine(
index, key, schedule, run_as, command, enabled, dict(scan.env), name, description
)
)
return jobs
def parse_anacrontab(text: str) -> list[CronLine]:
"""The jobs of an anacrontab: period, delay, identifier, command; all run as root."""
scan, jobs = _Scan(), []
for index, raw in enumerate(text.splitlines()):
if _remember(scan, raw):
continue
found = _content(raw)
split = _anacron_split(found[0]) if found else None
if found is None or split is None:
continue
period, identifier, command = split
key = scan.key(f"anacron:{identifier}")
jobs.append(CronLine(index, key, period, "root", command, found[1], dict(scan.env)))
return jobs
def _replace(text: str, index: int, lines: list[str]) -> str:
rows = text.splitlines()
rows[index : index + 1] = lines
return "\n".join(rows) + "\n"
def switched(text: str, line: CronLine, *, enabled: bool) -> str:
"""*text* with *line* switched on or off: the marker added or taken away, nothing else."""
raw = text.splitlines()[line.index]
found = _content(raw)
if found is None: # pragma: no cover - a CronLine always points at a job
raise ValueError(f"line {line.index} is no job")
content = found[0]
return _replace(text, line.index, [content if enabled else DISABLED_MARKER + content])
def marked_lines(name: str, description: str, line: str, *, enabled: bool) -> list[str]:
"""An application's job as crontab lines: its marker, then the job, maybe switched off."""
marker = f"{JOB_MARKER}{name} {description}".rstrip()
return [marker, line if enabled else DISABLED_MARKER + line]
def with_job(text: str, jobs: list[CronLine], name: str, lines: list[str]) -> str:
"""*text* with the application's job *name* as *lines*: replaced, or added at the end."""
current = next((j for j in jobs if j.name == name), None)
if current is None:
body = text if not text or text.endswith("\n") else text + "\n"
return body + "\n".join(lines) + "\n"
rows = text.splitlines()
rows[current.index - 1 : current.index + 1] = lines
return "\n".join(rows) + "\n"
def without_job(text: str, jobs: list[CronLine], name: str) -> str:
"""*text* without the application's job *name* and its marker."""
current = next(j for j in jobs if j.name == name)
rows = text.splitlines()
del rows[current.index - 1 : current.index + 1]
return "\n".join(rows) + ("\n" if rows else "")
def _unescape(part: str) -> str:
return part.replace("\\%", "%")
def run_script(command: str, env: dict[str, str]) -> str:
"""*command* as ``sh -c`` takes it, the way cron would run it.
The crontab's variables are exported first. The first ``%`` that is not
escaped ends the command; what follows is its stdin, with every further
``%`` a newline, and ``\\%`` is a ``%``.
"""
parts = re.split(r"(?<!\\)%", command)
exports = "".join(f"export {k}={quote(v)}; " for k, v in env.items() if k != "MAILTO")
head = _unescape(parts[0])
if len(parts) == 1:
return exports + head
stdin = "\n".join(_unescape(p) for p in parts[1:])
return f"{exports}printf '%s\\n' {quote(stdin)} | {{ {head}\n}}"
+417
View File
@@ -0,0 +1,417 @@
"""The scheduled-jobs report of a POSIX host: the command that prints it, and its parse.
:mod:`napalm_device_types.scheduled_jobs_posix` sends :func:`reading_command` and
hands what came back to :func:`parse_reading`. Pure: no I/O.
**One framed report, as root.** Every crontab, cron.d file, anacrontab and job
script is printed behind a header line and with ``|`` before each of its lines,
so no line of a file can pass for a header; run-parts scripts by their execute
bit and checksum; systemd's timers and the units they start by ``systemctl show``;
then the time zone and which package owns each definition file. A report whose
end marker is missing raises: a list cut short must never read as jobs that went
away.
"""
from __future__ import annotations
import re
from dataclasses import dataclass, field
from datetime import UTC, datetime
from shlex import quote
from napalm_device_types._crontab import JOB_MARKER, CronLine, parse_anacrontab, parse_crontab
from napalm_device_types.scheduled_jobs import DELETE, EDIT, job_digest
from napalm_device_types.terminal import strip_terminal_codes
_BEGIN = "SJ_BEGIN"
_END = "SJ_END"
SCHEDULE_HEADER = "# schedule: "
TIMER_PROPERTIES = (
"Id,Unit,Description,TimersCalendar,TimersMonotonic,NextElapseUSecRealtime,"
"LastTriggerUSec,UnitFileState,ActiveState,FragmentPath"
)
UNIT_PROPERTIES = "Id,ExecStart,User,Result,FragmentPath"
#: The names run-parts runs (Debian's ``--lsbsysinit``-less default).
_RUN_PARTS_NAME = re.compile(r"^[A-Za-z0-9_-]+$")
_SECTION = re.compile(r"^\[([a-z-]+)(?: (.*))?\]$")
#: Reads definition paths on stdin, prints ``<package>: <path>`` for each a package owns.
OWNERS_SH = (
"if command -v dpkg >/dev/null 2>&1; then xargs dpkg -S 2>/dev/null; "
"elif command -v rpm >/dev/null 2>&1; then while read -r p; do "
'o=$(rpm -qf --qf \'%{NAME}\' -- "$p" 2>/dev/null) && printf \'%s: %s\\n\' "$o" "$p"; done; '
"elif command -v apk >/dev/null 2>&1; then while read -r p; do "
'o=$(apk info -qW -- "$p" 2>/dev/null) && [ -n "$o" ] && printf \'%s: %s\\n\' "$o" "$p"; done; '
"else cat >/dev/null; fi"
)
#: Prints the host's time zone: an IANA name, or a path ending in ``zoneinfo/<name>``.
TIME_ZONE_SH = "cat /etc/timezone 2>/dev/null || readlink /etc/localtime 2>/dev/null"
@dataclass(frozen=True)
class Paths:
"""Where a host keeps its scheduled jobs."""
system_files: tuple[str, ...]
cron_d_dirs: tuple[str, ...]
spool_dirs: tuple[str, ...]
run_parts: tuple[tuple[str, str], ...] # (period, directory)
anacrontab: str
script_dir: str
systemd_marker: str = "/run/systemd/system"
owners_sh: str = OWNERS_SH
time_zone_sh: str = TIME_ZONE_SH
@dataclass
class Reading:
"""What one report said, beyond the jobs: what writes need to know."""
jobs: list[dict[str, object]] = field(default_factory=list)
time_zone: str | None = None
systemd: bool = False
#: per key: the unit a timer starts, the crontab line a cron job is
units: dict[str, str] = field(default_factory=dict)
lines: dict[str, CronLine] = field(default_factory=dict)
def _q(path: str) -> str:
return quote(path)
def _files(section: str, pattern: str) -> str:
return (
f'for f in {pattern}; do [ -f "$f" ] || continue; printf \'[%s %s]\\n\' {section} "$f"; '
# GNU sed keeps a missing last newline; without one of its own the next
# header would run on into the file's last line.
'sed \'s/^/|/\' "$f"; [ -n "$(tail -c 1 "$f")" ] && echo; P="$P\n$f"; done; '
)
def _run_parts(period: str, directory: str) -> str:
return (
f'for f in {_q(directory)}/*; do [ -f "$f" ] || continue; '
'[ -x "$f" ] && x=1 || x=0; c=$(cksum < "$f" | cut -d" " -f1); '
f'printf \'[run-parts %s %s %s %s]\\n\' {period} "$x" "$c" "$f"; P="$P\n$f"; done; '
)
def _timers(paths: Paths) -> str:
show = f"systemctl show --no-pager -p {TIMER_PROPERTIES}"
units = f"systemctl show --no-pager -p {UNIT_PROPERTIES}"
return (
f"if [ -d {_q(paths.systemd_marker)} ]; then echo '[systemd]'; "
"t=$(systemctl list-unit-files --type=timer --no-legend --no-pager 2>/dev/null "
"| awk '$1 !~ /@\\./ {print $1}'); "
'if [ -n "$t" ]; then '
f"s=$({show} --timestamp=unix $t 2>/dev/null || {show} $t 2>/dev/null); "
"echo '[timers]'; printf '%s\\n' \"$s\" | sed 's/^/|/'; "
"u=$(printf '%s\\n' \"$s\" | sed -n 's/^Unit=//p'); "
f"echo '[units]'; {units} $u 2>/dev/null | sed 's/^/|/'; "
"P=\"$P\n$(printf '%s\\n' \"$s\" | sed -n 's/^FragmentPath=//p')\"; fi; fi; "
)
def reading_command(paths: Paths) -> str:
"""One POSIX ``sh`` script that prints every scheduled job of the host; run it as root."""
parts = ["printf '%s%s\\n' SJ_ BEGIN; P=''; "]
parts += [_files("system", _q(f)) for f in paths.system_files]
parts += [_files("system", f"{_q(d)}/*") for d in paths.cron_d_dirs]
parts += [_files("user", f"{_q(d)}/*") for d in paths.spool_dirs]
parts.append(_files("anacron", _q(paths.anacrontab)))
parts += [_run_parts(period, d) for period, d in paths.run_parts]
parts.append(_files("script", f"{_q(paths.script_dir)}/*"))
parts.append(_timers(paths))
parts.append(
f"echo '[tz]'; {{ {paths.time_zone_sh}; }}; "
f"echo '[owners]'; printf '%s\\n' \"$P\" | sed '/^$/d' | {{ {paths.owners_sh}; }}; "
"printf '%s%s\\n' SJ_ END"
)
return "".join(parts)
# -- parse ------------------------------------------------------------------------------------
@dataclass
class _Section:
name: str
args: str
lines: list[str] = field(default_factory=list)
def _sections(output: str) -> list[_Section]:
text = strip_terminal_codes(output)
begin, end = text.rfind(_BEGIN), text.rfind(_END)
if begin < 0 or end < begin:
raise ValueError("the scheduled-jobs report is cut short (no end marker)")
sections: list[_Section] = []
for line in text[begin + len(_BEGIN) : end].splitlines():
header = _SECTION.match(line)
if header:
sections.append(_Section(header[1], header[2] or ""))
elif sections:
sections[-1].lines.append(line[1:] if line.startswith("|") else line)
return sections
def _owned(lines: list[str]) -> set[str]:
owned: set[str] = set()
for line in lines:
package, sep, path = line.partition(": ")
if sep and path.startswith("/") and not package.startswith("diversion"):
owned.add(path.strip())
return owned
def _time_zone(lines: list[str]) -> str | None:
value = next((ln.strip() for ln in lines if ln.strip()), "")
if "zoneinfo/" in value:
value = value.split("zoneinfo/", 1)[1]
return value or None
def _blocks(lines: list[str]) -> list[dict[str, list[str]]]:
blocks: list[dict[str, list[str]]] = [{}]
for line in lines:
if not line.strip():
blocks.append({})
continue
key, sep, value = line.partition("=")
if sep:
blocks[-1].setdefault(key, []).append(value)
return [b for b in blocks if b.get("Id")]
def _one(block: dict[str, list[str]], key: str) -> str:
return (block.get(key) or [""])[0].strip()
def _timestamp(value: str) -> str | None:
"""systemd's ``@<seconds>`` (or a UTC time) as ISO 8601; None when it never was."""
value = value.strip()
if value.startswith("@") and value[1:].isdigit() and int(value[1:]) > 0:
return datetime.fromtimestamp(int(value[1:]), UTC).isoformat()
match = re.search(r"(\d{4}-\d\d-\d\d \d\d:\d\d:\d\d) UTC$", value)
if match:
return datetime.strptime(match[1], "%Y-%m-%d %H:%M:%S").replace(tzinfo=UTC).isoformat()
return None
def _timer_entries(block: dict[str, list[str]], key: str) -> list[str]:
"""``{ OnCalendar=daily ; next_elapse=… }`` → ``OnCalendar=daily``, one per entry.
``systemctl show`` names a monotonic setting by its property (``OnUnitActiveUSec``);
it is given as the unit file says it (``OnUnitActiveSec``).
"""
entries = (v.strip(" {}").split(" ; ")[0].strip() for v in block.get(key, []) if v.strip())
return [re.sub(r"^(On\w+)USec=", r"\1Sec=", e) for e in entries]
def _argv(exec_start: list[str]) -> str:
commands = []
for value in exec_start:
match = re.search(r"argv\[\]=(.*?) ; ", value)
if match:
commands.append(match[1].strip())
return "; ".join(commands)
@dataclass
class _Script:
name: str
description: str
schedule: str
command: str
def _script(path: str, lines: list[str]) -> _Script | None:
"""An application's job script: its marker, schedule and command; None for any other file."""
marker = next((ln for ln in lines if ln.startswith(JOB_MARKER)), None)
schedule = next((ln for ln in lines if ln.startswith(SCHEDULE_HEADER)), None)
if marker is None or schedule is None:
return None
name, _, description = marker[len(JOB_MARKER) :].partition(" ")
body = [ln for ln in lines if ln not in (marker, schedule) and not ln.startswith("#!")]
return _Script(
name, description.strip(), schedule[len(SCHEDULE_HEADER) :], "\n".join(body).strip()
)
def _job(
*,
key: str,
kind: str,
source: str,
run_as: str,
schedule: str,
schedule_kind: str,
command: str,
enabled: bool,
managed_by: str,
actions: list[str],
content: str = "",
**extra: object,
) -> dict[str, object]:
job: dict[str, object] = {
"key": key,
"kind": kind,
"source": source,
"run_as": run_as,
"schedule": schedule,
"schedule_kind": schedule_kind,
"command": command,
"enabled": enabled,
"managed_by": managed_by,
"actions": actions,
}
job["digest"] = job_digest(job, content=content)
job.update({k: v for k, v in extra.items() if v not in (None, "")})
return job
_SWITCH = ["enable", "disable", "run"]
_OWN = [*_SWITCH, EDIT, DELETE]
def _cron_jobs(section: _Section, owned: set[str], reading: Reading) -> None:
path = section.args
text = "\n".join(section.lines)
if section.name == "anacron":
found = parse_anacrontab(text)
else:
user = path.rsplit("/", 1)[-1]
found = parse_crontab(path, text, system=section.name == "system", user=user)
kind = "anacron" if section.name == "anacron" else "cron"
for line in found:
own = bool(line.name)
reading.lines[line.key] = line
reading.jobs.append(
_job(
key=line.key,
kind=kind,
source=path,
run_as=line.run_as,
schedule=line.schedule,
schedule_kind="period" if kind == "anacron" else "cron",
command=line.command,
enabled=line.enabled,
managed_by="application" if own else ("package" if path in owned else "admin"),
actions=_OWN if own else _SWITCH,
name=line.name,
description=line.description,
)
)
def _run_parts_job(section: _Section, owned: set[str], reading: Reading) -> None:
period, executable, checksum, path = section.args.split(" ", 3)
if not _RUN_PARTS_NAME.match(path.rsplit("/", 1)[-1]):
return
reading.jobs.append(
_job(
key=f"run-parts:{path}",
kind="run_parts",
source=path,
run_as="root",
schedule=period,
schedule_kind="period",
command=path,
enabled=executable == "1",
managed_by="package" if path in owned else "admin",
actions=_SWITCH,
content=checksum,
)
)
def _timer_job(
timer: dict[str, list[str]],
unit: dict[str, list[str]],
scripts: dict[str, _Script],
owned: set[str],
) -> dict[str, object]:
fragment = _one(timer, "FragmentPath")
calendar = _timer_entries(timer, "TimersCalendar")
schedule = "; ".join(calendar + _timer_entries(timer, "TimersMonotonic"))
command = _argv(unit.get("ExecStart", []))
script = scripts.get(command.split(" ", 1)[0])
last_run = _timestamp(_one(timer, "LastTriggerUSec"))
switchable = _one(timer, "UnitFileState") not in ("static", "generated", "transient")
common = {
"kind": "systemd_timer",
"source": fragment,
"run_as": _one(unit, "User") or "root",
"enabled": _one(timer, "ActiveState") == "active",
"last_run": last_run,
"last_result": _one(unit, "Result") or None if last_run else None,
"next_run": _timestamp(_one(timer, "NextElapseUSecRealtime")),
}
if script is not None:
return _job(
key=f"job:{script.name}",
schedule=script.schedule,
schedule_kind="cron",
command=script.command,
managed_by="application",
actions=_OWN,
name=script.name,
description=script.description,
**common, # type: ignore[arg-type]
)
packaged = fragment in owned or fragment.startswith(("/usr/lib/", "/lib/"))
return _job(
key=f"timer:{_one(timer, 'Id')}",
schedule=schedule,
schedule_kind="calendar" if calendar else "monotonic",
command=command,
managed_by="package" if packaged else "admin",
actions=_SWITCH if switchable else ["run"],
description=_one(timer, "Description"),
**common, # type: ignore[arg-type]
)
def _timer_jobs(
sections: dict[str, _Section], scripts: dict[str, _Script], owned: set[str], reading: Reading
) -> None:
timers = _blocks(sections["timers"].lines) if "timers" in sections else []
units = (
{_one(b, "Id"): b for b in _blocks(sections["units"].lines)} if "units" in sections else {}
)
for timer in timers:
unit = units.get(_one(timer, "Unit"), {})
job = _timer_job(timer, unit, scripts, owned)
reading.units[str(job["key"])] = _one(timer, "Unit")
reading.jobs.append(job)
def parse_reading(output: str) -> Reading:
"""The jobs, time zone and write context of a report from :func:`reading_command`.
:raises ValueError: when the report is cut short.
"""
sections = _sections(output)
named = {s.name: s for s in sections if s.name in ("timers", "units", "tz", "owners")}
owned = _owned(named["owners"].lines) if "owners" in named else set()
reading = Reading(
time_zone=_time_zone(named["tz"].lines) if "tz" in named else None,
systemd=any(s.name == "systemd" for s in sections),
)
for section in sections:
if section.name in ("system", "user", "anacron"):
_cron_jobs(section, owned, reading)
elif section.name == "run-parts":
_run_parts_job(section, owned, reading)
_timer_jobs(named, _scripts(sections), owned, reading)
return reading
def _scripts(sections: list[_Section]) -> dict[str, _Script]:
"""An application's job scripts, by path."""
scripts = ((s.args, _script(s.args, s.lines)) for s in sections if s.name == "script")
return {path: script for path, script in scripts if script is not None}
+238
View File
@@ -0,0 +1,238 @@
"""
Abstract base class for backup server drivers.
Usage::
from napalm_device_types import BackupServerDriver, StorageDriver
class ProxmoxBackupDriver(BackupServerDriver, StorageDriver):
def get_backup_datastores(self):
...
"""
from typing import TYPE_CHECKING
from napalm_device_types.base import DeviceTypeDriver
if TYPE_CHECKING: # pragma: no cover - for type checkers only
from napalm_device_types.backups import (
BackupDatastoreDict,
BackupGroupDict,
BackupJobKind,
BackupRemoteDict,
BackupServerJobDict,
BackupSnapshotDict,
BackupSnapshotRefDict,
BackupTaskDict,
BackupTaskStatusDict,
)
class BackupServerDriver(DeviceTypeDriver):
"""
Abstract intermediate driver for backup servers (e.g. Proxmox Backup
Server): the device hypervisors and hosts write their backups to.
A backup server keeps its backups in datastores, one group per source (a
VM, a container, a host) and one snapshot per backup run, and keeps them
healthy with scheduled jobs: garbage collection, verify, prune, sync with
other servers and tape. None of that is a storage pool, a volume or a
replication job, so it is not modelled through ``StorageDriver``. A server
that also manages its own disks and pools lists ``StorageDriver`` as a
second role, after this one.
The actions start a task on the server and return at once with its id;
:meth:`get_backup_task` follows it.
"""
#: Stable key netOrk exposes as ``device_class``. The order in which a
#: driver lists its role bases is the ranking; see
#: :func:`napalm_device_types.roles.primary_role_of`.
ROLE: str = "backup_server"
TYPE_LABEL: str = "Backup Server"
if TYPE_CHECKING:
# ------------------------------------------------------------------
# Read
# ------------------------------------------------------------------
def get_backup_datastores(self) -> list[BackupDatastoreDict]:
"""
Returns the server's datastores with their usage.
Example::
[
{
"name": "store1",
"path": "/mnt/datastore/store1",
"backend": "local",
"total": 4000787030016,
"used": 1529008357376,
"available": 2471778672640,
"estimated_full": 1798761600,
"maintenance": None,
"comment": None,
},
]
:raises Exception: when the server could not be read; never an
empty list for a failed read.
"""
...
def get_backup_groups(self, datastore: str = "") -> list[BackupGroupDict]:
"""
Returns the backup groups of every namespace of *datastore*, or of
every datastore when it is empty, each with its newest snapshot
only. :meth:`get_backup_snapshots` lists all of a group's.
Example::
[
{
"datastore": "store1",
"namespace": "",
"type": "vm",
"id": "101",
"owner": "backup@pbs",
"count": 14,
"newest": {
"time": 1793660400,
"size": 34359738368,
"verify": "ok",
"protected": False,
"comment": "web-01",
},
"oldest_time": 1792537200,
"comment": None,
},
]
:raises Exception: when the server could not be read.
"""
...
def get_backup_snapshots(
self, datastore: str, namespace: str, group_type: str, group_id: str
) -> list[BackupSnapshotDict]:
"""
Returns every snapshot of one group, newest first.
:param namespace: ``""`` for the datastore's root.
:raises ValueError: when the datastore or the group does not exist.
"""
...
def get_backup_jobs(self) -> list[BackupServerJobDict]:
"""
Returns the server's scheduled jobs of every kind, garbage
collection included (one per datastore, ``id`` = the datastore).
Example::
[
{
"id": "store1",
"kind": "gc",
"datastore": "store1",
"namespace": None,
"schedule": "daily",
"enabled": True,
"last_run": 1793664000,
"last_state": "ok",
"last_error": None,
"next_run": 1793750400,
"remote": None,
"direction": None,
"comment": None,
},
{
"id": "offsite",
"kind": "sync",
"datastore": "store1",
"namespace": "",
"schedule": "sat 03:00",
"enabled": True,
"last_run": 1793502000,
"last_state": "error",
"last_error": "connection refused",
"next_run": 1794106800,
"remote": "pbs-offsite",
"direction": "push",
"comment": None,
},
]
:raises Exception: when the server could not be read.
"""
...
def get_backup_remotes(self) -> list[BackupRemoteDict]:
"""
Returns the other backup servers this one syncs with, without their
secrets.
:raises Exception: when the server could not be read.
"""
...
# ------------------------------------------------------------------
# Actions
# ------------------------------------------------------------------
def start_garbage_collection(self, datastore: str) -> BackupTaskDict:
"""
Starts garbage collection on *datastore*.
:raises ValueError: when the datastore does not exist.
:raises RuntimeError: when the server refuses (e.g. one is running).
"""
...
def run_backup_job(self, kind: BackupJobKind, job_id: str) -> BackupTaskDict:
"""
Runs a scheduled job now, outside its schedule.
:param kind: a :data:`~napalm_device_types.backups.BackupJobKind`;
``"gc"`` runs the datastore named *job_id*'s garbage collection.
:raises ValueError: when there is no such job.
:raises RuntimeError: when the server refuses.
"""
...
def verify_backup_snapshot(self, snapshot: BackupSnapshotRefDict) -> BackupTaskDict:
"""
Verifies one snapshot now.
:raises ValueError: when the snapshot does not exist.
"""
...
def set_backup_snapshot_protected(
self, snapshot: BackupSnapshotRefDict, protected: bool
) -> None:
"""
Protects a snapshot from pruning and deletion, or lifts that.
:raises ValueError: when the snapshot does not exist.
"""
...
def delete_backup_snapshot(self, snapshot: BackupSnapshotRefDict) -> None:
"""
Deletes one snapshot. The data is gone once garbage collection has
run; there is no undo.
:raises ValueError: when the snapshot does not exist.
:raises RuntimeError: when it is protected.
"""
...
def get_backup_task(self, task: str) -> BackupTaskStatusDict:
"""
Returns the state and the log so far of a task an action started.
:raises ValueError: when the server does not know the task.
"""
...
+119
View File
@@ -0,0 +1,119 @@
"""The shapes of a backup server (``BackupServerDriver``).
Apart from ``models.py``, which is over the file size limit. A value the
platform does not report is None; it is never guessed.
"""
from typing import Literal, TypedDict, get_args
#: What a backup server's jobs do. Garbage collection is one of them: it has a
#: schedule and a last run like the others.
BackupJobKind = Literal["gc", "verify", "prune", "sync", "tape"]
BACKUP_JOB_KINDS: tuple[str, ...] = get_args(BackupJobKind)
#: How a job's or a task's run ended; "running" while it has not.
BackupRunState = Literal["ok", "warning", "error", "running"]
#: The outcome of the last verification of a snapshot.
BackupVerifyState = Literal["ok", "failed"]
class BackupDatastoreDict(TypedDict):
"""A datastore: where a backup server keeps its backups."""
name: str
path: str | None # local path; for an object-store backend the local cache, if any
backend: str # "local", "s3", "removable", ... as the platform names it
total: int | None # bytes
used: int | None # bytes
available: int | None # bytes
estimated_full: int | None # unix seconds the platform expects it to be full
maintenance: str | None # "read-only", "offline", ...; None when in service
comment: str | None
class BackupSnapshotDict(TypedDict):
"""One backup of a group: the state of its source at one time."""
time: int # unix seconds the backup was taken
size: int | None # bytes
verify: BackupVerifyState | None # None: never verified
protected: bool # exempt from pruning and deletion
comment: str | None
class BackupGroupDict(TypedDict):
"""The backups of one source (a VM, a container, a host) in one namespace."""
datastore: str
namespace: str # "" for the datastore's root
type: str # "vm", "ct", "host"
id: str # the guest's id ("101") or the host's name
owner: str | None # the account that writes the group's backups
count: int # snapshots in the group
newest: BackupSnapshotDict | None # None for a group without a snapshot
oldest_time: int | None # unix seconds of the oldest snapshot
comment: str | None
class BackupSnapshotRefDict(TypedDict):
"""Names one snapshot, for an action on it."""
datastore: str
namespace: str # "" for the datastore's root
group_type: str # BackupGroupDict.type
group_id: str # BackupGroupDict.id
time: int # BackupSnapshotDict.time
class BackupServerJobDict(TypedDict):
"""A scheduled job of a backup server."""
id: str # for "gc", the datastore's name
kind: BackupJobKind
datastore: str | None
namespace: str | None
schedule: str | None # as the platform writes it: "daily", "sat 02:00", "*:0/30"
enabled: bool
last_run: int | None # unix seconds the last run ended (or started, while running)
last_state: BackupRunState | None # None: never ran
last_error: str | None
next_run: int | None # unix seconds
remote: str | None # a sync job's BackupRemoteDict.name
direction: Literal["pull", "push"] | None # a sync job's
comment: str | None
class BackupRemoteDict(TypedDict):
"""Another backup server this one syncs with. Never its secret."""
name: str
host: str
port: int | None
fingerprint: str | None # of the remote's TLS certificate
auth_id: str | None # the account the sync logs in as
comment: str | None
class BackupTaskDict(TypedDict):
"""A task an action started on the server, to follow with ``get_backup_task``."""
task: str # the platform's task id
class BackupTaskStatusDict(TypedDict):
task: str
running: bool
state: BackupRunState | None # "running" while it runs
error: str | None
log: list[str] # the task's log lines so far
class BackupServerRefDict(TypedDict):
"""Which backup server and datastore a hypervisor's backup storage writes to."""
server: str # host name or address, as the hypervisor has it configured
port: int | None
datastore: str
namespace: str | None
fingerprint: str | None # of the server's TLS certificate, as the hypervisor pins it
+5
View File
@@ -5,6 +5,8 @@ Apart from ``models.py``, which is over the file size limit.
from typing import NotRequired, TypedDict
from napalm_device_types.backups import BackupServerRefDict
class ClusterNodeDict(TypedDict):
"""One host of a cluster, as its cluster sees it. A value the platform does not
@@ -34,6 +36,9 @@ class ClusterStorageDict(TypedDict):
nodes: list[str] # the nodes it is available on
total: int | None # bytes; None where the platform does not report it
used: int | None # bytes
# A backup storage that writes to a backup server: which one. Left out for every
# other storage; None where the platform does not say.
backup_server: NotRequired[BackupServerRefDict | None]
class ClusterNetworkDict(TypedDict):
+203
View File
@@ -0,0 +1,203 @@
"""Cron expressions: what they mean, when they fire next, and as systemd ``OnCalendar``.
A scheduled job is given as a cron expression on every host, and a host with
systemd runs it as a timer. So the expression has to mean the same in three
places -- on the host's cron, in a preview of the next runs, and in the
``OnCalendar`` lines a timer is written with -- and the meaning lives here once.
**What cron means is Vixie cron's.** Five fields (minute, hour, day of month,
month, day of week), each ``*``, a value, a range ``a-b``, a step ``*/n`` or
``a-b/n``, or a list of those; month and weekday names in any case; ``7`` is
Sunday as well as ``0``; the ``@hourly`` … ``@yearly`` aliases. When both day
fields are restricted -- neither starts with ``*`` -- a day matches if *either*
does; otherwise both must (``*/2`` counts as starting with ``*``).
**``OnCalendar`` by expansion.** Every field is written as its values, so a range
with a step needs no systemd syntax of its own, and a timer fires on exactly the
minutes cron would. The *either* rule is two ``OnCalendar`` lines, which systemd
ORs. ``@reboot`` has no calendar; a timer takes it as ``OnBootSec``.
**Next runs are in the host's time zone.** A local time the clock skips when
summer time starts is left out; cron implementations disagree on it, and a
preview that promised it would promise too much.
"""
from __future__ import annotations
from datetime import date, datetime, timedelta
from typing import NamedTuple
from zoneinfo import ZoneInfo
REBOOT = "@reboot"
ALIASES = {
"@yearly": "0 0 1 1 *",
"@annually": "0 0 1 1 *",
"@monthly": "0 0 1 * *",
"@weekly": "0 0 * * 0",
"@daily": "0 0 * * *",
"@midnight": "0 0 * * *",
"@hourly": "0 * * * *",
}
_MONTHS = ("jan", "feb", "mar", "apr", "may", "jun", "jul", "aug", "sep", "oct", "nov", "dec")
_WEEKDAYS = ("sun", "mon", "tue", "wed", "thu", "fri", "sat")
_SYSTEMD_WEEKDAYS = ("Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat")
#: (lowest, highest, names) per field; a name stands for its index plus the lowest.
_FIELDS = (
(0, 59, ()),
(0, 23, ()),
(1, 31, ()),
(1, 12, _MONTHS),
(0, 7, _WEEKDAYS),
)
#: How far ahead a next run is looked for: a 29 February comes within eight years.
_HORIZON_DAYS = 8 * 366
class CronSchedule(NamedTuple):
"""An expression's values per field, and whether a day matches on either day field."""
minutes: frozenset[int]
hours: frozenset[int]
days: frozenset[int]
months: frozenset[int]
weekdays: frozenset[int] # 0 = Sunday
either_day: bool
def is_reboot(expr: str) -> bool:
"""Whether *expr* is ``@reboot``: once at boot, no calendar."""
return expr.strip().lower() == REBOOT
def _value(token: str, low: int, names: tuple[str, ...]) -> int:
if token.isdigit():
return int(token)
if token.lower() in names:
return names.index(token.lower()) + low
raise ValueError(f"not a value: {token!r}")
def _part(part: str, low: int, high: int, names: tuple[str, ...]) -> set[int]:
span, _, step_text = part.partition("/")
step = 1
if step_text:
if not step_text.isdigit() or int(step_text) == 0:
raise ValueError(f"not a step: {part!r}")
step = int(step_text)
if span == "*":
first, last = low, high
elif "-" in span:
a, _, b = span.partition("-")
first, last = _value(a, low, names), _value(b, low, names)
else:
first = _value(span, low, names)
last = high if step_text else first
if not low <= first <= last <= high:
raise ValueError(f"out of range {low}-{high}: {part!r}")
return set(range(first, last + 1, step))
def _field(text: str, low: int, high: int, names: tuple[str, ...]) -> frozenset[int]:
values: set[int] = set()
for part in text.split(","):
if not part:
raise ValueError(f"empty list item in {text!r}")
values |= _part(part, low, high, names)
return frozenset(values)
def parse_cron(expr: str) -> CronSchedule:
"""The values *expr* stands for.
:raises ValueError: for anything Vixie cron would not take, and for
``@reboot``, which names no time.
"""
text = expr.strip()
if is_reboot(text):
raise ValueError("@reboot names no time; it runs once at boot")
if text.startswith("@"):
if text.lower() not in ALIASES:
raise ValueError(f"unknown alias {text!r}")
text = ALIASES[text.lower()]
fields = text.split()
if len(fields) != len(_FIELDS):
raise ValueError(f"a cron expression has five fields, not {len(fields)}: {expr!r}")
minutes, hours, days, months, weekdays = (
_field(f, low, high, names) for f, (low, high, names) in zip(fields, _FIELDS, strict=True)
)
if 7 in weekdays:
weekdays = (weekdays - {7}) | {0}
either = not fields[2].startswith("*") and not fields[4].startswith("*")
return CronSchedule(minutes, hours, days, months, frozenset(weekdays), either)
def _day_matches(s: CronSchedule, day: date) -> bool:
if day.month not in s.months:
return False
on_day = day.day in s.days
on_weekday = (day.isoweekday() % 7) in s.weekdays
return (on_day or on_weekday) if s.either_day else (on_day and on_weekday)
def _local(day: date, hour: int, minute: int, tz: ZoneInfo) -> datetime | None:
"""*day* at *hour*:*minute* in *tz*; None when the clock skips that time."""
naive = datetime(day.year, day.month, day.day, hour, minute)
aware = naive.replace(tzinfo=tz)
if aware.astimezone(ZoneInfo("UTC")).astimezone(tz).replace(tzinfo=None) != naive:
return None
return aware
def next_runs(expr: str, after: datetime, tz: ZoneInfo, count: int = 5) -> list[datetime]:
"""The next *count* times *expr* fires after *after*, in *tz*.
``@reboot`` and a day that never comes (31 February) give none.
:raises ValueError: for an expression :func:`parse_cron` refuses.
"""
if is_reboot(expr):
return []
s = parse_cron(expr)
start = after.astimezone(tz)
runs: list[datetime] = []
for offset in range(_HORIZON_DAYS):
day = start.date() + timedelta(days=offset)
if not _day_matches(s, day):
continue
for hour in sorted(s.hours):
for minute in sorted(s.minutes):
at = _local(day, hour, minute, tz)
if at is None or at <= after:
continue
runs.append(at)
if len(runs) == count:
return runs
return runs
def _values(values: frozenset[int], low: int, high: int) -> str:
if values == frozenset(range(low, high + 1)):
return "*"
return ",".join(f"{v:02d}" for v in sorted(values))
def to_on_calendar(expr: str) -> list[str]:
"""``OnCalendar`` lines that fire exactly when *expr* does.
:raises ValueError: for an expression :func:`parse_cron` refuses, ``@reboot``
among them -- a timer takes that as ``OnBootSec``.
"""
s = parse_cron(expr)
time = f"{_values(s.hours, 0, 23)}:{_values(s.minutes, 0, 59)}:00"
months = _values(s.months, 1, 12)
days = _values(s.days, 1, 31)
weekdays = ",".join(_SYSTEMD_WEEKDAYS[d] for d in (1, 2, 3, 4, 5, 6, 0) if d in s.weekdays)
every_weekday = s.weekdays == frozenset(range(7))
if s.either_day:
return [f"*-{months}-{days} {time}", f"{weekdays} *-{months}-* {time}"]
prefix = "" if every_weekday else f"{weekdays} "
return [f"{prefix}*-{months}-{days} {time}"]
+97
View File
@@ -0,0 +1,97 @@
"""LLDP neighbours as rows, and the two shapes NAPALM asks for.
A driver reads its neighbours one row per neighbour -- from ``lldpctl``
(:func:`~napalm_device_types.lldpctl.parse_lldpctl_keyvalue`), an API or a CLI
-- and ``get_lldp_neighbors`` and ``get_lldp_neighbors_detail`` are the same
projection of those rows for every driver. A port with several neighbours (a
phone and the PC behind it, an unmanaged switch) is several rows, and stays
several entries in both shapes.
"""
from __future__ import annotations
import re
from collections.abc import Iterable
from typing import NotRequired, TypedDict
from napalm.base.models import LLDPNeighborDetailDict
#: A chassis id of type ``mac`` that really is a MAC address.
_MAC_RE = re.compile(r"^([0-9A-Fa-f]{2}:){5}[0-9A-Fa-f]{2}$")
class LldpNeighborRow(TypedDict):
"""One LLDP neighbour, as the local port heard it.
``chassis_id_type`` and ``port_id_type`` name what the id is: ``mac``,
``ip``, ``ifname``, ``ifalias``, ``local`` or ``unhandled`` (an id type
lldpd does not decode), ``""`` when the neighbour sent no id.
``capabilities`` and ``enabled_capabilities`` use NAPALM's names
(``bridge``, ``router``, ``wlan-access-point``, ``telephone``, ...).
"""
local_port: str
chassis_id: str
chassis_id_type: str
port_id: str
port_id_type: str
port_description: str
system_name: str
system_description: str
management_addresses: list[str]
capabilities: list[str]
enabled_capabilities: list[str]
class LldpNeighborDict(TypedDict):
"""NAPALM's ``get_lldp_neighbors`` entry (``hostname``, ``port``), plus the chassis MAC.
``mac`` (lower case) is there only when the neighbour names its chassis by a
MAC address; an application can match the neighbour by it, where a name
may be ambiguous.
"""
hostname: str
port: str
mac: NotRequired[str]
def lldp_neighbors(rows: Iterable[LldpNeighborRow]) -> dict[str, list[LldpNeighborDict]]:
"""``get_lldp_neighbors``: per local port, every neighbour's name, port and chassis MAC."""
neighbors: dict[str, list[LldpNeighborDict]] = {}
for row in rows:
entry: LldpNeighborDict = {"hostname": row["system_name"], "port": row["port_id"]}
if row["chassis_id_type"] == "mac" and _MAC_RE.match(row["chassis_id"]):
entry["mac"] = row["chassis_id"].lower()
neighbors.setdefault(row["local_port"], []).append(entry)
return neighbors
def lldp_neighbors_detail(
rows: Iterable[LldpNeighborRow], interface: str = ""
) -> dict[str, list[LLDPNeighborDetailDict]]:
"""``get_lldp_neighbors_detail``: per local port, every neighbour in full.
With ``interface``, that port only. ``parent_interface`` stays empty: a row
does not know which aggregate its port belongs to.
"""
details: dict[str, list[LLDPNeighborDetailDict]] = {}
for row in rows:
if interface and row["local_port"] != interface:
continue
details.setdefault(row["local_port"], []).append(
{
"parent_interface": "",
"remote_port": row["port_id"],
"remote_port_description": row["port_description"],
"remote_chassis_id": row["chassis_id"],
"remote_system_name": row["system_name"],
"remote_system_description": row["system_description"],
"remote_system_capab": list(row["capabilities"]),
"remote_system_enable_capab": list(row["enabled_capabilities"]),
}
)
return details
__all__ = ["LldpNeighborDict", "LldpNeighborRow", "lldp_neighbors", "lldp_neighbors_detail"]
+204
View File
@@ -0,0 +1,204 @@
"""``lldpctl -f keyvalue``: lldpd's neighbours, one row each.
Every driver for a host that runs lldpd reads its neighbours the same way, so
the command and its parse live here once and a driver only carries the command
across. lldpd's keyvalue writer (``src/client/kv_writer.c``) prints one
``key=value`` line per field, the key being the field's path joined by dots
(a newline inside a value becomes a space)::
lldp.eth0.1.via=LLDP
lldp.eth0.1.chassis.mac=00:11:22:33:44:01
lldp.eth0.1.chassis.Tel.enabled=on
lldp.eth0.1.port.local=gi1/0/7
**A neighbour is a block, not a port.** Every neighbour on a port prints the
same keys, so grouping the lines by interface merges them. A neighbour opens
with ``via`` (after the local interface's ``alias``, when it has one); the next
``via`` on the same port is the next neighbour.
**An interface name may hold dots** (``eth0.1``, ``br-lan.10``). It is
everything between ``lldp.`` and the neighbour's first field, never the second
dotted part of the key.
**An id is printed under its type**, ``chassis.<type>=`` and ``port.<type>=``,
with lldpd's names for the type (``mac``, ``ip``, ``ifname``, ``ifalias``,
``local``, ``unhandled``); there is no ``chassis.id`` key.
**A failed read raises.** lldpctl exits non-zero when it cannot reach lldpd;
:data:`LLDPCTL_KEYVALUE_COMMAND` brings that status and lldpctl's message back,
and the parse raises on it instead of reading it as no neighbours. Output that
carries no exit status (a driver that runs plain ``lldpctl -f keyvalue``) raises
when it holds text but not a single neighbour line, as a shell's ``lldpctl: not
found`` does.
"""
from __future__ import annotations
import re
from napalm_device_types.lldp import LldpNeighborRow
_RC = "__LLDPCTL_RC"
#: One line, POSIX ``sh``, read-only. Without arguments lldpctl shows the
#: neighbours in detail; its error text and exit status come back with them.
LLDPCTL_KEYVALUE_COMMAND = f"lldpctl -f keyvalue 2>&1; echo {_RC}=$?"
_RC_RE = re.compile(rf"^{_RC}=(\d+)$")
_PREFIX = "lldp."
#: The fields a neighbour opens with: the local interface's alias, then ``via``.
_OPENERS = frozenset({"alias", "via"})
#: The first part after the interface name of every key lldpd prints for a
#: neighbour (``src/client/display.c``, ``display_interface``).
_CATEGORIES = _OPENERS | {
"rid",
"age",
"status",
"chassis",
"port",
"ttl",
"vlan",
"ppvid",
"pi",
"lldp-med",
"unknown-tlvs",
}
#: lldpd's id types (``src/lib/atoms/chassis.c``, ``port.c``), in the order read.
_CHASSIS_ID_TYPES = ("mac", "local", "ip", "ifname", "ifalias", "unhandled")
_PORT_ID_TYPES = ("ifname", "mac", "local", "ip", "ifalias", "unhandled")
#: lldpd's capability symbols in NAPALM's names; one NAPALM lacks is lowercased.
_CAPABILITIES = {
"Other": "other",
"Repeater": "repeater",
"Bridge": "bridge",
"Router": "router",
"Wlan": "wlan-access-point",
"Tel": "telephone",
"Docsis": "docsis-cable-device",
"Station": "station",
}
_SWITCH = {"on": True, "off": False}
_Fields = dict[str, list[str]]
def parse_lldpctl_keyvalue(output: str) -> list[LldpNeighborRow]:
"""One row per neighbour, in the order lldpctl listed them.
:raises ValueError: when lldpctl failed -- a non-zero exit status, or text
without a single neighbour line and no status at all.
"""
return [_row(port, fields) for port, fields in _neighbours(_pairs(output))]
def _pairs(output: str) -> list[tuple[str, str]]:
"""``(key without "lldp.", value)`` of every neighbour line; raises on a failed run."""
pairs: list[tuple[str, str]] = []
other: list[str] = []
status: int | None = None
for line in output.splitlines():
line = line.strip()
key, sep, value = line.partition("=")
if _RC_RE.match(line):
status = int(value)
elif sep and key.startswith(_PREFIX):
pairs.append((key[len(_PREFIX) :], value))
elif line:
other.append(line)
_raise_on_failure(status, other, bool(pairs))
return pairs
def _raise_on_failure(status: int | None, other: list[str], listed: bool) -> None:
"""A non-zero status, or text without a status and without a neighbour line."""
if status or (status is None and other and not listed):
said = " ".join(other)[:200] or f"exit status {status}"
raise ValueError(f"lldpctl did not list the neighbours: {said}")
def _neighbours(pairs: list[tuple[str, str]]) -> list[tuple[str, _Fields]]:
"""Split the lines into neighbours: ``(local interface, {field: [values]})``."""
neighbours: list[tuple[str, _Fields]] = []
port: str | None = None
fields: _Fields = {}
for key, value in pairs:
split = _split(key, port)
if split is None:
continue
name, field = split
if name != port or field == "alias" or (field == "via" and "via" in fields):
port, fields = name, {}
neighbours.append((port, fields))
fields.setdefault(field, []).append(value)
return neighbours
def _split(key: str, port: str | None) -> tuple[str, str] | None:
"""``(interface, field)`` of a key, or None when it is no neighbour field.
``via`` and ``alias`` end their key, so they name the interface exactly;
the keys after them continue that interface. Only a key that does neither
ends its interface name at the first known field.
"""
head, _, last = key.rpartition(".")
if head and last in _OPENERS:
return head, last
if port is not None and key.startswith(port + "."):
field = key[len(port) + 1 :]
if field.split(".", 1)[0] in _CATEGORIES:
return port, field
parts = key.split(".")
for i in range(1, len(parts)):
if parts[i] in _CATEGORIES:
return ".".join(parts[:i]), ".".join(parts[i:])
return None
def _row(port: str, fields: _Fields) -> LldpNeighborRow:
chassis_type, chassis_id = _id(fields, "chassis", _CHASSIS_ID_TYPES)
port_type, port_id = _id(fields, "port", _PORT_ID_TYPES)
capabilities, enabled = _capabilities(fields)
return {
"local_port": port,
"chassis_id": chassis_id,
"chassis_id_type": chassis_type,
"port_id": port_id,
"port_id_type": port_type,
"port_description": _first(fields, "port.descr"),
"system_name": _first(fields, "chassis.name"),
"system_description": _first(fields, "chassis.descr"),
"management_addresses": list(fields.get("chassis.mgmt-ip", [])),
"capabilities": capabilities,
"enabled_capabilities": enabled,
}
def _first(fields: _Fields, field: str) -> str:
return fields.get(field, [""])[0]
def _id(fields: _Fields, category: str, types: tuple[str, ...]) -> tuple[str, str]:
"""``(type, id)`` of the chassis or the port; ``("", "")`` when none was sent."""
for kind in types:
if f"{category}.{kind}" in fields:
return kind, _first(fields, f"{category}.{kind}")
return "", ""
def _capabilities(fields: _Fields) -> tuple[list[str], list[str]]:
"""``chassis.<Symbol>.enabled=on|off``: the capabilities offered, and those enabled."""
offered: list[str] = []
enabled: list[str] = []
for field, values in fields.items():
parts = field.split(".")
switch = _SWITCH.get(values[0])
if len(parts) != 3 or parts[0] != "chassis" or parts[2] != "enabled" or switch is None:
continue
name = _CAPABILITIES.get(parts[1], parts[1].lower())
offered.append(name)
if switch:
enabled.append(name)
return offered, enabled
__all__ = ["LLDPCTL_KEYVALUE_COMMAND", "parse_lldpctl_keyvalue"]
-9
View File
@@ -1038,15 +1038,6 @@ class ProcessDict(TypedDict):
command: str # full command line
class CronJobDict(TypedDict):
"""A scheduled cron task."""
user: str # owner of the crontab entry
schedule: str # five-field cron expression, e.g. "0 * * * *"
command: str # shell command to execute
description: NotRequired[str] # optional inline comment
class ApplyUpdatesResultDict(TypedDict):
"""Result of an ``apply_updates()`` call."""
-29
View File
@@ -15,7 +15,6 @@ from typing import TYPE_CHECKING
from napalm_device_types.base import DeviceTypeDriver
from napalm_device_types.health_metrics import HealthMetricsMixin
from napalm_device_types.models import (
CronJobDict,
DeviceActionResultDict,
ProcessDict,
SNMPConfigDict,
@@ -131,34 +130,6 @@ class OSDriver(
"""
...
# ------------------------------------------------------------------
# Cron jobs
# ------------------------------------------------------------------
def get_cron_jobs(self) -> list[CronJobDict]:
"""
Returns scheduled cron tasks from all user crontabs and ``/etc/cron.d``.
Each entry contains:
* user (string) - owner of the crontab entry
* schedule (string) - five-field cron expression (e.g. ``"0 * * * *"``)
* command (string) - shell command to execute
* description (string, optional) - inline comment text if present
Example::
[
{
"user": "root",
"schedule": "0 4 * * *",
"command": "/usr/local/bin/backup.sh",
"description": "nightly backup",
},
]
"""
...
# ------------------------------------------------------------------
# SNMP
# ------------------------------------------------------------------
+5 -2
View File
@@ -28,9 +28,12 @@ APT_UPGRADABLE_COMMAND = "{ LC_ALL=C apt list --upgradable 2>/dev/null; echo __A
#: Read-only. Lists the packages that a pending security advisory covers. A
#: failed dnf prints nothing, the same as no advisories, so its exit status is
#: printed inside the group, as apt's is.
#: printed inside the group, as apt's is. It reads the cached metadata only, as
#: apt reads its local index: an expired cache otherwise makes dnf download the
#: repositories' metadata first, which can take minutes (#76).
DNF_SECURITY_COMMAND = (
"{ LC_ALL=C dnf updateinfo list --security --quiet 2>/dev/null; echo __DNF_RC=$?; } | cat"
"{ LC_ALL=C dnf --cacheonly updateinfo list --security --quiet 2>/dev/null;"
" echo __DNF_RC=$?; } | cat"
)
# openssl/noble-updates,noble-security 3.0.13-0ubuntu3.6 amd64 [upgradable from: 3.0.13-0ubuntu3.5]
+196
View File
@@ -0,0 +1,196 @@
"""Scheduled jobs: what a host runs on its own schedule, and switching it on, off and now.
A host runs work on its own: cron lines, run-parts scripts, anacron, systemd
timers, FreeBSD's ``periodic``, a firewall's or a NAS's own scheduled jobs, the
Windows Task Scheduler. :class:`ScheduledJobsMixin` reads all of them as one
shape, :class:`~napalm_device_types.models.ScheduledJobDict`, and acts on them.
Declared under ``if TYPE_CHECKING`` like a role: nothing exists at runtime until
a driver implements it. :class:`~napalm_device_types.scheduled_jobs_posix.PosixScheduledJobsMixin`
is the implementation for a host reached through the command channel.
**Who owns a definition** is on every job (``managed_by``): ``package`` (installed
by one), ``admin`` (written on the host), ``platform`` (generated by the device's
own configuration -- OPNsense, OpenMediaVault, QTS, Windows), ``application`` (written
through :meth:`put_scheduled_job`, under the ``name`` the application gave it).
What the application makes of that is the application's.
**What may be done is on every job too** (``actions``): every job a driver can
switch takes ``enable``, ``disable`` and ``run``; only an ``application`` job takes
``edit`` and ``delete``. A driver never edits or removes a job it did not write.
**Never over a change it did not see.** Every write names the ``digest`` the caller
saw. The driver reads the job again right before it writes and raises
:class:`ScheduledJobChanged` when the definition differs.
"""
from __future__ import annotations
import hashlib
import json
import re
from typing import TYPE_CHECKING, Any, NotRequired, TypedDict
from napalm_device_types.cron import is_reboot, parse_cron
#: What :meth:`ScheduledJobsMixin.manage_scheduled_job` takes.
ACTIONS = ("enable", "disable", "run")
#: What a job's ``actions`` may list beyond :data:`ACTIONS`.
EDIT = "edit"
DELETE = "delete"
KINDS = ("cron", "run_parts", "anacron", "systemd_timer", "periodic", "task")
MANAGED_BY = ("package", "admin", "platform", "application")
#: Seconds a run now may take on the host before ``timeout`` stops it.
RUN_TIMEOUT = 300
#: A job's name becomes a file and a unit name: lower case, digits, dashes.
_NAME = re.compile(r"[a-z0-9][a-z0-9-]{0,62}")
#: A user name as useradd and the BSDs take it.
_USER = re.compile(r"[A-Za-z_][A-Za-z0-9_.-]{0,31}\$?")
#: The parts of a job that are its definition; what the host says of its runs is not.
_DEFINITION = ("kind", "source", "run_as", "schedule", "command", "enabled")
class ScheduledJobDict(TypedDict):
"""A job the host runs on its own schedule (``get_scheduled_jobs``).
A cron line, a run-parts script, an anacron job, a systemd timer, a FreeBSD
``periodic`` script, a platform's own job, a Windows task -- one shape for all
of them (``napalm_device_types.scheduled_jobs``).
"""
key: str # stable name on this host: "timer:apt-daily.timer", "cron:/etc/crontab:1a2b3c4d"
kind: str # cron | run_parts | anacron | systemd_timer | periodic | task
source: str # the file, unit file or configuration it is defined in
run_as: str # the user it runs as
schedule: str # as the host has it: "*/5 * * * *", "OnCalendar=daily", "daily"
schedule_kind: str # cron | calendar | monotonic | period | triggers
command: str # as the host has it
enabled: bool
managed_by: str # package | admin | platform | application
digest: str # job_digest(): the definition as it is now
actions: list[str] # what the driver can do to this job: enable, disable, run, edit, delete
name: NotRequired[str] # the name an application gave a job it wrote (put_scheduled_job)
description: NotRequired[str]
last_run: NotRequired[str | None] # ISO 8601, UTC; where the host knows it
last_result: NotRequired[str | None] # "success", or what the host says went wrong
next_run: NotRequired[str | None] # ISO 8601, UTC; where the host knows it
class ScheduledJobsReadingDict(TypedDict):
"""Every scheduled job of a host, and the time zone its schedules are in."""
jobs: list[ScheduledJobDict]
time_zone: str | None # IANA name ("Europe/Berlin"); None when the host does not say
class ScheduledJobSpecDict(TypedDict):
"""A job an application wants on the host (``put_scheduled_job``)."""
name: str # the job's name on the host, chosen by the application: [a-z0-9][a-z0-9-]*
schedule: str # a cron expression (napalm_device_types.cron), "@reboot" included
run_as: str
command: str # one or more lines of POSIX sh; a platform may take its own (OPNsense: configd)
enabled: bool
description: NotRequired[str]
class ScheduledJobResultDict(TypedDict):
"""What an action on a scheduled job did."""
success: bool
output: str # what the host said; for a run, what the job printed
class ScheduledJobChanged(Exception): # noqa: N818 - a state, as the callers word it
"""The job's definition differs from the digest the caller saw: nothing was written."""
class ScheduledJobRefused(ValueError):
"""The job cannot take what was asked of it, or the request itself is not valid."""
def job_digest(job: dict[str, Any], *, content: str = "") -> str:
"""The definition of *job* in 16 hex characters.
Over kind, source, run_as, schedule, command and enabled -- so enabling a job
changes it -- and *content*: what the definition points at that the job's
fields do not show, such as a run-parts script's checksum.
"""
parts = [job.get(field) for field in _DEFINITION] + [content]
return hashlib.sha256(json.dumps(parts).encode()).hexdigest()[:16]
def check_job_name(name: str) -> None:
""":raises ScheduledJobRefused: for a name a file or a unit could not carry safely."""
if not _NAME.fullmatch(name or ""):
raise ScheduledJobRefused(f"not a job name: {name!r} (a-z, 0-9 and -, at most 63)")
def check_spec(spec: ScheduledJobSpecDict) -> None:
""":raises ScheduledJobRefused: for a spec no host could be given as it is."""
check_job_name(spec["name"])
if not _USER.fullmatch(spec.get("run_as") or ""):
raise ScheduledJobRefused(f"not a user name: {spec.get('run_as')!r}")
if not (spec.get("command") or "").strip():
raise ScheduledJobRefused("a job needs a command")
if "\n" in (spec.get("description") or ""):
raise ScheduledJobRefused("a description is one line")
if not is_reboot(spec["schedule"]):
try:
parse_cron(spec["schedule"])
except ValueError as exc:
raise ScheduledJobRefused(str(exc)) from exc
class ScheduledJobsMixin:
"""Declares the scheduled-jobs contract; a driver that can implements it.
A verb a driver cannot do at all is ``None`` on the driver, as ``manage_service``
is; a verb one job cannot take is missing from that job's ``actions``.
"""
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
def get_scheduled_jobs(self) -> ScheduledJobsReadingDict:
"""Every scheduled job of the host.
:raises ValueError: when the reading came back cut short -- never a
shorter list.
"""
...
def manage_scheduled_job(
self, key: str, action: str, *, digest: str
) -> ScheduledJobResultDict:
"""Enable, disable or run the job *key* now.
:raises ScheduledJobChanged: when the job is no longer as *digest* says.
:raises ScheduledJobRefused: when the job does not take *action*.
:raises LookupError: when the host has no job *key*.
"""
...
def put_scheduled_job(
self, spec: ScheduledJobSpecDict, *, digest: str | None = None
) -> ScheduledJobResultDict:
"""Create the job ``spec["name"]`` (*digest* None) or change it.
:raises ScheduledJobChanged: when the job is no longer as *digest*
says, or when a job of that name is there although *digest* is None.
:raises ScheduledJobRefused: for a spec :func:`check_spec` refuses, or
one the host cannot take.
"""
...
def delete_scheduled_job(self, key: str, *, digest: str) -> ScheduledJobResultDict:
"""Delete the job *key*, which an application wrote.
:raises ScheduledJobChanged: when the job is no longer as *digest* says.
:raises ScheduledJobRefused: when the job is not one an application wrote.
:raises LookupError: when the host has no job *key*.
"""
...
+493
View File
@@ -0,0 +1,493 @@
"""Scheduled jobs on a POSIX host, over the command channel: cron and systemd timers.
:class:`PosixScheduledJobsMixin` implements
:class:`~napalm_device_types.scheduled_jobs.ScheduledJobsMixin` for a driver that has
:meth:`~napalm_device_types.channel.CommandChannelMixin.run_command`. The driver
mixes it in and sets the paths its host keeps jobs in; the defaults are Debian's.
**Reading** is one framed report as root (:mod:`napalm_device_types._scheduled_reading`).
**Switching** a cron or anacron line comments it out behind ``#napalm-disabled: ``
and takes exactly that marker away again; a run-parts script loses or gets its
execute bit; a timer is ``systemctl disable --now`` / ``enable --now``. A timer that
cannot be switched (``static``) only runs.
**Writing** stages the new text as the login user and installs it as root with one
fixed command: ``crontab -u <user> <file>`` for a user's crontab, ``install`` with
the file's own mode and owner for a system file. Root never reads stdin. Before any
write the job is read again and compared with the digest the caller saw.
**An application's own job** (:meth:`put_scheduled_job`) is a timer on a host with
systemd -- ``<name>.timer`` and ``<name>.service`` in :attr:`UNIT_DIR`, the command
in :attr:`JOB_SCRIPT_DIR` ``/<name>`` behind a marker and its cron expression --
else a file in :attr:`OWN_CRON_D_DIR`, else marked lines in the user's crontab.
**Running now** is bounded by :data:`~napalm_device_types.scheduled_jobs.RUN_TIMEOUT`:
a timer's unit by ``systemctl start`` and its journal since then, a cron line's
command as its user the way cron runs it (``_crontab.run_script``).
"""
from __future__ import annotations
import fnmatch
import shlex
from collections.abc import Sequence
from typing import TYPE_CHECKING, ClassVar
from napalm_device_types._crontab import (
JOB_MARKER,
CronLine,
marked_lines,
parse_anacrontab,
parse_crontab,
run_script,
switched,
with_job,
without_job,
)
from napalm_device_types._scheduled_reading import (
OWNERS_SH,
SCHEDULE_HEADER,
TIME_ZONE_SH,
Paths,
Reading,
parse_reading,
reading_command,
)
from napalm_device_types.cron import is_reboot, to_on_calendar
from napalm_device_types.scheduled_jobs import (
ACTIONS,
RUN_TIMEOUT,
ScheduledJobChanged,
ScheduledJobRefused,
ScheduledJobResultDict,
ScheduledJobsMixin,
ScheduledJobSpecDict,
ScheduledJobsReadingDict,
check_spec,
)
if TYPE_CHECKING:
from napalm_device_types.channel import CommandResult
_STAGE = 't=$(mktemp) && chmod 644 "$t" && cat > "$t" && printf %s "$t"'
#: What a run keeps of the job's output.
_OUTPUT_LIMIT = 64 * 1024
_READ_TIMEOUT = 120
def _command(argv: Sequence[str]) -> str:
return shlex.join(argv)
def _said(result: CommandResult) -> str:
return "\n".join(p for p in (result.stdout.strip(), result.stderr.strip()) if p)[
-_OUTPUT_LIMIT:
]
def _result(result: CommandResult, done: str) -> ScheduledJobResultDict:
if result.exit_code != 0:
return {"success": False, "output": _said(result) or f"exit {result.exit_code}"}
return {"success": True, "output": _said(result) or done}
class PosixScheduledJobsMixin(ScheduledJobsMixin):
"""Scheduled jobs of a host reached through ``run_command``; the paths are the driver's."""
CRON_SYSTEM_FILES: ClassVar[tuple[str, ...]] = ("/etc/crontab",)
CRON_D_DIRS: ClassVar[tuple[str, ...]] = ("/etc/cron.d",)
CRON_SPOOL_DIRS: ClassVar[tuple[str, ...]] = (
"/var/spool/cron/crontabs",
"/var/spool/cron",
"/etc/crontabs",
)
RUN_PARTS_DIRS: ClassVar[tuple[tuple[str, str], ...]] = (
("hourly", "/etc/cron.hourly"),
("daily", "/etc/cron.daily"),
("weekly", "/etc/cron.weekly"),
("monthly", "/etc/cron.monthly"),
)
ANACRONTAB: ClassVar[str] = "/etc/anacrontab"
JOB_SCRIPT_DIR: ClassVar[str] = "/usr/local/libexec/scheduled-jobs"
UNIT_DIR: ClassVar[str] = "/etc/systemd/system"
#: Where a host without systemd takes an application's job as a file, if the directory
#: is there; else, or None, the user's crontab.
OWN_CRON_D_DIR: ClassVar[str | None] = "/etc/cron.d"
ROOT_GROUP: ClassVar[str] = "root"
#: Prints "<octal mode> <owner> <group>" of the path appended.
STAT_ARGV: ClassVar[tuple[str, ...]] = ("stat", "-c", "%a %U %G", "--")
#: ``install`` and how it makes missing directories (FreeBSD's ``-D`` is a destdir).
INSTALL: ClassVar[tuple[str, ...]] = ("install", "-D")
#: ``sh`` that reads definition paths on stdin and prints ``<package>: <path>``.
OWNERS_SH: ClassVar[str] = OWNERS_SH
#: ``sh`` that prints the time zone: an IANA name, or a path ending in ``zoneinfo/<name>``.
TIME_ZONE_SH: ClassVar[str] = TIME_ZONE_SH
#: ``fnmatch`` patterns on a job's ``source``: what the platform generates is ``platform``.
PLATFORM_SOURCES: ClassVar[tuple[str, ...]] = ()
#: Run as root after a user's crontab was written; for a host that keeps it elsewhere too.
CRONTAB_AFTER_WRITE: ClassVar[tuple[str, ...] | None] = None
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
def run_command(
self,
command: str,
*,
privileged: bool = False,
timeout: float = 60,
stdin: bytes | None = None,
) -> CommandResult: ...
# -- reading ---------------------------------------------------------------------------------
def _sj_paths(self) -> Paths:
return Paths(
system_files=self.CRON_SYSTEM_FILES,
cron_d_dirs=self.CRON_D_DIRS,
spool_dirs=self.CRON_SPOOL_DIRS,
run_parts=self.RUN_PARTS_DIRS,
anacrontab=self.ANACRONTAB,
script_dir=self.JOB_SCRIPT_DIR,
owners_sh=self.OWNERS_SH,
time_zone_sh=self.TIME_ZONE_SH,
)
def _sj_read(self) -> Reading:
result = self.run_command(
reading_command(self._sj_paths()), privileged=True, timeout=_READ_TIMEOUT
)
reading = parse_reading(result.stdout)
for job in reading.jobs:
source = str(job["source"])
if job["managed_by"] != "application" and any(
fnmatch.fnmatchcase(source, p) for p in self.PLATFORM_SOURCES
):
job["managed_by"] = "platform"
return reading
def get_scheduled_jobs(self) -> ScheduledJobsReadingDict:
"""Every scheduled job of the host, in one round trip.
:raises ValueError: when the report came back cut short.
"""
reading = self._sj_read()
return {"jobs": reading.jobs, "time_zone": reading.time_zone} # type: ignore[typeddict-item]
def _sj_job(self, reading: Reading, key: str, digest: str) -> dict[str, object]:
job = next((j for j in reading.jobs if j["key"] == key), None)
if job is None:
raise LookupError(f"no scheduled job {key}")
if job["digest"] != digest:
raise ScheduledJobChanged(f"{key} changed on the host since it was read")
return job
# -- commands --------------------------------------------------------------------------------
def _sj_root(self, argv: Sequence[str], timeout: float = 60) -> CommandResult:
return self.run_command(_command(argv), privileged=True, timeout=timeout)
def _sj_checked(self, argv: Sequence[str]) -> CommandResult:
result = self._sj_root(argv)
if result.exit_code != 0:
raise RuntimeError(f"{_command(argv)}: {_said(result) or result.exit_code}")
return result
def _sj_stage(self, content: str) -> str:
result = self.run_command(_STAGE, timeout=30, stdin=content.encode())
path = result.stdout.strip().splitlines()[-1:] or [""]
if result.exit_code != 0 or not path[0].startswith("/"):
raise RuntimeError(f"cannot stage a file: {_said(result)}")
return path[0]
def _sj_install(self, content: str, path: str, mode: str, owner: str, group: str) -> None:
staged = self._sj_stage(content)
try:
self._sj_checked(
[*self.INSTALL, "-m", mode, "-o", owner, "-g", group, "--", staged, path]
)
finally:
self.run_command(_command(["rm", "-f", "--", staged]), timeout=30)
# -- crontab files ---------------------------------------------------------------------------
def _sj_spool_user(self, path: str) -> str | None:
"""The user whose crontab *path* is; None for a system file."""
directory, _, user = path.rpartition("/")
return user if directory in self.CRON_SPOOL_DIRS else None
def _sj_crontab(self, user: str) -> str:
result = self._sj_root(["crontab", "-u", user, "-l"])
if result.exit_code != 0:
if "no crontab" in (result.stderr + result.stdout).lower():
return ""
raise RuntimeError(f"cannot read the crontab of {user}: {_said(result)}")
return result.stdout
def _sj_text(self, path: str) -> str:
user = self._sj_spool_user(path)
if user is not None:
return self._sj_crontab(user)
return self._sj_checked(["cat", "--", path]).stdout
def _sj_write_crontab(self, user: str, text: str) -> None:
staged = self._sj_stage(text)
try:
self._sj_checked(["crontab", "-u", user, staged])
finally:
self.run_command(_command(["rm", "-f", "--", staged]), timeout=30)
if self.CRONTAB_AFTER_WRITE:
self._sj_checked(self.CRONTAB_AFTER_WRITE)
def _sj_write(self, path: str, text: str) -> None:
user = self._sj_spool_user(path)
if user is not None:
self._sj_write_crontab(user, text)
return
# Through a symbolic link to the file it names, with that file's mode: the
# link's own mode is 777, and cron ignores a file anyone may write.
real = self._sj_checked(["readlink", "-f", "--", path]).stdout.strip() or path
mode, owner, group = self._sj_checked([*self.STAT_ARGV, real]).stdout.split()[:3]
self._sj_install(text, real, mode, owner, group)
def _sj_line(self, job: dict[str, object], text: str) -> CronLine:
"""The job's line in *text* as it is now, read again for the write."""
path = str(job["source"])
if job["kind"] == "anacron":
found = parse_anacrontab(text)
else:
user = self._sj_spool_user(path)
found = parse_crontab(path, text, system=user is None, user=user or "")
line = next((ln for ln in found if ln.key == job["key"]), None)
if line is None:
raise ScheduledJobChanged(f"{job['key']} is no longer in {path}")
return line
# -- switching and running -------------------------------------------------------------------
def _sj_timer_unit(self, job: dict[str, object]) -> str:
key = str(job["key"])
return f"{job['name']}.timer" if key.startswith("job:") else key.removeprefix("timer:")
def _sj_switch(self, job: dict[str, object], enabled: bool) -> ScheduledJobResultDict:
word = "enabled" if enabled else "disabled"
if job["kind"] == "systemd_timer":
verb = "enable" if enabled else "disable"
return _result(
self._sj_root(["systemctl", verb, "--now", "--", self._sj_timer_unit(job)]), word
)
if job["kind"] == "run_parts":
return _result(
self._sj_root(["chmod", "+x" if enabled else "-x", "--", str(job["source"])]), word
)
path = str(job["source"])
text = self._sj_text(path)
self._sj_write(path, switched(text, self._sj_line(job, text), enabled=enabled))
return {"success": True, "output": f"{job['key']} {word}"}
def _sj_as(self, user: str, script: str) -> list[str]:
"""*script* run by ``sh`` as *user*, from root."""
if user == "root":
return ["sh", "-c", script]
return ["su", "-s", "/bin/sh", user, "-c", script]
def _sj_run(self, job: dict[str, object], reading: Reading) -> ScheduledJobResultDict:
bound = ["timeout", str(RUN_TIMEOUT)]
wait = RUN_TIMEOUT + 30
if job["kind"] == "systemd_timer":
unit = reading.units[str(job["key"])]
since = self.run_command("date +%s", timeout=30).stdout.strip()
started = self._sj_root([*bound, "systemctl", "start", "--", unit], timeout=wait)
journal = self._sj_root(
[
"journalctl",
"-u",
unit,
"--since",
f"@{since}",
"--no-pager",
"-o",
"cat",
"-n",
"200",
]
)
output = "\n".join(p for p in (_said(started), journal.stdout.strip()) if p)
return {"success": started.exit_code == 0, "output": output or f"{unit} ran"}
if job["kind"] == "run_parts":
return _result(self._sj_root([*bound, str(job["source"])], timeout=wait), "ran")
path = str(job["source"])
line = self._sj_line(job, self._sj_text(path))
script = run_script(line.command, line.env)
return _result(
self._sj_root([*bound, *self._sj_as(line.run_as, script)], timeout=wait), "ran"
)
def manage_scheduled_job(self, key: str, action: str, *, digest: str) -> ScheduledJobResultDict:
"""Enable, disable or run the job *key* now; see :class:`ScheduledJobsMixin`."""
if action not in ACTIONS:
raise ScheduledJobRefused(f"unknown action {action!r}; allowed: {list(ACTIONS)}")
reading = self._sj_read()
job = self._sj_job(reading, key, digest)
if action not in job["actions"]: # type: ignore[operator]
raise ScheduledJobRefused(f"{key} cannot be told to {action} on this host")
if action == "run":
return self._sj_run(job, reading)
return self._sj_switch(job, action == "enable")
# -- an application's own jobs ---------------------------------------------------------------
def _sj_own(
self, reading: Reading, spec: ScheduledJobSpecDict, digest: str | None
) -> dict[str, object] | None:
current = next((j for j in reading.jobs if j["key"] == f"job:{spec['name']}"), None)
if digest is None:
if current is not None:
raise ScheduledJobChanged(f"a job {spec['name']} is there already")
return None
if current is None:
raise LookupError(f"no scheduled job job:{spec['name']}")
if current["digest"] != digest:
raise ScheduledJobChanged(f"job:{spec['name']} changed on the host since it was read")
return current
def _sj_group(self, user: str) -> str:
result = self.run_command(_command(["id", "-gn", "--", user]), timeout=30)
if result.exit_code != 0:
raise ScheduledJobRefused(f"the host has no user {user}")
return result.stdout.strip()
def _sj_calendar(self, schedule: str) -> list[str]:
if is_reboot(schedule):
return ["OnBootSec=1min"]
lines = to_on_calendar(schedule)
for line in lines:
checked = self.run_command(
_command(["systemd-analyze", "calendar", "--", line]), timeout=30
)
if checked.exit_code != 0:
raise ScheduledJobRefused(f"systemd does not take {line!r}: {_said(checked)}")
return [f"OnCalendar={line}" for line in lines]
def _sj_put_timer(self, spec: ScheduledJobSpecDict) -> None:
name, description = spec["name"], spec.get("description") or spec["name"]
script = f"{self.JOB_SCRIPT_DIR}/{name}"
unit_description = description.replace("%", "%%")
timer = "\n".join(
[
"[Unit]",
f"Description={unit_description}",
"",
"[Timer]",
*self._sj_calendar(spec["schedule"]),
f"Unit={name}.service",
"",
"[Install]",
"WantedBy=timers.target",
"",
]
)
service = "\n".join(
[
"[Unit]",
f"Description={unit_description}",
"",
"[Service]",
"Type=oneshot",
f"User={spec['run_as']}",
f"ExecStart={script}",
"",
]
)
body = f"#!/bin/sh\n{JOB_MARKER}{name} {spec.get('description') or ''}".rstrip()
body += f"\n{SCHEDULE_HEADER}{spec['schedule']}\n{spec['command'].strip()}\n"
self._sj_install(body, script, "750", "root", self._sj_group(spec["run_as"]))
self._sj_install(service, f"{self.UNIT_DIR}/{name}.service", "644", "root", self.ROOT_GROUP)
self._sj_install(timer, f"{self.UNIT_DIR}/{name}.timer", "644", "root", self.ROOT_GROUP)
self._sj_checked(["systemctl", "daemon-reload"])
verb = "enable" if spec["enabled"] else "disable"
self._sj_checked(["systemctl", verb, "--now", "--", f"{name}.timer"])
def _sj_cron_text(self, spec: ScheduledJobSpecDict, *, system: bool) -> list[str]:
if "\n" in spec["command"].strip():
raise ScheduledJobRefused("a cron line holds one line of command")
owner = f" {spec['run_as']}" if system else ""
line = f"{spec['schedule']}{owner} {spec['command'].strip()}"
return marked_lines(
spec["name"], spec.get("description") or "", line, enabled=spec["enabled"]
)
def _sj_cron_d(self) -> str | None:
""":attr:`OWN_CRON_D_DIR` where the host has it; BusyBox crond reads none."""
directory = self.OWN_CRON_D_DIR
if directory is None:
return None
found = self.run_command(_command(["test", "-d", directory]), timeout=30)
return directory if found.exit_code == 0 else None
def _sj_put_cron(self, spec: ScheduledJobSpecDict, current: dict[str, object] | None) -> None:
if self._sj_cron_d() is not None:
self._sj_group(spec["run_as"])
text = "\n".join(self._sj_cron_text(spec, system=True)) + "\n"
self._sj_install(
text, f"{self.OWN_CRON_D_DIR}/{spec['name']}", "644", "root", self.ROOT_GROUP
)
return
if current is not None and current["run_as"] != spec["run_as"]:
self._sj_delete_cron(current)
user = spec["run_as"]
text = self._sj_crontab(user)
jobs = parse_crontab("", text, system=False, user=user)
self._sj_write_crontab(
user, with_job(text, jobs, spec["name"], self._sj_cron_text(spec, system=False))
)
def put_scheduled_job(
self, spec: ScheduledJobSpecDict, *, digest: str | None = None
) -> ScheduledJobResultDict:
"""Create or change the application's own job; see :class:`ScheduledJobsMixin`."""
check_spec(spec)
reading = self._sj_read()
current = self._sj_own(reading, spec, digest)
if reading.systemd:
self._sj_put_timer(spec)
else:
self._sj_put_cron(spec, current)
return {
"success": True,
"output": f"job:{spec['name']} {'changed' if current else 'created'}",
}
def _sj_delete_cron(self, job: dict[str, object]) -> None:
path = str(job["source"])
if self._sj_spool_user(path) is None:
self._sj_checked(["rm", "-f", "--", path])
return
text = self._sj_text(path)
jobs = parse_crontab(path, text, system=False, user=str(job["run_as"]))
self._sj_write(path, without_job(text, jobs, str(job["name"])))
def delete_scheduled_job(self, key: str, *, digest: str) -> ScheduledJobResultDict:
"""Delete the application's job *key*; see :class:`ScheduledJobsMixin`."""
job = self._sj_job(self._sj_read(), key, digest)
if job["managed_by"] != "application":
raise ScheduledJobRefused(
f"{key} was not written by an application; it is only switched"
)
name = str(job["name"])
if job["kind"] != "systemd_timer":
self._sj_delete_cron(job)
return {"success": True, "output": f"{key} deleted"}
self._sj_root(["systemctl", "disable", "--now", "--", f"{name}.timer"])
self._sj_checked(
[
"rm",
"-f",
"--",
f"{self.UNIT_DIR}/{name}.timer",
f"{self.UNIT_DIR}/{name}.service",
f"{self.JOB_SCRIPT_DIR}/{name}",
]
)
self._sj_checked(["systemctl", "daemon-reload"])
return {"success": True, "output": f"{key} deleted"}
+18 -2
View File
@@ -15,7 +15,9 @@ accident.
``/etc/ssh/sshd_config.d/*.conf`` -- Debian and Ubuntu do, at the top -- the
lists go to :data:`DROPIN_PATH`, which sorts first and so wins. sshd_config
stays as the distribution shipped it, and removing the file undoes the change.
Only without such an ``Include`` is sshd_config itself rewritten, after a copy.
Only without such an ``Include`` is sshd_config itself rewritten, after a copy --
and not at all on a platform that generates it
(:attr:`OpenSshServerMixin.SSHD_CONFIG_GENERATED`).
**Nobody is locked out, by three checks.**
@@ -426,9 +428,16 @@ class OpenSshServerMixin:
whose platform writes sshd_config itself (an appliance that regenerates it
from its own database) must not inherit this, or must opt out with
``harden_ssh_server = None``: its file would be overwritten on the next
apply.
apply. Where that generated file still includes the drop-in directory, the
driver sets :attr:`SSHD_CONFIG_GENERATED` instead and keeps the hardening.
"""
#: True where the platform writes sshd_config from its own database
#: (OpenMediaVault): only the drop-in directory is written, and a
#: sshd_config that does not include it is left alone, since the platform
#: would write over a change to it on its next apply.
SSHD_CONFIG_GENERATED: bool = False
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
def run_command(
@@ -565,6 +574,13 @@ class OpenSshServerMixin:
main = self._ssh_read(MAIN_CONFIG)
if main is None:
raise RuntimeError(f"cannot read {MAIN_CONFIG}")
if self.SSHD_CONFIG_GENERATED and not has_dropin_include(main):
result.message = (
f"Not changed: {MAIN_CONFIG} does not include {DROPIN_DIR}, and this"
" platform writes it from its own settings."
)
say(result.message)
return result
restore = self._write_change(plan, main, result, say)
return self._apply_or_restore(
result,
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "napalm-device-types"
version = "4.23.0"
version = "4.27.0"
description = "Abstract device-type base classes for NAPALM drivers"
readme = "README.md"
requires-python = ">=3.12"
+1 -1
View File
@@ -32,7 +32,7 @@
},
"mypy_relaxed": [],
"python_file_lines": {
"napalm_device_types/models.py": 1463
"napalm_device_types/models.py": 1454
},
"untyped_public": []
}
+234
View File
@@ -0,0 +1,234 @@
"""A backup server is a role of its own (NAPALM/napalm-device-types#82).
What makes a device a backup server -- datastores, the backup groups in them,
the jobs that keep them (garbage collection, verify, prune, sync, tape) and the
remotes it syncs with -- has no counterpart in ``StorageDriver``. A backup
server that also manages its own disks lists ``StorageDriver`` as a second role.
The declarations sit under ``TYPE_CHECKING`` (see test_role_contracts), so the
signatures are read from the source rather than from the class.
"""
from __future__ import annotations
import ast
import inspect
from typing import NotRequired, get_args, get_type_hints
import pytest
import napalm_device_types.backup_server as backup_server_module
from napalm_device_types import (
BACKUP_JOB_KINDS,
BackupDatastoreDict,
BackupGroupDict,
BackupRemoteDict,
BackupServerDriver,
BackupServerJobDict,
BackupServerRefDict,
BackupSnapshotDict,
BackupSnapshotRefDict,
BackupTaskDict,
BackupTaskStatusDict,
ClusterStorageDict,
OSDriver,
StorageDriver,
)
from napalm_device_types.backups import BackupJobKind, BackupRunState, BackupVerifyState
from napalm_device_types.roles import primary_role_of, role_keys_of
def _declared(name: str) -> ast.FunctionDef:
tree = ast.parse(inspect.getsource(backup_server_module))
for node in ast.walk(tree):
if isinstance(node, ast.FunctionDef) and node.name == name:
return node
raise AssertionError(f"BackupServerDriver does not declare {name}()")
def _signature(name: str) -> tuple[list[str], str]:
fn = _declared(name)
params = ast.unparse(fn.args).split(", ")[1:]
return params, ast.unparse(fn.returns) if fn.returns else ""
class TestRole:
def test_role_key_and_label(self):
assert vars(BackupServerDriver)["ROLE"] == "backup_server"
assert BackupServerDriver.TYPE_LABEL == "Backup Server"
def test_backup_server_first_storage_second(self):
"""A PBS manages its disks and ZFS pools too; that part is storage."""
class Pbs(BackupServerDriver, StorageDriver, OSDriver):
pass
assert role_keys_of(Pbs) == ["backup_server", "storage", "linux"]
assert primary_role_of(Pbs) == "backup_server"
class TestDeclarations:
@pytest.mark.parametrize(
("name", "params", "returns"),
[
("get_backup_datastores", [], "list[BackupDatastoreDict]"),
("get_backup_groups", ["datastore: str=''"], "list[BackupGroupDict]"),
(
"get_backup_snapshots",
["datastore: str", "namespace: str", "group_type: str", "group_id: str"],
"list[BackupSnapshotDict]",
),
("get_backup_jobs", [], "list[BackupServerJobDict]"),
("get_backup_remotes", [], "list[BackupRemoteDict]"),
("start_garbage_collection", ["datastore: str"], "BackupTaskDict"),
("run_backup_job", ["kind: BackupJobKind", "job_id: str"], "BackupTaskDict"),
("verify_backup_snapshot", ["snapshot: BackupSnapshotRefDict"], "BackupTaskDict"),
(
"set_backup_snapshot_protected",
["snapshot: BackupSnapshotRefDict", "protected: bool"],
"None",
),
("delete_backup_snapshot", ["snapshot: BackupSnapshotRefDict"], "None"),
("get_backup_task", ["task: str"], "BackupTaskStatusDict"),
],
)
def test_declared_with_its_signature(self, name, params, returns):
got_params, got_returns = _signature(name)
assert [p.replace(" ", "") for p in got_params] == [p.replace(" ", "") for p in params]
assert got_returns == returns
@pytest.mark.parametrize(
"name",
[
"get_backup_datastores",
"get_backup_groups",
"get_backup_snapshots",
"get_backup_jobs",
"get_backup_remotes",
"start_garbage_collection",
"run_backup_job",
"verify_backup_snapshot",
"set_backup_snapshot_protected",
"delete_backup_snapshot",
"get_backup_task",
],
)
def test_absent_until_a_driver_implements_it(self, name):
assert not hasattr(BackupServerDriver, name)
def test_every_declaration_has_a_docstring(self):
tree = ast.parse(inspect.getsource(backup_server_module))
missing = [
n.name
for n in ast.walk(tree)
if isinstance(n, ast.FunctionDef) and not ast.get_docstring(n)
]
assert missing == []
class TestShapes:
"""Neutral names: nothing here is Proxmox's. A value the platform does not
report is None; it is never guessed."""
def test_datastore(self):
assert set(get_type_hints(BackupDatastoreDict)) == {
"name",
"path",
"backend",
"total",
"used",
"available",
"estimated_full",
"maintenance",
"comment",
}
def test_group_carries_its_newest_snapshot_not_all_of_them(self):
assert set(get_type_hints(BackupGroupDict)) == {
"datastore",
"namespace",
"type",
"id",
"owner",
"count",
"newest",
"oldest_time",
"comment",
}
assert get_type_hints(BackupGroupDict)["newest"] == BackupSnapshotDict | None
def test_snapshot(self):
assert set(get_type_hints(BackupSnapshotDict)) == {
"time",
"size",
"verify",
"protected",
"comment",
}
assert set(get_args(BackupVerifyState)) == {"ok", "failed"}
def test_snapshot_reference_names_one_snapshot(self):
assert set(get_type_hints(BackupSnapshotRefDict)) == {
"datastore",
"namespace",
"group_type",
"group_id",
"time",
}
def test_job_kinds_include_garbage_collection(self):
"""GC is a job with a schedule and a last run like the others, so it is
listed with them rather than restated on the datastore."""
assert set(get_args(BackupJobKind)) == {"gc", "verify", "prune", "sync", "tape"}
assert set(BACKUP_JOB_KINDS) == set(get_args(BackupJobKind))
def test_job(self):
assert set(get_type_hints(BackupServerJobDict)) == {
"id",
"kind",
"datastore",
"namespace",
"schedule",
"enabled",
"last_run",
"last_state",
"last_error",
"next_run",
"remote",
"direction",
"comment",
}
assert set(get_args(BackupRunState)) == {"ok", "warning", "error", "running"}
def test_remote_never_carries_its_secret(self):
fields = set(get_type_hints(BackupRemoteDict))
assert fields == {"name", "host", "port", "fingerprint", "auth_id", "comment"}
assert not fields & {"password", "secret", "token"}
def test_task(self):
assert set(get_type_hints(BackupTaskDict)) == {"task"}
assert set(get_type_hints(BackupTaskStatusDict)) == {
"task",
"running",
"state",
"error",
"log",
}
class TestClusterStorageNamesItsBackupServer:
"""A hypervisor's backup storage says which server and datastore it points
at, so an application can tie the server's groups to the cluster's guests."""
def test_backup_server_reference_is_optional_on_a_storage(self):
hints = get_type_hints(ClusterStorageDict, include_extras=True)
assert hints["backup_server"] == NotRequired[BackupServerRefDict | None]
def test_reference(self):
assert set(get_type_hints(BackupServerRefDict)) == {
"server",
"port",
"datastore",
"namespace",
"fingerprint",
}
+6 -1
View File
@@ -25,6 +25,7 @@ import napalm_device_types.hypervisor as hypervisor_module
from napalm_device_types import (
BackupDict,
BackupJobDict,
BackupServerRefDict,
ClusterDict,
ClusterNetworkDict,
ClusterNodeDict,
@@ -92,7 +93,8 @@ class TestClusterShape:
assert "votes" not in ClusterNodeDict.__required_keys__
def test_storage(self):
assert get_type_hints(ClusterStorageDict) == {
hints = get_type_hints(ClusterStorageDict, include_extras=True)
assert hints == {
"name": str,
"type": str,
"shared": bool,
@@ -100,7 +102,10 @@ class TestClusterShape:
"nodes": list[str],
"total": int | None,
"used": int | None,
"backup_server": NotRequired[BackupServerRefDict | None],
}
# A driver written before backup servers were part of the report stays valid.
assert "backup_server" not in ClusterStorageDict.__required_keys__
def test_network(self):
hints = get_type_hints(ClusterNetworkDict, include_extras=True)
+141
View File
@@ -0,0 +1,141 @@
"""Cron expressions: what they mean, when they fire next, and as systemd ``OnCalendar``.
One module answers all three, because a job netOrk writes as a timer must fire
exactly when the cron expression the user gave says, and the preview must show
the same runs the host will make (NetOrk/netork#1933).
"""
from __future__ import annotations
from datetime import UTC, datetime
from zoneinfo import ZoneInfo
import pytest
from napalm_device_types.cron import is_reboot, next_runs, parse_cron, to_on_calendar
BERLIN = ZoneInfo("Europe/Berlin")
class TestParse:
def test_every_field_is_expanded_to_its_values(self):
s = parse_cron("*/15 6-8 1,15 * *")
assert s.minutes == frozenset({0, 15, 30, 45})
assert s.hours == frozenset({6, 7, 8})
assert s.days == frozenset({1, 15})
assert s.months == frozenset(range(1, 13))
assert s.weekdays == frozenset(range(7))
def test_a_range_takes_a_step(self):
assert parse_cron("1-30/10 * * * *").minutes == frozenset({1, 11, 21})
def test_names_are_months_and_weekdays_in_any_case(self):
s = parse_cron("0 0 * Jan,MAR mon-Fri")
assert s.months == frozenset({1, 3})
assert s.weekdays == frozenset({1, 2, 3, 4, 5})
def test_seven_is_sunday_too(self):
assert parse_cron("0 0 * * 7").weekdays == frozenset({0})
@pytest.mark.parametrize(
("alias", "expanded"),
[
("@hourly", "0 * * * *"),
("@daily", "0 0 * * *"),
("@midnight", "0 0 * * *"),
("@weekly", "0 0 * * 0"),
("@monthly", "0 0 1 * *"),
("@yearly", "0 0 1 1 *"),
("@annually", "0 0 1 1 *"),
],
)
def test_an_alias_is_its_expression(self, alias, expanded):
assert parse_cron(alias) == parse_cron(expanded)
def test_reboot_is_no_calendar(self):
assert is_reboot("@reboot")
assert not is_reboot("0 0 * * *")
with pytest.raises(ValueError, match="reboot"):
parse_cron("@reboot")
@pytest.mark.parametrize(
"bad",
[
"",
"* * * *",
"* * * * * *",
"60 * * * *",
"* 24 * * *",
"* * 0 * *",
"* * * 13 *",
"* * * * 8",
"*/0 * * * *",
"5-1 * * * *",
"a * * * *",
"@often",
"1,,2 * * * *",
],
)
def test_what_cron_would_not_take_is_refused(self, bad):
with pytest.raises(ValueError):
parse_cron(bad)
def test_day_of_month_and_weekday_are_or_only_when_both_are_restricted(self):
assert parse_cron("0 0 1 * 1").either_day
assert not parse_cron("0 0 */2 * 1").either_day
assert not parse_cron("0 0 1 * *").either_day
class TestNextRuns:
def test_runs_after_the_instant_in_the_hosts_time(self):
after = datetime(2026, 11, 3, 5, 50, tzinfo=UTC) # 06:50 in Berlin
runs = next_runs("0 7 * * *", after, BERLIN, count=2)
assert runs == [
datetime(2026, 11, 3, 7, 0, tzinfo=BERLIN),
datetime(2026, 11, 4, 7, 0, tzinfo=BERLIN),
]
def test_the_instant_itself_is_not_a_next_run(self):
after = datetime(2026, 11, 3, 6, 0, tzinfo=UTC)
assert next_runs("0 6 * * *", after, ZoneInfo("UTC"), count=1)[0].day == 4
def test_either_day_fires_on_both(self):
after = datetime(2026, 11, 30, 12, 0, tzinfo=UTC) # a Monday
runs = next_runs("0 0 1 * 1", after, ZoneInfo("UTC"), count=3)
assert [r.day for r in runs] == [1, 7, 14]
def test_a_day_that_never_comes_gives_no_runs(self):
assert next_runs("0 0 31 2 *", datetime(2026, 1, 1, tzinfo=UTC), BERLIN) == []
def test_a_time_the_clock_skips_is_left_out(self):
# 2026-03-29 02:30 does not exist in Berlin.
after = datetime(2026, 3, 28, 12, 0, tzinfo=UTC)
runs = next_runs("30 2 * * *", after, BERLIN, count=2)
assert [r.day for r in runs] == [30, 31]
def test_reboot_has_no_next_run(self):
assert next_runs("@reboot", datetime(2026, 1, 1, tzinfo=UTC), BERLIN) == []
class TestOnCalendar:
def test_every_field_as_its_values(self):
assert to_on_calendar("*/15 6 * * *") == ["*-*-* 06:00,15,30,45:00"]
def test_a_restricted_weekday_leads(self):
assert to_on_calendar("30 4 * * 1-5") == ["Mon,Tue,Wed,Thu,Fri *-*-* 04:30:00"]
def test_months_and_days(self):
assert to_on_calendar("0 0 1,15 1,7 *") == ["*-01,07-01,15 00:00:00"]
def test_either_day_is_two_lines(self):
assert to_on_calendar("0 0 1 * 0") == ["*-*-01 00:00:00", "Sun *-*-* 00:00:00"]
def test_both_days_with_a_star_are_one_line(self):
assert to_on_calendar("0 0 */10 * 6") == ["Sat *-*-01,11,21,31 00:00:00"]
def test_an_alias(self):
assert to_on_calendar("@daily") == ["*-*-* 00:00:00"]
def test_reboot_is_refused(self):
with pytest.raises(ValueError, match="reboot"):
to_on_calendar("@reboot")
+145
View File
@@ -0,0 +1,145 @@
"""Crontabs as text: the jobs in them, and changing one line (NetOrk/netork#1933)."""
from __future__ import annotations
import subprocess
import pytest
from napalm_device_types._crontab import (
DISABLED_MARKER,
marked_lines,
parse_anacrontab,
parse_crontab,
run_script,
switched,
with_job,
without_job,
)
SYSTEM = """\
# /etc/crontab: system-wide crontab
SHELL=/bin/sh
PATH="/usr/local/sbin:/usr/bin:/bin"
17 * * * * root cd / && run-parts --report /etc/cron.hourly
@reboot root /usr/local/bin/warm-cache # keeps the # in the command
#napalm-disabled: 0 4 * * * backup /usr/local/bin/backup
garbage line
0 4 * * * backup /usr/local/bin/backup
"""
USER = """\
MAILTO=ops@example.org
*/5 * * * * /usr/bin/php /var/www/cron.php
# napalm-scheduled-job: netork-cleanup tidy the cache
0 3 * * * find /tmp -mtime +7 -delete
"""
class TestParse:
def test_a_system_crontab_names_the_user(self):
jobs = parse_crontab("/etc/crontab", SYSTEM, system=True)
first = jobs[0]
assert (first.schedule, first.run_as) == ("17 * * * *", "root")
assert first.command == "cd / && run-parts --report /etc/cron.hourly"
assert first.env == {"SHELL": "/bin/sh", "PATH": "/usr/local/sbin:/usr/bin:/bin"}
assert first.key.startswith("cron:/etc/crontab:")
def test_an_alias_and_a_hash_inside_the_command(self):
reboot = parse_crontab("/etc/crontab", SYSTEM, system=True)[1]
assert reboot.schedule == "@reboot"
assert reboot.command == "/usr/local/bin/warm-cache # keeps the # in the command"
def test_a_line_switched_off_is_a_job_that_is_off(self):
jobs = parse_crontab("/etc/crontab", SYSTEM, system=True)
off = jobs[2]
assert not off.enabled
assert off.run_as == "backup"
def test_equal_lines_get_their_own_keys_and_switching_off_keeps_the_key(self):
jobs = parse_crontab("/etc/crontab", SYSTEM, system=True)
assert jobs[3].key == jobs[2].key + ":2"
assert len(jobs) == 4 # the garbage line is left out
def test_a_user_crontab_runs_as_its_user(self):
jobs = parse_crontab("/var/spool/cron/crontabs/www", USER, system=False, user="www-data")
assert jobs[0].run_as == "www-data"
assert jobs[0].env == {"MAILTO": "ops@example.org"}
def test_an_applications_job_is_keyed_by_its_name(self):
job = parse_crontab("/x", USER, system=False, user="root")[1]
assert (job.key, job.name, job.description) == (
"job:netork-cleanup",
"netork-cleanup",
"tidy the cache",
)
def test_an_anacrontab(self):
text = "SHELL=/bin/sh\n1\t5\tcron.daily\trun-parts --report /etc/cron.daily\n@monthly 15 cron.monthly run-parts /etc/cron.monthly\nbad line here\n"
jobs = parse_anacrontab(text)
assert [(j.key, j.schedule, j.run_as) for j in jobs] == [
("anacron:cron.daily", "1", "root"),
("anacron:cron.monthly", "@monthly", "root"),
]
assert jobs[0].command == "run-parts --report /etc/cron.daily"
class TestEdit:
def test_switching_off_and_on_again_gives_the_text_back(self):
jobs = parse_crontab("/etc/crontab", SYSTEM, system=True)
off = switched(SYSTEM, jobs[0], enabled=False)
assert DISABLED_MARKER + "17 *\t* * *\troot" in off
again = parse_crontab("/etc/crontab", off, system=True)
assert again[0].key == jobs[0].key and not again[0].enabled
assert switched(off, again[0], enabled=True) == SYSTEM.replace(
"17 *\t* * *\troot\tcd", "17 *\t* * *\troot\tcd"
)
def test_an_applications_job_is_added_replaced_and_removed(self):
lines = marked_lines("netork-x", "a job", "0 1 * * * /bin/true", enabled=True)
added = with_job("A=1\n", [], "netork-x", lines)
assert added == "A=1\n# napalm-scheduled-job: netork-x a job\n0 1 * * * /bin/true\n"
jobs = parse_crontab("/x", added, system=False, user="root")
changed = with_job(
added,
jobs,
"netork-x",
marked_lines("netork-x", "", "0 2 * * * /bin/true", enabled=False),
)
assert (
changed
== "A=1\n# napalm-scheduled-job: netork-x\n#napalm-disabled: 0 2 * * * /bin/true\n"
)
jobs = parse_crontab("/x", changed, system=False, user="root")
assert without_job(changed, jobs, "netork-x") == "A=1\n"
def test_adding_to_a_text_without_a_last_newline(self):
assert with_job("A=1", [], "n", ["# m", "x"]) == "A=1\n# m\nx\n"
def test_removing_the_only_job_leaves_nothing(self):
text = "# napalm-scheduled-job: n\n0 1 * * * x\n"
jobs = parse_crontab("/x", text, system=False, user="root")
assert without_job(text, jobs, "n") == ""
class TestRunScript:
def _run(self, script: str) -> str:
return subprocess.run(
["sh", "-c", script], capture_output=True, text=True, check=True
).stdout
def test_the_crontabs_variables_are_set(self):
assert (
self._run(run_script('echo "$GREETING"', {"GREETING": "hi there", "MAILTO": "x"}))
== "hi there\n"
)
def test_a_percent_starts_stdin_and_further_ones_are_newlines(self):
assert self._run(run_script("cat%one%two", {})) == "one\ntwo\n"
def test_an_escaped_percent_is_a_percent(self):
assert self._run(run_script("date -d @0 -u +\\%Y", {})) == "1970\n"
@pytest.mark.parametrize("command", ["echo plain", "echo 'quoted'"])
def test_without_percent_the_command_is_as_it_is(self, command):
assert run_script(command, {}) == command
+2 -1
View File
@@ -11,6 +11,7 @@ import pytest
from napalm_device_types import (
AccessPointDriver,
BackupServerDriver,
EnsurePackageMixin,
FirewallDriver,
HypervisorDriver,
@@ -28,7 +29,7 @@ from napalm_device_types import (
)
ROLE_BASES = [
AccessPointDriver, FirewallDriver, HypervisorDriver, ManagementControllerDriver, MediaDriver,
AccessPointDriver, BackupServerDriver, FirewallDriver, HypervisorDriver, ManagementControllerDriver, MediaDriver,
OSDriver, PhoneDriver, ResidentialGatewayDriver, StorageDriver, SwitchDriver,
] # fmt: skip
+127
View File
@@ -0,0 +1,127 @@
"""LLDP rows projected into NAPALM's ``get_lldp_neighbors`` and ``get_lldp_neighbors_detail``.
A driver reads its neighbours into :class:`LldpNeighborRow` (from ``lldpctl``,
an API, a CLI) and the two NAPALM shapes come from those rows the same way for
every driver (NAPALM/napalm-device-types#10).
"""
from __future__ import annotations
from typing import get_type_hints
from napalm_device_types import LldpNeighborRow, lldp_neighbors, lldp_neighbors_detail
def _row(local_port: str, name: str, **fields: object) -> LldpNeighborRow:
row: LldpNeighborRow = {
"local_port": local_port,
"chassis_id": "",
"chassis_id_type": "",
"port_id": "",
"port_id_type": "",
"port_description": "",
"system_name": name,
"system_description": "",
"management_addresses": [],
"capabilities": [],
"enabled_capabilities": [],
}
row.update(fields) # type: ignore[typeddict-item]
return row
PHONE = _row(
"eth0.1",
"desk-phone",
chassis_id="00:11:22:33:44:AB",
chassis_id_type="mac",
port_id="00:11:22:33:44:01",
port_id_type="mac",
port_description="WAN",
system_description="SIP phone",
capabilities=["bridge", "telephone"],
enabled_capabilities=["telephone"],
)
PC = _row("eth0.1", "desk-pc", port_id="eno1", port_id_type="ifname")
SWITCH = _row("eth1", "access-sw", port_id="gi1/0/7", port_id_type="local")
def test_a_row_names_its_fields():
assert set(get_type_hints(LldpNeighborRow)) == {
"local_port",
"chassis_id",
"chassis_id_type",
"port_id",
"port_id_type",
"port_description",
"system_name",
"system_description",
"management_addresses",
"capabilities",
"enabled_capabilities",
}
class TestNeighbors:
def test_every_neighbour_of_a_port_is_listed(self):
assert lldp_neighbors([PHONE, PC, SWITCH]) == {
"eth0.1": [
{"hostname": "desk-phone", "port": "00:11:22:33:44:01", "mac": "00:11:22:33:44:ab"},
{"hostname": "desk-pc", "port": "eno1"},
],
"eth1": [{"hostname": "access-sw", "port": "gi1/0/7"}],
}
def test_no_rows_is_no_neighbour(self):
assert lldp_neighbors([]) == {}
def test_a_chassis_mac_is_reported_in_lower_case(self):
# netOrk matches a neighbour by this MAC first (NAPALM/napalm-openwrt#31).
assert lldp_neighbors([PHONE])["eth0.1"][0]["mac"] == "00:11:22:33:44:ab"
def test_a_chassis_named_otherwise_has_no_mac(self):
switch = _row("eth1", "sw", chassis_id="00:11:22:33:44:55", chassis_id_type="local")
assert "mac" not in lldp_neighbors([switch])["eth1"][0]
def test_a_mac_id_that_is_no_mac_address_is_left_out(self):
odd = _row("eth1", "sw", chassis_id="00:11", chassis_id_type="mac")
assert "mac" not in lldp_neighbors([odd])["eth1"][0]
class TestNeighborsDetail:
def test_every_field_napalm_names(self):
assert lldp_neighbors_detail([PHONE]) == {
"eth0.1": [
{
"parent_interface": "",
"remote_port": "00:11:22:33:44:01",
"remote_port_description": "WAN",
"remote_chassis_id": "00:11:22:33:44:AB",
"remote_system_name": "desk-phone",
"remote_system_description": "SIP phone",
"remote_system_capab": ["bridge", "telephone"],
"remote_system_enable_capab": ["telephone"],
}
]
}
def test_every_neighbour_of_a_port_is_listed(self):
detail = lldp_neighbors_detail([PHONE, PC, SWITCH])
assert [n["remote_system_name"] for n in detail["eth0.1"]] == ["desk-phone", "desk-pc"]
assert [n["remote_system_name"] for n in detail["eth1"]] == ["access-sw"]
def test_one_interface_only(self):
assert list(lldp_neighbors_detail([PHONE, PC, SWITCH], "eth1")) == ["eth1"]
def test_an_interface_without_neighbours_is_empty(self):
assert lldp_neighbors_detail([PHONE], "eth9") == {}
def test_the_lists_are_the_rows_own_copies(self):
detail = lldp_neighbors_detail([PHONE])
detail["eth0.1"][0]["remote_system_capab"].append("router")
assert PHONE["capabilities"] == ["bridge", "telephone"]
+308
View File
@@ -0,0 +1,308 @@
"""``lldpctl -f keyvalue``: one row per LLDP neighbour, for every driver that reads lldpd.
napalm-linux, napalm-openwrt and napalm-proxmox each parsed this output on their
own, and each copy grouped the lines by interface, so two neighbours on one port
merged (NAPALM/napalm-openwrt#9, NAPALM/napalm-linux#58), split ``eth0.1`` at its
dot (NAPALM/napalm-openwrt#26) and looked for ``chassis.id.value``, a key lldpd
never prints. The parse lives here once now.
"""
from __future__ import annotations
import pytest
from napalm_device_types import LLDPCTL_KEYVALUE_COMMAND, parse_lldpctl_keyvalue
# The recording napalm-openwrt's tests have used since the driver was written
# (tests/unit/test_driver.py, ``LLDPCTL_KV``): one neighbour on eth0. Its two
# ``chassis.cap.*`` lines are not lldpd's format (lldpd prints one
# ``chassis.<Capability>.enabled=on|off`` line per capability), so they yield none.
OPENWRT_ONE_NEIGHBOUR = """\
lldp.eth0.via=LLDP
lldp.eth0.rid=1
lldp.eth0.age=0 day, 01:23:45
lldp.eth0.chassis.mac=00:aa:bb:cc:dd:ee
lldp.eth0.chassis.name=core-router
lldp.eth0.chassis.descr=RouterOS 7.x
lldp.eth0.chassis.mgmt-ip=10.0.0.1
lldp.eth0.chassis.cap.available=Router, Bridge
lldp.eth0.chassis.cap.enabled=Router
lldp.eth0.port.ifname=ether1
lldp.eth0.port.descr=uplink
"""
# Laid out the way lldpd 1.0's keyvalue writer prints ``show neighbors details``
# (src/client/display.c ``display_interface``, ``display_chassis``,
# ``display_port``, ``display_vlans``, ``display_med``; src/client/kv_writer.c):
# a neighbour opens with the local interface's optional ``alias`` and ``via``,
# an id is printed under its type (``chassis.mac``, ``port.local``), a
# capability as ``chassis.<Symbol>.enabled``, a management address once per
# address. Not recorded from a device.
#
# On ``eth0.1`` (a VLAN interface, its name holds a dot) a phone and a PC behind
# an unmanaged switch; on ``eth1`` a switch that names its port by a local id and
# itself by an address.
TWO_ON_ONE_DOTTED_PORT = """\
lldp.eth0.1.alias=desk
lldp.eth0.1.via=LLDP
lldp.eth0.1.rid=1
lldp.eth0.1.age=0 day, 00:10:00
lldp.eth0.1.chassis.mac=00:11:22:33:44:01
lldp.eth0.1.chassis.name=desk-phone
lldp.eth0.1.chassis.descr=SIP phone, firmware a=1
lldp.eth0.1.chassis.mgmt-ip=10.0.0.21
lldp.eth0.1.chassis.mgmt-iface=2
lldp.eth0.1.chassis.mgmt-ip=fe80::211:22ff:fe33:4401
lldp.eth0.1.chassis.mgmt-iface=2
lldp.eth0.1.chassis.Bridge.enabled=off
lldp.eth0.1.chassis.Tel.enabled=on
lldp.eth0.1.port.mac=00:11:22:33:44:01
lldp.eth0.1.port.descr=WAN
lldp.eth0.1.port.ttl=120
lldp.eth0.1.port.auto-negotiation.supported=yes
lldp.eth0.1.port.auto-negotiation.enabled=yes
lldp.eth0.1.port.auto-negotiation.100Base-TX.hd=yes
lldp.eth0.1.port.auto-negotiation.100Base-TX.fd=yes
lldp.eth0.1.port.auto-negotiation.current=100BaseTXFD - 2 pair category 5 UTP, full duplex mode
lldp.eth0.1.vlan.vlan-id=10
lldp.eth0.1.vlan.pvid=yes
lldp.eth0.1.vlan=voice
lldp.eth0.1.lldp-med.device-type=Communication Device Endpoint (Class III)
lldp.eth0.1.lldp-med.Capabilities.available=yes
lldp.eth0.1.lldp-med.inventory.software=2.1.4
lldp.eth0.1.alias=desk
lldp.eth0.1.via=LLDP
lldp.eth0.1.rid=2
lldp.eth0.1.age=0 day, 00:09:00
lldp.eth0.1.chassis.mac=00:11:22:33:44:02
lldp.eth0.1.chassis.name=desk-pc
lldp.eth0.1.chassis.descr=Debian GNU/Linux 12 (bookworm) Linux 6.1.0-25-amd64
lldp.eth0.1.chassis.Bridge.enabled=off
lldp.eth0.1.chassis.Router.enabled=off
lldp.eth0.1.chassis.Wlan.enabled=off
lldp.eth0.1.chassis.Station.enabled=on
lldp.eth0.1.port.ifname=eno1
lldp.eth0.1.port.ttl=120
lldp.eth0.1.unknown-tlvs.unknown-tlv.oui=00,80,C2
lldp.eth0.1.unknown-tlvs.unknown-tlv.subtype=9
lldp.eth0.1.unknown-tlvs.unknown-tlv.len=1
lldp.eth0.1.unknown-tlvs.unknown-tlv=01
lldp.eth1.via=LLDP
lldp.eth1.rid=3
lldp.eth1.age=1 day, 02:00:00
lldp.eth1.chassis.ip=192.0.2.7
lldp.eth1.chassis.name=access-sw
lldp.eth1.chassis.Bridge.enabled=on
lldp.eth1.chassis.Router.enabled=off
lldp.eth1.chassis.CVLAN.enabled=on
lldp.eth1.port.local=gi1/0/7
lldp.eth1.port.descr=GigabitEthernet1/0/7
lldp.eth1.ppvid.supported=no
lldp.eth1.ppvid.enabled=no
lldp.eth1.pi=88cc
"""
def _rows(output: str = TWO_ON_ONE_DOTTED_PORT) -> list[dict[str, object]]:
return [dict(row) for row in parse_lldpctl_keyvalue(output)]
class TestOneRowPerNeighbour:
def test_the_openwrt_recording_is_one_neighbour_with_its_fields(self):
assert _rows(OPENWRT_ONE_NEIGHBOUR) == [
{
"local_port": "eth0",
"chassis_id": "00:aa:bb:cc:dd:ee",
"chassis_id_type": "mac",
"port_id": "ether1",
"port_id_type": "ifname",
"port_description": "uplink",
"system_name": "core-router",
"system_description": "RouterOS 7.x",
"management_addresses": ["10.0.0.1"],
"capabilities": [],
"enabled_capabilities": [],
}
]
def test_two_neighbours_on_one_port_stay_apart(self):
rows = _rows()
assert [(r["local_port"], r["system_name"]) for r in rows] == [
("eth0.1", "desk-phone"),
("eth0.1", "desk-pc"),
("eth1", "access-sw"),
]
def test_a_neighbour_keeps_only_its_own_fields(self):
pc = _rows()[1]
assert pc["chassis_id"] == "00:11:22:33:44:02"
assert pc["port_description"] == ""
assert pc["management_addresses"] == []
def test_neighbours_without_alias_are_split_at_via(self):
output = OPENWRT_ONE_NEIGHBOUR + OPENWRT_ONE_NEIGHBOUR.replace("core-router", "edge")
assert [r["system_name"] for r in _rows(output)] == ["core-router", "edge"]
class TestInterfaceName:
def test_a_dotted_interface_name_survives(self):
assert {r["local_port"] for r in _rows()} == {"eth0.1", "eth1"}
def test_an_interface_named_like_a_field_survives(self):
output = (
"lldp.vlan.100.via=LLDP\n"
"lldp.vlan.100.chassis.mac=00:aa:bb:cc:dd:01\n"
"lldp.vlan.100.port.ifname=port1\n"
)
(row,) = _rows(output)
assert (row["local_port"], row["chassis_id"], row["port_id"]) == (
"vlan.100",
"00:aa:bb:cc:dd:01",
"port1",
)
def test_without_via_the_name_ends_at_the_first_field(self):
output = "lldp.br-lan.10.chassis.name=sw\nlldp.br-lan.10.port.ifname=1\n"
(row,) = _rows(output)
assert (row["local_port"], row["system_name"], row["port_id"]) == ("br-lan.10", "sw", "1")
class TestIds:
def test_a_port_given_by_its_mac_is_read(self):
phone = _rows()[0]
assert (phone["port_id"], phone["port_id_type"]) == ("00:11:22:33:44:01", "mac")
def test_a_port_given_by_a_local_id_is_read(self):
switch = _rows()[2]
assert (switch["port_id"], switch["port_id_type"]) == ("gi1/0/7", "local")
def test_a_chassis_given_by_its_address_is_read(self):
switch = _rows()[2]
assert (switch["chassis_id"], switch["chassis_id_type"]) == ("192.0.2.7", "ip")
@pytest.mark.parametrize("kind", ["local", "ifname", "ifalias", "unhandled"])
def test_every_id_type_lldpd_names_is_read(self, kind):
output = f"lldp.eth0.via=LLDP\nlldp.eth0.chassis.{kind}=c1\nlldp.eth0.port.{kind}=p1\n"
(row,) = _rows(output)
assert (row["chassis_id"], row["chassis_id_type"]) == ("c1", kind)
assert (row["port_id"], row["port_id_type"]) == ("p1", kind)
def test_a_neighbour_without_ids_has_empty_ones(self):
(row,) = _rows("lldp.eth0.via=CDPv2\nlldp.eth0.chassis.name=sw\n")
assert (row["chassis_id"], row["chassis_id_type"]) == ("", "")
assert (row["port_id"], row["port_id_type"]) == ("", "")
class TestOtherFields:
def test_every_management_address_is_kept(self):
phone = _rows()[0]
assert phone["management_addresses"] == ["10.0.0.21", "fe80::211:22ff:fe33:4401"]
def test_a_value_may_hold_an_equals_sign(self):
assert _rows()[0]["system_description"] == "SIP phone, firmware a=1"
def test_capabilities_in_napalm_names_and_which_are_enabled(self):
phone, pc, switch = _rows()
assert phone["capabilities"] == ["bridge", "telephone"]
assert phone["enabled_capabilities"] == ["telephone"]
assert pc["capabilities"] == ["bridge", "router", "wlan-access-point", "station"]
assert pc["enabled_capabilities"] == ["station"]
assert switch["capabilities"] == ["bridge", "router", "cvlan"]
assert switch["enabled_capabilities"] == ["bridge", "cvlan"]
def test_every_napalm_capability_name(self):
symbols = ["Other", "Repeater", "Bridge", "Router", "Wlan", "Tel", "Docsis", "Station"]
output = "lldp.eth0.via=LLDP\n" + "".join(
f"lldp.eth0.chassis.{s}.enabled=on\n" for s in symbols
)
(row,) = _rows(output)
assert row["capabilities"] == [
"other",
"repeater",
"bridge",
"router",
"wlan-access-point",
"telephone",
"docsis-cable-device",
"station",
]
assert row["enabled_capabilities"] == row["capabilities"]
def test_a_capability_line_needs_on_or_off(self):
(row,) = _rows(OPENWRT_ONE_NEIGHBOUR)
assert row["capabilities"] == []
class TestReadOrFailure:
"""A failed read raises; ``[]`` only when lldpctl ran and listed nobody."""
def test_no_output_is_no_neighbour(self):
assert parse_lldpctl_keyvalue("") == []
def test_a_zero_exit_status_without_neighbours_is_no_neighbour(self):
assert parse_lldpctl_keyvalue("__LLDPCTL_RC=0\n") == []
def test_a_missing_lldpctl_raises(self):
with pytest.raises(ValueError, match="lldpctl: not found"):
parse_lldpctl_keyvalue("sh: lldpctl: not found\n")
def test_a_non_zero_exit_status_raises_with_what_lldpctl_said(self):
output = (
"[WARN/control] unable to connect to socket /var/run/lldpd.socket: "
"No such file or directory\n"
"__LLDPCTL_RC=1\n"
)
with pytest.raises(ValueError, match="unable to connect to socket"):
parse_lldpctl_keyvalue(output)
def test_a_non_zero_exit_status_without_a_message_names_the_status(self):
with pytest.raises(ValueError, match="exit status 127"):
parse_lldpctl_keyvalue("__LLDPCTL_RC=127\n")
def test_with_status_zero_a_warning_beside_the_neighbours_is_ignored(self):
output = (
"lldpctl -f keyvalue 2>&1; echo __LLDPCTL_RC=$?\n"
"[WARN/lldpctl] something lldpd wants to say\n"
+ OPENWRT_ONE_NEIGHBOUR
+ "__LLDPCTL_RC=0\n"
)
assert [r["system_name"] for r in _rows(output)] == ["core-router"]
def test_a_line_outside_lldp_is_no_neighbour(self):
output = "lldpd.config.tx-delay=30\n" + OPENWRT_ONE_NEIGHBOUR
assert len(_rows(output)) == 1
def test_a_key_without_a_known_field_is_skipped(self):
output = "lldp.summary=1\nlldp.eth9.future-field=x\n" + OPENWRT_ONE_NEIGHBOUR
assert [r["local_port"] for r in _rows(output)] == ["eth0"]
class TestCommand:
def test_the_command_reads_the_neighbours_in_keyvalue_form(self):
assert LLDPCTL_KEYVALUE_COMMAND.startswith("lldpctl -f keyvalue ")
def test_the_command_brings_back_the_error_text_and_the_exit_status(self):
assert "2>&1" in LLDPCTL_KEYVALUE_COMMAND
assert LLDPCTL_KEYVALUE_COMMAND.endswith("echo __LLDPCTL_RC=$?")
+12
View File
@@ -176,6 +176,18 @@ __DNF_RC=0
with pytest.raises(ValueError, match="exit status 1"):
parse_dnf_security(out)
def test_the_command_reads_only_the_cached_metadata(self, tmp_path):
"""An expired cache made dnf download the repositories' metadata inside the
poll, which outlasted its whole budget (#76)."""
args = tmp_path / "args"
dnf = tmp_path / "dnf"
dnf.write_text(f"#!/bin/sh\nprintf '%s\\n' \"$@\" > {args}\n")
dnf.chmod(0o755)
env = {"PATH": f"{tmp_path}:{os.environ.get('PATH', '/usr/bin:/bin')}"}
subprocess.run(["sh", "-c", DNF_SECURITY_COMMAND], capture_output=True, env=env)
assert "--cacheonly" in args.read_text().split()
@pytest.mark.parametrize(
("nevra", "name"),
[
+4
View File
@@ -18,6 +18,7 @@ import pytest
from napalm_device_types import (
AccessPointDriver,
BackupServerDriver,
DeviceTypeDriver,
FirewallDriver,
HypervisorDriver,
@@ -33,6 +34,7 @@ from napalm_device_types.roles import primary_role_of, roles_of
ROLE_BASES = [
AccessPointDriver,
BackupServerDriver,
FirewallDriver,
HypervisorDriver,
ManagementControllerDriver,
@@ -75,6 +77,8 @@ class TestDeclaredMethodsDoNotExistAtRuntime:
@pytest.mark.parametrize(
("base", "method"),
[
(BackupServerDriver, "get_backup_datastores"),
(BackupServerDriver, "run_backup_job"),
(StorageDriver, "get_disks"),
(StorageDriver, "get_volumes"),
(HypervisorDriver, "get_vms"),
+108
View File
@@ -0,0 +1,108 @@
"""The scheduled-jobs contract: its digest, the names and specs it takes (NetOrk/netork#1933)."""
from __future__ import annotations
import pytest
from napalm_device_types import ScheduledJobsMixin
from napalm_device_types.scheduled_jobs import (
ACTIONS,
ScheduledJobChanged,
ScheduledJobRefused,
check_job_name,
check_spec,
job_digest,
)
JOB = {
"kind": "cron",
"source": "/etc/crontab",
"run_as": "root",
"schedule": "*/5 * * * *",
"command": "/usr/local/bin/backup",
"enabled": True,
}
class TestDigest:
def test_sixteen_hex_characters(self):
digest = job_digest(JOB)
assert len(digest) == 16
int(digest, 16)
def test_the_same_definition_has_the_same_digest(self):
assert job_digest(JOB) == job_digest(dict(JOB))
@pytest.mark.parametrize(
"field, value",
[
("schedule", "0 * * * *"),
("command", "/usr/local/bin/other"),
("run_as", "nobody"),
("enabled", False),
("source", "/etc/cron.d/backup"),
("kind", "anacron"),
],
)
def test_every_part_of_the_definition_changes_it(self, field, value):
assert job_digest({**JOB, field: value}) != job_digest(JOB)
def test_content_the_definition_points_at_changes_it(self):
assert job_digest(JOB, content="12345 678") != job_digest(JOB)
def test_what_the_host_says_of_its_runs_does_not(self):
assert job_digest({**JOB, "last_run": "2026-11-03T06:00:00Z"}) == job_digest(JOB)
class TestNames:
@pytest.mark.parametrize("name", ["backup", "netork-backup-db", "a", "x1"])
def test_a_good_name(self, name):
check_job_name(name)
@pytest.mark.parametrize("name", ["", "-x", "Backup", "a b", "a/b", "a.timer", "a" * 64])
def test_a_name_a_file_or_unit_could_not_carry_safely(self, name):
with pytest.raises(ScheduledJobRefused):
check_job_name(name)
class TestSpec:
SPEC = {
"name": "backup",
"schedule": "0 3 * * *",
"run_as": "root",
"command": "/usr/local/bin/backup",
"enabled": True,
}
def test_a_good_spec(self):
check_spec(self.SPEC)
check_spec({**self.SPEC, "schedule": "@reboot"})
@pytest.mark.parametrize(
"field, value",
[
("schedule", "every day"),
("run_as", "root; rm"),
("run_as", ""),
("command", ""),
("command", " "),
("description", "two\nlines"),
],
)
def test_a_spec_that_cannot_be_written(self, field, value):
with pytest.raises(ScheduledJobRefused):
check_spec({**self.SPEC, field: value})
class TestContract:
def test_the_mixin_declares_nothing_at_runtime(self):
driver = ScheduledJobsMixin()
for method in ("get_scheduled_jobs", "manage_scheduled_job", "put_scheduled_job"):
assert not hasattr(driver, method)
def test_the_actions(self):
assert ACTIONS == ("enable", "disable", "run")
def test_a_refusal_is_a_value_error_and_a_change_is_not(self):
assert issubclass(ScheduledJobRefused, ValueError)
assert not issubclass(ScheduledJobChanged, ValueError)
+540
View File
@@ -0,0 +1,540 @@
"""Scheduled jobs on a POSIX host: the report as root, switching, running, an application's jobs.
The reading command is run by a real ``sh`` over a directory tree that stands in
for the host, with ``systemctl``, ``crontab``, ``dpkg``, ``install`` and
``journalctl`` replaced by scripts that keep a log -- so the shell itself is what
is tested, not a hand-written copy of its output (NetOrk/netork#1933).
"""
from __future__ import annotations
import os
import subprocess
from pathlib import Path
import pytest
from napalm_device_types import PosixScheduledJobsMixin
from napalm_device_types._scheduled_reading import parse_reading
from napalm_device_types.channel import CommandResult
from napalm_device_types.scheduled_jobs import ScheduledJobChanged, ScheduledJobRefused
TIMERS = """\
Id=apt-daily.timer
Unit=apt-daily.service
Description=Daily apt download activities
TimersCalendar={ OnCalendar=*-*-* 06,18:00:00 ; next_elapse=@1762192800 }
NextElapseUSecRealtime=@1762192800
LastTriggerUSec=@1762149600
UnitFileState=enabled
ActiveState=active
FragmentPath=/usr/lib/systemd/system/apt-daily.timer
Id=fstrim.timer
Unit=fstrim.service
Description=Discard unused blocks once a week
TimersMonotonic={ OnUnitActiveUSec=1w ; next_elapse=n/a }
NextElapseUSecRealtime=n/a
LastTriggerUSec=n/a
UnitFileState=static
ActiveState=inactive
FragmentPath=/etc/systemd/system/fstrim.timer
Id=backup.timer
Unit=backup.service
Description=netork backup
TimersCalendar={ OnCalendar=*-*-* 03:00:00 ; next_elapse=n/a }
NextElapseUSecRealtime=Tue 2026-11-04 03:00:00 UTC
LastTriggerUSec=n/a
UnitFileState=enabled
ActiveState=active
FragmentPath=/etc/systemd/system/backup.timer
"""
UNITS = """\
Id=apt-daily.service
ExecStart={ path=/usr/lib/apt/apt.systemd.daily ; argv[]=/usr/lib/apt/apt.systemd.daily update ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }
User=
Result=success
FragmentPath=/usr/lib/systemd/system/apt-daily.service
Id=fstrim.service
ExecStart={ path=/sbin/fstrim ; argv[]=/sbin/fstrim --listed-in /etc/fstab ; ignore_errors=no }
Result=success
FragmentPath=/usr/lib/systemd/system/fstrim.service
Id=backup.service
ExecStart={ path=SCRIPTS/backup ; argv[]=SCRIPTS/backup ; ignore_errors=no }
User=backup
Result=exit-code
FragmentPath=/etc/systemd/system/backup.service
"""
SYSTEMCTL = """#!/bin/sh
echo "systemctl $*" >> "$LOG"
case "$1" in
list-unit-files) printf 'apt-daily.timer enabled enabled\\nfstrim.timer static -\\nbackup.timer enabled enabled\\ngetty@.timer static -\\n' ;;
show) case "$*" in *Unit,Description*) sed "s#SCRIPTS#$SCRIPTS#g" "$FIX/timers" ;; *) sed "s#SCRIPTS#$SCRIPTS#g" "$FIX/units" ;; esac ;;
esac
"""
CRONTAB = """#!/bin/sh
echo "crontab $*" >> "$LOG"
[ "$1" = -u ] || exit 2
if [ "$3" = -l ]; then
[ -f "$SPOOL/$2" ] || { echo "no crontab for $2" >&2; exit 1; }
cat "$SPOOL/$2"
else
cp "$3" "$SPOOL/$2"
fi
"""
DPKG = """#!/bin/sh
for p in "$@"; do case " $OWNED " in *" $p "*) echo "cron: $p" ;; esac; done
"""
INSTALL = """#!/bin/sh
echo "install $*" >> "$LOG"
args=""
while [ $# -gt 0 ]; do case "$1" in -o|-g) shift 2 ;; *) args="$args '$1'"; shift ;; esac; done
eval "/usr/bin/install $args"
"""
ECHO_LOG = """#!/bin/sh
echo "$(basename "$0") $*" >> "$LOG"
[ "$(basename "$0")" = journalctl ] && echo "backup ran fine"
exit 0
"""
class Host(PosixScheduledJobsMixin):
"""A host in a directory; every command is run by sh, root or not."""
def __init__(self, root: Path, *, systemd: bool = True, own_cron_d: bool = True) -> None:
self.root = root
etc = root / "etc"
self.CRON_SYSTEM_FILES = (str(etc / "crontab"),) # type: ignore[misc]
self.CRON_D_DIRS = (str(etc / "cron.d"),) # type: ignore[misc]
self.CRON_SPOOL_DIRS = (str(root / "spool"),) # type: ignore[misc]
self.RUN_PARTS_DIRS = (("daily", str(etc / "cron.daily")),) # type: ignore[misc]
self.ANACRONTAB = str(etc / "anacrontab") # type: ignore[misc]
self.JOB_SCRIPT_DIR = str(root / "scripts") # type: ignore[misc]
self.UNIT_DIR = str(root / "units") # type: ignore[misc]
self.OWN_CRON_D_DIR = str(etc / "cron.d") if own_cron_d else None # type: ignore[misc]
self.ROOT_GROUP = subprocess.run(
["id", "-gn"], capture_output=True, text=True
).stdout.strip() # type: ignore[misc]
self.systemd_dir = root / ("run-systemd" if systemd else "no-systemd")
self.commands: list[str] = []
def _sj_paths(self): # type: ignore[no-untyped-def]
paths = super()._sj_paths()
from dataclasses import replace
return replace(paths, systemd_marker=str(self.systemd_dir))
def run_command(self, command, *, privileged=False, timeout=60, stdin=None): # type: ignore[no-untyped-def]
self.commands.append(command)
env = {
**os.environ,
"PATH": f"{self.root / 'bin'}:{os.environ['PATH']}",
"LOG": str(self.root / "log"),
"SPOOL": str(self.root / "spool"),
"FIX": str(self.root / "fix"),
"SCRIPTS": str(self.root / "scripts"),
"OWNED": str(self.root / "etc" / "crontab"),
}
done = subprocess.run(
["sh", "-c", command], input=stdin, capture_output=True, env=env, timeout=30
)
return CommandResult(done.stdout.decode(), done.stderr.decode(), done.returncode)
def log(self) -> str:
log = self.root / "log"
return log.read_text() if log.exists() else ""
def _tool(root: Path, name: str, body: str) -> None:
path = root / "bin" / name
path.write_text(body)
path.chmod(0o755)
@pytest.fixture
def root(tmp_path: Path) -> Path:
for d in (
"etc/cron.d",
"etc/cron.daily",
"spool",
"scripts",
"units",
"bin",
"fix",
"run-systemd",
):
(tmp_path / d).mkdir(parents=True)
(tmp_path / "etc/crontab").write_text(
"SHELL=/bin/sh\nOUT="
+ str(tmp_path / "out")
+ "\n17 * * * * root cd / && run-parts /etc/cron.hourly\n"
'*/5 * * * * root echo "$OUT" > ' + str(tmp_path / "ran") + "\n"
)
(tmp_path / "etc/cron.d/certbot").write_text("0 */12 * * * root certbot -q renew\n")
(tmp_path / "etc/anacrontab").write_text("1 5 cron.daily run-parts /etc/cron.daily\n")
(tmp_path / "spool/www-data").write_text("*/5 * * * * php /var/www/cron.php\n")
for name, mode in (("logrotate", 0o755), ("off", 0o644), ("x.dpkg-old", 0o755)):
script = tmp_path / "etc/cron.daily" / name
script.write_text("#!/bin/sh\necho rotated\n")
script.chmod(mode)
(tmp_path / "scripts/backup").write_text(
"#!/bin/sh\n# napalm-scheduled-job: backup nightly backup\n# schedule: 0 3 * * *\n/usr/local/bin/backup --all\n"
)
(tmp_path / "fix/timers").write_text(TIMERS)
(tmp_path / "fix/units").write_text(UNITS)
for name, body in (
("systemctl", SYSTEMCTL),
("crontab", CRONTAB),
("dpkg", DPKG),
("install", INSTALL),
("journalctl", ECHO_LOG),
("systemd-analyze", ECHO_LOG),
):
_tool(tmp_path, name, body)
return tmp_path
def _jobs(host: Host) -> dict[str, dict]:
return {j["key"]: j for j in host.get_scheduled_jobs()["jobs"]}
class TestReading:
def test_every_source_is_read(self, root: Path):
jobs = _jobs(Host(root))
kinds = sorted({j["kind"] for j in jobs.values()})
assert kinds == ["anacron", "cron", "run_parts", "systemd_timer"]
users = {j["run_as"] for j in jobs.values() if j["kind"] == "cron"}
assert users == {"root", "www-data"}
def test_a_package_owns_what_it_installed(self, root: Path):
jobs = _jobs(Host(root))
crontab = [j for j in jobs.values() if j["source"] == str(root / "etc/crontab")]
assert {j["managed_by"] for j in crontab} == {"package"}
certbot = next(j for j in jobs.values() if j["source"].endswith("certbot"))
assert certbot["managed_by"] == "admin"
def test_run_parts_by_execute_bit_and_the_names_run_parts_runs(self, root: Path):
jobs = _jobs(Host(root))
daily = root / "etc/cron.daily"
assert jobs[f"run-parts:{daily}/logrotate"]["enabled"]
assert not jobs[f"run-parts:{daily}/off"]["enabled"]
assert f"run-parts:{daily}/x.dpkg-old" not in jobs
def test_a_changed_script_changes_its_digest(self, root: Path):
host = Host(root)
key = f"run-parts:{root}/etc/cron.daily/logrotate"
before = _jobs(host)[key]["digest"]
(root / "etc/cron.daily/logrotate").write_text("#!/bin/sh\necho something else\n")
assert _jobs(host)[key]["digest"] != before
def test_timers_with_their_unit_and_runs(self, root: Path):
apt = _jobs(Host(root))["timer:apt-daily.timer"]
assert apt["schedule"] == "OnCalendar=*-*-* 06,18:00:00"
assert apt["schedule_kind"] == "calendar"
assert apt["command"] == "/usr/lib/apt/apt.systemd.daily update"
assert (apt["run_as"], apt["managed_by"], apt["enabled"]) == ("root", "package", True)
assert apt["last_run"] == "2025-11-03T06:00:00+00:00"
assert apt["next_run"] == "2025-11-03T18:00:00+00:00"
assert apt["last_result"] == "success"
assert apt["actions"] == ["enable", "disable", "run"]
def test_a_static_timer_only_runs(self, root: Path):
fstrim = _jobs(Host(root))["timer:fstrim.timer"]
assert fstrim["schedule"] == "OnUnitActiveSec=1w"
assert fstrim["schedule_kind"] == "monotonic"
assert fstrim["actions"] == ["run"]
assert fstrim.get("last_run") is None
assert fstrim["managed_by"] == "admin"
def test_an_applications_timer_by_its_script(self, root: Path):
backup = _jobs(Host(root))["job:backup"]
assert backup["managed_by"] == "application"
assert (backup["name"], backup["description"]) == ("backup", "nightly backup")
assert (backup["schedule"], backup["schedule_kind"]) == ("0 3 * * *", "cron")
assert backup["command"] == "/usr/local/bin/backup --all"
assert backup["run_as"] == "backup"
assert "edit" in backup["actions"] and "delete" in backup["actions"]
assert backup["next_run"] == "2026-11-04T03:00:00+00:00"
assert "last_result" not in backup
def test_a_file_without_a_last_newline_does_not_swallow_the_next(self, root: Path):
(root / "etc/cron.d/certbot").write_text("0 */12 * * * root certbot -q renew")
jobs = _jobs(Host(root))
assert any(j["source"].endswith("certbot") for j in jobs.values())
assert any(j["source"].endswith("spool/www-data") for j in jobs.values())
def test_without_systemd_no_timers(self, root: Path):
jobs = _jobs(Host(root, systemd=False))
assert not [j for j in jobs.values() if j["kind"] == "systemd_timer"]
def test_a_report_cut_short_raises(self, root: Path):
host = Host(root)
full = host.run_command
host.run_command = lambda c, **kw: full(c, **kw)._replace(
stdout=full(c, **kw).stdout.split("SJ_END")[0][:-3]
) # type: ignore[method-assign]
with pytest.raises(ValueError, match="cut short"):
host.get_scheduled_jobs()
class TestDriverSettings:
def test_what_the_platform_generates(self, root: Path):
host = Host(root)
host.PLATFORM_SOURCES = ("*/cron.d/cert*",) # type: ignore[misc]
certbot = next(j for j in _jobs(host).values() if j["source"].endswith("certbot"))
assert certbot["managed_by"] == "platform"
def test_the_time_zone_and_owners_commands_are_the_drivers(self, root: Path):
host = Host(root)
host.TIME_ZONE_SH = "echo America/New_York" # type: ignore[misc]
host.OWNERS_SH = "cat >/dev/null" # type: ignore[misc]
reading = host.get_scheduled_jobs()
assert reading["time_zone"] == "America/New_York"
assert {j["managed_by"] for j in reading["jobs"] if j["kind"] == "cron"} == {"admin"}
class TestParseReading:
def test_the_time_zone_from_either_file(self):
out = "SJ_BEGIN\n[tz]\n/usr/share/zoneinfo/Europe/Berlin\nSJ_END\n"
assert parse_reading(out).time_zone == "Europe/Berlin"
assert parse_reading("SJ_BEGIN\n[tz]\nEtc/UTC\nSJ_END").time_zone == "Etc/UTC"
assert parse_reading("SJ_BEGIN\n[tz]\nSJ_END").time_zone is None
def test_owners_skip_diversions(self):
out = "SJ_BEGIN\n[owners]\ndiversion by x from: /etc/crontab\ncron, cron-daemon: /etc/cron.d/x\nSJ_END"
reading = parse_reading(out + "")
assert reading.jobs == []
def test_a_time_not_in_utc_is_left_unknown(self):
from napalm_device_types._scheduled_reading import _timestamp
assert _timestamp("Tue 2026-11-04 03:00:00 CET") is None
assert _timestamp("@0") is None
class TestSwitching:
def _cron(self, host: Host, path: str) -> dict:
return next(j for j in _jobs(host).values() if j["source"] == path and j["kind"] == "cron")
def test_a_system_line_switched_off_and_on(self, root: Path):
host = Host(root)
path = str(root / "etc/cron.d/certbot")
job = self._cron(host, path)
mode = os.stat(path).st_mode
assert host.manage_scheduled_job(job["key"], "disable", digest=job["digest"])["success"]
assert (
root / "etc/cron.d/certbot"
).read_text() == "#napalm-disabled: 0 */12 * * * root certbot -q renew\n"
off = _jobs(host)[job["key"]]
assert not off["enabled"]
host.manage_scheduled_job(job["key"], "enable", digest=off["digest"])
assert (root / "etc/cron.d/certbot").read_text() == "0 */12 * * * root certbot -q renew\n"
assert f"install -D -m {mode & 0o777:o}" in host.log()
assert os.stat(path).st_mode == mode
def test_a_linked_file_is_written_through_its_link(self, root: Path):
real = root / "etc/vzdump.cron"
real.write_text("0 1 * * * root vzdump --all\n")
real.chmod(0o640)
link = root / "etc/cron.d/vzdump"
link.symlink_to(real)
host = Host(root)
job = self._cron(host, str(link))
host.manage_scheduled_job(job["key"], "disable", digest=job["digest"])
assert link.is_symlink()
assert real.read_text() == "#napalm-disabled: 0 1 * * * root vzdump --all\n"
assert real.stat().st_mode & 0o777 == 0o640
def test_a_users_line_goes_through_crontab(self, root: Path):
host = Host(root)
job = self._cron(host, str(root / "spool/www-data"))
host.manage_scheduled_job(job["key"], "disable", digest=job["digest"])
assert (root / "spool/www-data").read_text().startswith("#napalm-disabled: */5")
assert "crontab -u www-data /" in host.log()
def test_a_change_it_did_not_see_is_refused(self, root: Path):
host = Host(root)
job = self._cron(host, str(root / "etc/cron.d/certbot"))
with pytest.raises(ScheduledJobChanged):
host.manage_scheduled_job(job["key"], "disable", digest="0" * 16)
def test_an_unknown_job_and_an_unknown_action(self, root: Path):
host = Host(root)
with pytest.raises(LookupError):
host.manage_scheduled_job("cron:/nowhere:1", "disable", digest="x")
with pytest.raises(ScheduledJobRefused):
host.manage_scheduled_job("cron:/nowhere:1", "explode", digest="x")
def test_a_static_timer_is_not_disabled(self, root: Path):
host = Host(root)
fstrim = _jobs(host)["timer:fstrim.timer"]
with pytest.raises(ScheduledJobRefused):
host.manage_scheduled_job("timer:fstrim.timer", "disable", digest=fstrim["digest"])
def test_a_timer_and_a_run_parts_script(self, root: Path):
host = Host(root)
jobs = _jobs(host)
apt = jobs["timer:apt-daily.timer"]
host.manage_scheduled_job(apt["key"], "disable", digest=apt["digest"])
assert "systemctl disable --now -- apt-daily.timer" in host.log()
key = f"run-parts:{root}/etc/cron.daily/off"
host.manage_scheduled_job(key, "enable", digest=jobs[key]["digest"])
assert os.access(root / "etc/cron.daily/off", os.X_OK)
def test_an_applications_timer_is_switched_by_its_name(self, root: Path):
host = Host(root)
backup = _jobs(host)["job:backup"]
host.manage_scheduled_job("job:backup", "disable", digest=backup["digest"])
assert "systemctl disable --now -- backup.timer" in host.log()
class TestRunning:
def test_a_cron_line_runs_as_cron_would(self, root: Path):
host = Host(root)
job = next(j for j in _jobs(host).values() if "ran" in j["command"])
result = host.manage_scheduled_job(job["key"], "run", digest=job["digest"])
assert result["success"]
assert (root / "ran").read_text() == f"{root / 'out'}\n"
assert "timeout 300 sh -c" in host.commands[-1]
def test_a_timer_runs_its_unit_and_says_what_its_journal_said(self, root: Path):
host = Host(root)
backup = _jobs(host)["job:backup"]
result = host.manage_scheduled_job("job:backup", "run", digest=backup["digest"])
assert result == {"success": True, "output": "backup ran fine"}
assert "systemctl start -- backup.service" in host.log()
def test_a_run_parts_script_runs(self, root: Path):
host = Host(root)
key = f"run-parts:{root}/etc/cron.daily/logrotate"
result = host.manage_scheduled_job(key, "run", digest=_jobs(host)[key]["digest"])
assert result == {"success": True, "output": "rotated"}
def test_another_user_runs_through_su(self):
assert Host(Path("/x"))._sj_as("www-data", "true") == [
"su",
"-s",
"/bin/sh",
"www-data",
"-c",
"true",
]
SPEC = {
"name": "netork-tidy",
"schedule": "*/30 1-2 * * *",
"run_as": "root",
"command": "find /tmp -mtime +7 -delete",
"enabled": True,
"description": "tidy %tmp",
}
class TestApplicationJobs:
def test_a_timer_on_a_host_with_systemd(self, root: Path):
host = Host(root)
assert host.put_scheduled_job(SPEC)["output"] == "job:netork-tidy created"
timer = (root / "units/netork-tidy.timer").read_text()
assert "OnCalendar=*-*-* 01,02:00,30:00" in timer
assert "Description=tidy %%tmp" in timer
service = (root / "units/netork-tidy.service").read_text()
assert f"ExecStart={root}/scripts/netork-tidy" in service and "User=root" in service
script = (root / "scripts/netork-tidy").read_text()
assert script.splitlines()[1:] == [
"# napalm-scheduled-job: netork-tidy tidy %tmp",
"# schedule: */30 1-2 * * *",
"find /tmp -mtime +7 -delete",
]
assert "systemctl daemon-reload" in host.log()
assert "systemctl enable --now -- netork-tidy.timer" in host.log()
def test_reboot_is_a_boot_timer(self, root: Path):
Host(root).put_scheduled_job({**SPEC, "schedule": "@reboot", "enabled": False})
assert "OnBootSec=1min" in (root / "units/netork-tidy.timer").read_text()
def test_creating_one_that_is_there_and_changing_one_that_is_not(self, root: Path):
host = Host(root)
with pytest.raises(ScheduledJobChanged):
host.put_scheduled_job({**SPEC, "name": "backup"})
with pytest.raises(LookupError):
host.put_scheduled_job(SPEC, digest="x")
with pytest.raises(ScheduledJobChanged):
host.put_scheduled_job({**SPEC, "name": "backup"}, digest="x")
def test_an_unknown_user_is_refused(self, root: Path):
with pytest.raises(ScheduledJobRefused, match="no user"):
Host(root).put_scheduled_job({**SPEC, "run_as": "no-such-user-here"})
def test_a_timer_deleted(self, root: Path):
host = Host(root)
backup = _jobs(host)["job:backup"]
host.delete_scheduled_job("job:backup", digest=backup["digest"])
assert not (root / "scripts/backup").exists()
assert "systemctl disable --now -- backup.timer" in host.log()
def test_only_an_applications_job_is_deleted(self, root: Path):
host = Host(root)
apt = _jobs(host)["timer:apt-daily.timer"]
with pytest.raises(ScheduledJobRefused):
host.delete_scheduled_job(apt["key"], digest=apt["digest"])
def test_a_cron_d_file_without_systemd(self, root: Path):
host = Host(root, systemd=False)
host.put_scheduled_job(SPEC)
assert (root / "etc/cron.d/netork-tidy").read_text() == (
"# napalm-scheduled-job: netork-tidy tidy %tmp\n*/30 1-2 * * * root find /tmp -mtime +7 -delete\n"
)
job = _jobs(host)["job:netork-tidy"]
assert job["managed_by"] == "application" and job["kind"] == "cron"
host.delete_scheduled_job("job:netork-tidy", digest=job["digest"])
assert not (root / "etc/cron.d/netork-tidy").exists()
def test_marked_lines_in_the_users_crontab_on_a_busybox_host(self, root: Path):
host = Host(root, systemd=False, own_cron_d=False)
host.put_scheduled_job({**SPEC, "run_as": "www-data"})
text = (root / "spool/www-data").read_text()
assert text.endswith(
"# napalm-scheduled-job: netork-tidy tidy %tmp\n*/30 1-2 * * * find /tmp -mtime +7 -delete\n"
)
job = _jobs(host)["job:netork-tidy"]
host.put_scheduled_job(
{**SPEC, "run_as": "www-data", "enabled": False}, digest=job["digest"]
)
assert "#napalm-disabled: */30" in (root / "spool/www-data").read_text()
job = _jobs(host)["job:netork-tidy"]
host.put_scheduled_job({**SPEC, "run_as": "nobody"}, digest=job["digest"])
assert "netork-tidy" not in (root / "spool/www-data").read_text()
assert "netork-tidy" in (root / "spool/nobody").read_text()
job = _jobs(host)["job:netork-tidy"]
host.delete_scheduled_job("job:netork-tidy", digest=job["digest"])
assert (root / "spool/nobody").read_text() == ""
def test_a_cron_d_that_is_not_there_is_the_crontab(self, root: Path):
host = Host(root, systemd=False)
host.OWN_CRON_D_DIR = str(root / "etc/no-cron.d") # type: ignore[misc]
host.put_scheduled_job(SPEC)
assert "netork-tidy" in (root / "spool/root").read_text()
def test_a_cron_line_holds_one_line(self, root: Path):
with pytest.raises(ScheduledJobRefused, match="one line"):
Host(root, systemd=False).put_scheduled_job({**SPEC, "command": "a\nb"})
def test_a_calendar_systemd_refuses(self, root: Path):
_tool(root, "systemd-analyze", "#!/bin/sh\necho 'Failed to parse' >&2\nexit 1\n")
with pytest.raises(ScheduledJobRefused, match="systemd does not take"):
Host(root).put_scheduled_job(SPEC)
def test_a_failed_install_says_so(self, root: Path):
_tool(root, "install", "#!/bin/sh\necho 'read-only file system' >&2\nexit 1\n")
with pytest.raises(RuntimeError, match="read-only"):
Host(root).put_scheduled_job(SPEC)
+27
View File
@@ -14,6 +14,7 @@ import pytest
from napalm_device_types.channel import CommandResult
from napalm_device_types.ssh_server import (
DROPIN_DIR,
DROPIN_PATH,
MAIN_CONFIG,
OpenSshServerMixin,
@@ -401,6 +402,32 @@ def test_mixin_falls_back_to_sshd_config_without_an_include():
assert host._effective()["mac"] == ["hmac-sha2-256"]
class GeneratedConfigHost(FakeHost):
"""A platform that writes sshd_config from its own database (OpenMediaVault)."""
SSHD_CONFIG_GENERATED = True
def test_generated_sshd_config_still_gets_the_dropin():
host = GeneratedConfigHost()
result = host.harden_ssh_server({"mac": ["hmac-sha1"]})
assert result.applied
assert result.path == DROPIN_PATH
assert host.files[MAIN_CONFIG] == "Include /etc/ssh/sshd_config.d/*.conf\nPort 22\n"
def test_generated_sshd_config_without_an_include_is_left_alone():
"""The platform would write its own file over the change on its next apply."""
host = GeneratedConfigHost(main="Port 22\nMACs hmac-sha1,hmac-sha2-256\n")
result = host.harden_ssh_server({"mac": ["hmac-sha1"]})
assert not result.applied and not result.rolled_back
assert result.path is None
assert host.files[MAIN_CONFIG] == "Port 22\nMACs hmac-sha1,hmac-sha2-256\n"
assert list(host.files) == [MAIN_CONFIG]
assert host.reloads == 0
assert DROPIN_DIR in result.message
def test_mixin_restores_when_sshd_rejects_the_config():
host = FakeHost()
host.syntax_ok = False
+2
View File
@@ -4,6 +4,7 @@ import pytest
from napalm_device_types import (
AccessPointDriver,
BackupServerDriver,
FirewallDriver,
HypervisorDriver,
ManagementControllerDriver,
@@ -15,6 +16,7 @@ from napalm_device_types import (
BASE_CLASSES = [
(AccessPointDriver, "Access Point"),
(BackupServerDriver, "Backup Server"),
(FirewallDriver, "Firewall"),
(HypervisorDriver, "Hypervisor"),
(ManagementControllerDriver, "Management Controller"),