Compare commits

..
42 Commits
Author SHA1 Message Date
christianmanivong 278a707157 Merge pull request 'feat(running-backups): the backups running on a device right now' (#102) from feature/running-backups-2318 into main
driver / lint (push) Successful in 13s
driver / test (3.13) (push) Successful in 58s
driver / lowest (push) Successful in 35s
driver / build (push) Successful in 26s
driver / security (push) Successful in 56s
driver / publish (push) Successful in 26s
driver / standard (push) Successful in 14s
driver / types (push) Successful in 59s
driver / test (3.12) (push) Successful in 1m13s
CI / driver (push) Successful in 2m36s
2026-11-11 07:22:54 +00:00
christianmanivong 56e5e82a59 feat(running-backups): the backups running on a device right now
driver / test (3.13) (pull_request) Successful in 1m5s
driver / lowest (pull_request) Successful in 37s
driver / build (pull_request) Successful in 29s
driver / security (pull_request) Successful in 59s
CI / driver (pull_request) Successful in 2m24s
driver / lint (pull_request) Successful in 17s
driver / standard (pull_request) Successful in 19s
driver / types (pull_request) Successful in 1m2s
driver / test (3.12) (pull_request) Successful in 1m15s
driver / publish (pull_request) Skipped
A restart aborts every backup that runs on the device it restarts, and a
guest's restart breaks the backup its hypervisor is taking of it. netOrk
asks right before it restarts a host after patching (NetOrk/netork#2318),
so the reading belongs in the contract first: RunningBackupsMixin declares
get_running_backups(), opt-in like VmStopRequestMixin, so hasattr tells a
caller whether the driver can tell at all.

Release 4.34.0: the drivers that implement it need it on PyPI.
2026-11-10 23:39:17 +01:00
christianmanivong e6f145d4a3 Merge pull request 'chore(release): 4.33.0' (#101) from chore/release-4.33.0 into main
driver / lint (push) Successful in 16s
driver / test (3.13) (push) Successful in 1m10s
driver / lowest (push) Successful in 54s
driver / build (push) Successful in 32s
driver / security (push) Successful in 1m5s
driver / publish (push) Successful in 33s
driver / types (push) Successful in 1m0s
driver / standard (push) Successful in 16s
driver / test (3.12) (push) Successful in 1m16s
CI / driver (push) Successful in 3m28s
2026-11-07 19:23:16 +00:00
christianmanivong 66037e878e chore(release): 4.33.0
driver / lowest (pull_request) Successful in 39s
driver / test (3.13) (pull_request) Successful in 1m22s
driver / build (pull_request) Successful in 44s
driver / security (pull_request) Successful in 1m9s
CI / driver (pull_request) Successful in 2m57s
driver / standard (pull_request) Successful in 19s
driver / lint (pull_request) Successful in 14s
driver / types (pull_request) Successful in 1m7s
driver / test (3.12) (pull_request) Successful in 1m15s
driver / publish (pull_request) Skipped
2026-11-07 20:18:14 +01:00
christianmanivong afffd0915c Merge pull request 'feat(host-status): tell an interrupted dnf, yum, apk or opkg' (#98) from feat/interrupted-pm-851 into main
driver / lint (push) Successful in 16s
driver / test (3.13) (push) Successful in 1m9s
driver / lowest (push) Successful in 41s
driver / build (push) Successful in 35s
driver / security (push) Successful in 1m7s
driver / publish (push) Skipped
driver / standard (push) Successful in 17s
driver / types (push) Successful in 1m7s
driver / test (3.12) (push) Successful in 1m17s
CI / driver (push) Successful in 2m34s
2026-11-07 19:18:08 +00:00
christianmanivong 92891473be fix(host-status): keep models.py at its recorded size, one literal for the dnf query
driver / test (3.13) (pull_request) Successful in 1m5s
driver / lowest (pull_request) Successful in 40s
driver / build (pull_request) Successful in 36s
driver / security (pull_request) Successful in 1m6s
CI / driver (pull_request) Successful in 2m38s
driver / standard (pull_request) Successful in 14s
driver / lint (pull_request) Successful in 13s
driver / types (pull_request) Successful in 58s
driver / test (3.12) (pull_request) Successful in 1m11s
driver / publish (pull_request) Skipped
The quality ratchet records models.py at 1454 lines and the new comments
had made it 1460. bandit read the dnf history query, built with an
f-string for its path, as SQL put together from strings (B608); it is a
fixed query, now one literal with the path spelt out, and a test holds
it to DNF_HISTORY.

Refs NetOrk/netork#851
2026-11-07 20:14:11 +01:00
christianmanivong 658f1c8ab2 Merge remote-tracking branch 'origin/main' into feat/interrupted-pm-851
# Conflicts:
#	CHANGELOG.md
2026-11-07 20:14:11 +01:00
christianmanivong aacde6be3f Merge pull request 'chore(release): 4.32.0' (#100) from chore/release-4.32.0 into main
driver / lint (push) Successful in 14s
driver / test (3.13) (push) Successful in 1m14s
driver / lowest (push) Successful in 37s
driver / build (push) Successful in 30s
driver / security (push) Successful in 1m7s
driver / publish (push) Successful in 35s
driver / standard (push) Successful in 17s
driver / types (push) Successful in 1m6s
driver / test (3.12) (push) Successful in 1m29s
CI / driver (push) Successful in 3m14s
2026-11-07 18:49:52 +00:00
christianmanivong 58c7616401 chore(release): 4.32.0
driver / test (3.13) (pull_request) Successful in 1m38s
driver / lowest (pull_request) Successful in 1m1s
driver / build (pull_request) Successful in 32s
driver / security (pull_request) Successful in 1m16s
CI / driver (pull_request) Successful in 3m13s
driver / standard (pull_request) Successful in 18s
driver / lint (pull_request) Successful in 16s
driver / types (pull_request) Successful in 1m5s
driver / test (3.12) (pull_request) Successful in 1m23s
driver / publish (pull_request) Skipped
2026-11-07 19:44:49 +01:00
christianmanivong b7e31205d5 Merge pull request 'feat(ssh-server): read how sshd lets people log in' (#99) from feat/ssh-server-auth into main
driver / lint (push) Successful in 15s
driver / test (3.13) (push) Successful in 1m16s
driver / lowest (push) Successful in 42s
driver / build (push) Successful in 33s
driver / security (push) Successful in 1m5s
driver / publish (push) Skipped
driver / standard (push) Successful in 19s
driver / types (push) Successful in 1m2s
driver / test (3.12) (push) Successful in 1m22s
CI / driver (push) Successful in 2m40s
2026-11-07 18:44:43 +00:00
christianmanivong 1d71ff8f91 feat(ssh-server): read how sshd lets people log in
driver / test (3.13) (pull_request) Successful in 1m25s
driver / lowest (pull_request) Successful in 44s
driver / build (pull_request) Successful in 32s
driver / security (pull_request) Successful in 1m5s
CI / driver (pull_request) Successful in 2m48s
driver / lint (pull_request) Successful in 15s
driver / standard (pull_request) Successful in 17s
driver / types (pull_request) Successful in 1m10s
driver / test (3.12) (pull_request) Successful in 1m32s
driver / publish (pull_request) Skipped
SshServerAuthMixin.get_ssh_server_auth() reads the authentication settings
from sshd -T as root (password, keyboard-interactive, public key, host-based,
GSSAPI, AuthenticationMethods, PermitRootLogin, UsePAM) and, with PAM on, the
modules of sshd's PAM auth stack from /etc/pam.d with its includes followed.
It reports the facts; whether a module is a second factor is the consumer's
call. OpenSshServerMixin carries it and both read sshd -T by the same fixed
command, so every driver that can harden sshd can say this too.

Refs NetOrk/netork#662
2026-11-07 15:09:47 +01:00
christianmanivong 58ed0f0a94 feat(host-status): tell an interrupted dnf, yum, apk or opkg
driver / lowest (pull_request) Successful in 39s
driver / build (pull_request) Successful in 36s
driver / test (3.13) (pull_request) Successful in 2m14s
driver / security (pull_request) Failing after 1m1s
CI / driver (pull_request) Failing after 2m42s
driver / standard (pull_request) Failing after 18s
driver / lint (pull_request) Successful in 17s
driver / types (pull_request) Successful in 1m9s
driver / test (3.12) (pull_request) Successful in 1m22s
driver / publish (pull_request) Skipped
get_host_status() read only dpkg's state. It now also reads the rpm
database and dnf's history (or yum's journal), apk's installed database
and opkg's status file, without root, and names the manager it read in
the new package_manager field. While one of them runs, the answer is
unknown, as for dpkg.

Refs NetOrk/netork#851
2026-11-07 14:44:55 +01:00
christianmanivong c1a05d6dba Merge pull request 'chore(release): 4.31.1' (#97) from chore/release-4.31.1 into main
driver / lint (push) Successful in 13s
driver / test (3.13) (push) Successful in 1m3s
driver / lowest (push) Successful in 37s
driver / build (push) Successful in 32s
driver / security (push) Successful in 55s
driver / publish (push) Successful in 27s
driver / types (push) Successful in 59s
driver / standard (push) Successful in 15s
driver / test (3.12) (push) Successful in 1m13s
CI / driver (push) Successful in 3m3s
2026-11-06 17:57:27 +00:00
christianmanivong fccf902c07 chore(release): 4.31.1
driver / lowest (pull_request) Successful in 39s
driver / build (pull_request) Successful in 30s
driver / security (pull_request) Successful in 1m4s
CI / driver (pull_request) Successful in 3m0s
driver / standard (pull_request) Successful in 18s
driver / lint (pull_request) Successful in 15s
driver / types (pull_request) Successful in 1m6s
driver / test (3.12) (pull_request) Successful in 1m23s
driver / test (3.13) (pull_request) Successful in 1m8s
driver / publish (pull_request) Skipped
2026-11-06 18:52:24 +01:00
christianmanivong f5a2811333 Merge pull request 'fix(log-sources): attribute a file only to the process writing it' (#96) from fix/log-writers-only into main
driver / lint (push) Successful in 12s
driver / test (3.13) (push) Successful in 1m4s
driver / lowest (push) Successful in 36s
driver / build (push) Successful in 35s
driver / security (push) Successful in 1m19s
driver / publish (push) Skipped
driver / standard (push) Successful in 16s
driver / types (push) Successful in 56s
driver / test (3.12) (push) Successful in 1m12s
CI / driver (push) Successful in 2m39s
2026-11-06 17:52:18 +00:00
christianmanivong 98d1dbef4c fix(log-sources): attribute a file only to the process writing it
driver / test (3.13) (pull_request) Successful in 1m8s
driver / lowest (pull_request) Successful in 40s
driver / build (pull_request) Successful in 29s
driver / security (pull_request) Successful in 59s
CI / driver (pull_request) Successful in 2m25s
driver / standard (pull_request) Successful in 16s
driver / lint (pull_request) Successful in 14s
driver / types (pull_request) Successful in 1m8s
driver / test (3.12) (pull_request) Successful in 1m15s
driver / publish (pull_request) Skipped
A process that holds a log open only to read it -- CrowdSec's
acquisition, Promtail, rsyslog imfile, a tail -F -- was named as the
file's writer whenever it had the lower pid. The command now reads each
descriptor's flags from /proc/<pid>/fdinfo in the same round trip, and
only an O_WRONLY or O_RDWR holder is the producer. A file only readers
hold stays listed without one; a file only a reader inside a container
holds is its view of a mount and is left out.

Refs: NetOrk/netork#2092
2026-11-06 18:45:31 +01:00
christianmanivong 252ea3b81c Merge pull request 'feat: where a host's logs go beyond it (4.31.0)' (#95) from feat/log-forwarding-2053 into main
driver / lint (push) Successful in 13s
driver / test (3.13) (push) Successful in 1m7s
driver / lowest (push) Successful in 37s
driver / build (push) Successful in 27s
driver / security (push) Successful in 53s
driver / publish (push) Successful in 27s
driver / standard (push) Successful in 16s
driver / types (push) Successful in 55s
driver / test (3.12) (push) Successful in 1m14s
CI / driver (push) Successful in 2m44s
2026-11-06 10:14:19 +00:00
christianmanivong b685b200cc refactor: the rsyslog reader's cutter and action in their own steps
driver / lowest (pull_request) Successful in 37s
driver / build (pull_request) Successful in 26s
driver / security (pull_request) Successful in 56s
CI / driver (pull_request) Successful in 2m31s
driver / standard (pull_request) Successful in 15s
driver / lint (pull_request) Successful in 13s
driver / types (pull_request) Successful in 53s
driver / test (3.12) (pull_request) Successful in 1m7s
driver / test (3.13) (pull_request) Successful in 1m2s
driver / publish (pull_request) Skipped
Keeps _pieces and _Reader.call within the standard's complexity limit.
2026-11-06 11:10:57 +01:00
christianmanivong 8557710724 feat: where a host's logs go beyond it (4.31.0)
driver / test (3.13) (pull_request) Successful in 1m4s
driver / lowest (pull_request) Successful in 38s
driver / build (pull_request) Successful in 30s
driver / security (pull_request) Successful in 1m2s
CI / driver (pull_request) Failing after 2m23s
driver / lint (pull_request) Successful in 16s
driver / standard (pull_request) Failing after 17s
driver / types (pull_request) Successful in 1m8s
driver / test (3.12) (pull_request) Successful in 1m15s
driver / publish (pull_request) Skipped
get_log_sources() reads, in the same command, whether rsyslog runs, its
forwarding rules (legacy @/@@/:omrelp: and RainerScript omfwd/omrelp,
with selector, ruleset and whether they take every message), its imfile
inputs, and whether the journal reaches it (imjournal, or imuxsock with
journald's ForwardToSyslog; failing a setting, the default the
distribution ships commented out). Returned as `forwarding`;
parse_log_forwarding and parse_rsyslog are public (NetOrk/netork#2053).
2026-11-06 11:06:07 +01:00
christianmanivong 1b600416f0 Merge pull request 'feat: which logs a host writes, and who writes each (4.30.0)' (#94) from feature/log-sources-2049 into main
driver / lint (push) Successful in 13s
driver / test (3.13) (push) Successful in 1m3s
driver / lowest (push) Successful in 36s
driver / build (push) Successful in 27s
driver / security (push) Successful in 55s
driver / publish (push) Successful in 27s
driver / standard (push) Successful in 15s
driver / types (push) Successful in 1m4s
driver / test (3.12) (push) Successful in 1m13s
CI / driver (push) Successful in 2m41s
2026-11-04 23:41:03 +00:00
christianmanivong e622b26d42 refactor: keep the log source shapes in log_sources, not in models
driver / lowest (pull_request) Successful in 38s
driver / build (pull_request) Successful in 29s
driver / security (pull_request) Successful in 1m7s
CI / driver (pull_request) Successful in 2m42s
driver / standard (pull_request) Successful in 16s
driver / lint (pull_request) Successful in 15s
driver / types (pull_request) Successful in 59s
driver / test (3.13) (pull_request) Successful in 1m17s
driver / test (3.12) (pull_request) Successful in 1m20s
driver / publish (pull_request) Skipped
models.py is over the file size limit and may not grow (quality ratchet);
the shapes sit with their mixin, as ScheduledJobDict does.

Refs NetOrk/netork#2049
2026-11-05 00:34:20 +01:00
christianmanivong 5daba5d37c feat: which logs a host writes, and who writes each (4.30.0)
driver / test (3.13) (pull_request) Successful in 1m11s
driver / lowest (pull_request) Successful in 38s
driver / build (pull_request) Successful in 33s
driver / security (pull_request) Successful in 1m14s
CI / driver (pull_request) Failing after 2m45s
driver / standard (pull_request) Failing after 19s
driver / lint (pull_request) Successful in 20s
driver / types (pull_request) Successful in 1m14s
driver / test (3.12) (pull_request) Successful in 1m18s
driver / publish (pull_request) Skipped
LogSourcesMixin.get_log_sources() reads, in one sh -c call as root, the
units journald holds entries for, the log files running processes hold
open (/proc/<pid>/fd: *.log, *.log.N, anything under /var/log) with size
and last write from stat -L on the descriptor, and the engine's own file
of each container's output as that container's stream. Each file names
the service or container of its first writer from /proc/<pid>/cgroup; a
file a container writes keeps the container's own path.

journald's store, deleted files and a database's write-ahead log
(LevelDB's 000004.log) are left out. Without root it reads again and says
so (attributed: false); a host whose find cannot list open files (busybox)
raises LogSourcesUnavailable rather than reading as no files.

Refs NetOrk/netork#2049
2026-11-05 00:24:24 +01:00
christianmanivong dacd10fcce Merge pull request 'chore(release): 4.29.0' (#93) from chore/release-4.29.0 into main
driver / lint (push) Successful in 17s
driver / test (3.13) (push) Successful in 1m21s
driver / lowest (push) Successful in 38s
driver / build (push) Successful in 36s
driver / security (push) Successful in 1m12s
driver / publish (push) Successful in 34s
driver / types (push) Successful in 1m13s
driver / standard (push) Successful in 18s
driver / test (3.12) (push) Successful in 1m38s
CI / driver (push) Successful in 3m56s
2026-11-04 14:34:50 +00:00
christianmanivong 8a54674e40 chore(release): 4.29.0
driver / test (3.13) (pull_request) Successful in 1m6s
driver / lowest (pull_request) Successful in 42s
driver / build (pull_request) Successful in 37s
driver / security (pull_request) Successful in 1m11s
CI / driver (pull_request) Successful in 2m58s
driver / standard (pull_request) Successful in 17s
driver / lint (pull_request) Successful in 16s
driver / types (pull_request) Successful in 1m6s
driver / test (3.12) (pull_request) Successful in 1m14s
driver / publish (pull_request) Skipped
2026-11-04 15:29:48 +01:00
christianmanivong d45455adf1 Merge pull request 'feat(local-accounts): remove_local_account_keys can clear the password too' (#92) from feat/remove-keys-clear-password into main
driver / lint (push) Successful in 16s
driver / test (3.13) (push) Successful in 1m11s
driver / lowest (push) Successful in 42s
driver / build (push) Successful in 34s
driver / security (push) Successful in 1m13s
driver / publish (push) Skipped
driver / standard (push) Successful in 19s
driver / types (push) Successful in 1m3s
driver / test (3.12) (push) Successful in 1m20s
CI / driver (push) Successful in 2m43s
2026-11-04 14:29:42 +00:00
christianmanivong ac49e2b5d4 feat(local-accounts): remove_local_account_keys can clear the password too
driver / lowest (pull_request) Successful in 39s
driver / build (pull_request) Successful in 30s
driver / security (pull_request) Successful in 1m2s
CI / driver (pull_request) Successful in 2m36s
driver / lint (pull_request) Successful in 16s
driver / standard (pull_request) Successful in 19s
driver / types (pull_request) Successful in 1m1s
driver / test (3.13) (pull_request) Successful in 1m15s
driver / test (3.12) (pull_request) Successful in 1m18s
driver / publish (pull_request) Skipped
A person who deletes their key loses the password that went with it in netOrk, but the devices kept it, and sudo with it. remove_local_account_keys(name, keys, clear_password=True) takes the keys, the password and sudo in one call; the account stays open. Each step runs when another failed.

See NetOrk/netork#2033.
2026-11-04 15:21:01 +01:00
christianmanivong c967635cba Merge pull request 'chore(release): 4.28.0' (#91) from chore/release-4.28.0 into main
driver / lint (push) Successful in 13s
driver / test (3.13) (push) Successful in 1m4s
driver / lowest (push) Successful in 43s
driver / build (push) Successful in 37s
driver / security (push) Successful in 1m17s
driver / publish (push) Successful in 34s
driver / types (push) Successful in 1m2s
driver / standard (push) Successful in 17s
driver / test (3.12) (push) Successful in 1m12s
CI / driver (push) Successful in 3m33s
2026-11-04 11:04:06 +00:00
christianmanivong c470686305 chore(release): 4.28.0
driver / lowest (pull_request) Successful in 56s
driver / build (pull_request) Successful in 39s
driver / security (pull_request) Successful in 1m16s
CI / driver (pull_request) Successful in 3m2s
driver / standard (pull_request) Successful in 17s
driver / lint (pull_request) Successful in 16s
driver / types (pull_request) Successful in 1m0s
driver / test (3.13) (pull_request) Successful in 1m10s
driver / test (3.12) (pull_request) Successful in 1m16s
driver / publish (pull_request) Skipped
2026-11-04 11:55:02 +01:00
christianmanivong 1edf458feb Merge pull request 'feat(local-accounts): replace and remove keys, clear the password' (#90) from fix/local-account-key-replace into main
driver / lint (push) Successful in 16s
driver / test (3.13) (push) Successful in 1m10s
driver / lowest (push) Successful in 41s
driver / build (push) Successful in 31s
driver / security (push) Successful in 1m7s
driver / publish (push) Skipped
driver / standard (push) Successful in 18s
driver / types (push) Successful in 1m14s
driver / test (3.12) (push) Successful in 1m14s
CI / driver (push) Successful in 2m55s
2026-11-04 10:54:06 +00:00
christianmanivong 00452d0d37 feat(local-accounts): replace and remove keys, clear the password
driver / test (3.13) (pull_request) Successful in 1m10s
driver / lowest (pull_request) Successful in 40s
driver / build (pull_request) Successful in 39s
driver / security (pull_request) Successful in 1m16s
CI / driver (pull_request) Successful in 2m44s
driver / lint (pull_request) Successful in 15s
driver / standard (pull_request) Successful in 18s
driver / types (pull_request) Successful in 1m1s
driver / test (3.12) (pull_request) Successful in 1m21s
driver / publish (pull_request) Skipped
A person who replaces a key, deletes one or removes their password had
no way to see that reach the device: add_local_account only ever added
a key and kept the password.

add_local_account takes replaces=[...] (the old keys go, matched by
their base64, the person's own keys stay) and clear_password=True (the
account logs in with a key only and cannot have sudo).
remove_local_account_keys takes keys off an account and leaves it open.
Drivers supply _account_clear_password and may override
_account_drop_keys for a key store of their own.

See NetOrk/netork#2013
2026-11-04 11:47:46 +01:00
christianmanivong 4e40f95bc1 Merge pull request 'feat(scheduled-jobs): ScheduledJobsMixin and its POSIX implementation (#86), release 4.27.0' (#87) from feature/scheduled-jobs into main
driver / lint (push) Successful in 14s
driver / test (3.13) (push) Successful in 1m0s
driver / lowest (push) Successful in 35s
driver / build (push) Successful in 30s
driver / security (push) Successful in 54s
driver / publish (push) Successful in 41s
driver / types (push) Successful in 59s
driver / standard (push) Successful in 15s
driver / test (3.12) (push) Successful in 1m6s
CI / driver (push) Successful in 3m7s
2026-11-03 07:37:25 +00:00
christianmanivong 70e1c9b218 chore(release): 4.27.0
driver / test (3.13) (pull_request) Successful in 1m11s
driver / lowest (pull_request) Successful in 40s
driver / build (pull_request) Successful in 34s
driver / security (pull_request) Successful in 1m5s
CI / driver (pull_request) Successful in 2m33s
driver / lint (pull_request) Successful in 14s
driver / standard (pull_request) Successful in 17s
driver / types (pull_request) Successful in 1m0s
driver / test (3.12) (pull_request) Successful in 1m20s
driver / publish (pull_request) Skipped
Scheduled jobs (NetOrk/netork#1933): ScheduledJobsMixin, its POSIX
implementation and napalm_device_types.cron. 4.26.0 went out meanwhile
with the sshd drop-in change, so this is 4.27.0.
2026-11-03 08:34:14 +01:00
christianmanivong a70501ff60 Merge remote-tracking branch 'origin/main' into feature/scheduled-jobs
# Conflicts:
#	CHANGELOG.md
2026-11-03 08:33:52 +01:00
christianmanivong c6d119eb83 fix(scheduled-jobs): a monotonic timer as its unit file says it, OnUnitActiveSec
driver / lowest (pull_request) Successful in 43s
driver / build (pull_request) Successful in 27s
driver / security (pull_request) Successful in 1m8s
CI / driver (pull_request) Successful in 2m47s
driver / standard (pull_request) Successful in 14s
driver / lint (pull_request) Successful in 13s
driver / types (pull_request) Successful in 57s
driver / test (3.12) (pull_request) Successful in 1m12s
driver / test (3.13) (pull_request) Successful in 1m14s
driver / publish (pull_request) Skipped
systemctl show names a timer's monotonic settings by their properties,
OnUnitActiveUSec=3h; the schedule now reads as the unit file writes it.
Found on a live Ubuntu host.

Refs #86
2026-11-03 08:11:48 +01:00
christianmanivong 8e9d21c368 Merge pull request 'feat(ssh): SSHD_CONFIG_GENERATED keeps the hardening to the drop-in (#88), release 4.26.0' (#89) from fix/ssh-hardening-generated-config into main
driver / lint (push) Successful in 23s
driver / test (3.13) (push) Successful in 1m41s
driver / lowest (push) Successful in 1m30s
driver / build (push) Successful in 1m5s
driver / security (push) Successful in 1m41s
driver / publish (push) Successful in 1m11s
driver / test (3.12) (push) Successful in 1m35s
driver / standard (push) Successful in 17s
driver / types (push) Successful in 2m7s
CI / driver (push) Successful in 8m25s
2026-11-03 07:00:40 +00:00
christianmanivong 25f28297e4 chore(release): 4.26.0
driver / test (3.13) (pull_request) Successful in 2m2s
driver / lowest (pull_request) Successful in 36s
driver / build (pull_request) Successful in 35s
driver / security (pull_request) Successful in 1m9s
CI / driver (pull_request) Successful in 3m28s
driver / lint (pull_request) Successful in 13s
driver / standard (pull_request) Successful in 17s
driver / types (pull_request) Successful in 2m9s
driver / test (3.12) (pull_request) Successful in 2m10s
driver / publish (pull_request) Skipped
2026-11-03 07:56:46 +01:00
christianmanivong 847c902193 feat(ssh): SSHD_CONFIG_GENERATED keeps the hardening to the drop-in (#88)
A platform that writes sshd_config from its own database (OpenMediaVault)
had to opt out of harden_ssh_server altogether. Where that generated file
includes /etc/ssh/sshd_config.d, the drop-in is safe; only the fallback
that rewrites sshd_config itself would be undone on the next apply. With
SSHD_CONFIG_GENERATED set, a sshd_config without the include is left alone
and the result says why.
2026-11-03 07:56:46 +01:00
christianmanivong ffbb194300 fix(scheduled-jobs): a file without a last newline no longer swallows the next header
driver / build (pull_request) Successful in 32s
driver / lowest (pull_request) Successful in 41s
driver / security (pull_request) Successful in 1m13s
CI / driver (pull_request) Successful in 2m43s
driver / standard (pull_request) Successful in 15s
driver / lint (pull_request) Successful in 14s
driver / types (pull_request) Successful in 1m4s
driver / test (3.12) (pull_request) Successful in 1m6s
driver / test (3.13) (pull_request) Successful in 1m4s
driver / publish (pull_request) Skipped
GNU sed keeps a missing final newline, so the next section's header ran
on into the file's last line and its jobs were lost. The report ends
every such file with a newline of its own.

Refs #86
2026-11-03 05:50:44 +01:00
christianmanivong edef6cf07c fix(scheduled-jobs): write a linked crontab through its link, with the target's mode
driver / lowest (pull_request) Successful in 40s
driver / build (pull_request) Successful in 36s
driver / security (pull_request) Successful in 1m8s
CI / driver (pull_request) Successful in 2m54s
driver / standard (pull_request) Successful in 16s
driver / lint (pull_request) Successful in 20s
driver / types (pull_request) Successful in 1m11s
driver / test (3.12) (pull_request) Successful in 1m25s
driver / test (3.13) (pull_request) Successful in 1m18s
driver / publish (pull_request) Skipped
stat without -L gave the link's own mode, 777, and install replaced the
link with a plain file of that mode: cron ignores a file anyone may write,
so every job in it stopped (/etc/cron.d/vzdump on older PVE links to
/etc/pve/vzdump.cron). The file the link names is written instead.

Refs #86
2026-11-03 05:39:57 +01:00
christianmanivong df476a12f2 feat(scheduled-jobs): the install command, time zone, owners and platform sources are the driver's
driver / lowest (pull_request) Successful in 39s
driver / build (pull_request) Successful in 30s
driver / security (pull_request) Successful in 1m6s
CI / driver (pull_request) Successful in 2m48s
driver / standard (pull_request) Successful in 17s
driver / lint (pull_request) Successful in 13s
driver / types (pull_request) Successful in 1m7s
driver / test (3.13) (pull_request) Successful in 1m4s
driver / test (3.12) (pull_request) Successful in 1m12s
driver / publish (pull_request) Skipped
A BSD install takes -D as a destdir, OpenWrt keeps its time zone in uci
and its package database in opkg, and a platform's generated files are its
own; each is a class attribute a driver sets.

Refs #86
2026-11-03 05:33:14 +01:00
christianmanivong 834cb2e0ea fix(scheduled-jobs): an own job goes to /etc/cron.d only where the host has it
driver / lowest (pull_request) Successful in 39s
driver / test (3.13) (pull_request) Successful in 1m21s
driver / build (pull_request) Successful in 32s
driver / security (pull_request) Successful in 1m7s
CI / driver (pull_request) Successful in 3m3s
driver / standard (pull_request) Successful in 19s
driver / lint (pull_request) Successful in 15s
driver / types (pull_request) Successful in 1m12s
driver / test (3.12) (pull_request) Successful in 1m21s
driver / publish (pull_request) Skipped
BusyBox crond reads no /etc/cron.d, so an Alpine host without systemd
takes the job in the user's crontab instead of a file it would never run.

Refs #86
2026-11-03 05:30:19 +01:00
christianmanivong f573fa53d2 feat(scheduled-jobs): ScheduledJobsMixin and its POSIX implementation (#86)
driver / lowest (pull_request) Successful in 46s
driver / build (pull_request) Successful in 43s
driver / security (pull_request) Successful in 1m27s
CI / driver (pull_request) Successful in 3m11s
driver / standard (pull_request) Successful in 20s
driver / lint (pull_request) Successful in 23s
driver / types (pull_request) Successful in 1m16s
driver / test (3.13) (pull_request) Successful in 1m18s
driver / test (3.12) (pull_request) Successful in 1m27s
driver / publish (pull_request) Skipped
What a host runs on its own schedule as one shape: cron lines, run-parts
scripts, anacron, systemd timers, periodic, a platform's own jobs, Windows
tasks. Every job says who owns its definition and what the driver can do
to it; every write names the digest the caller saw and is refused when the
host's definition changed since.

PosixScheduledJobsMixin reads everything in one framed report as root,
switches a cron line behind #napalm-disabled:, a timer by systemctl and a
run-parts script by its execute bit, runs a job now bounded by timeout, and
writes an application's job as a timer where systemd runs, else in
/etc/cron.d or the user's crontab. napalm_device_types.cron parses cron
expressions, gives their next runs and writes them as OnCalendar lines.

get_cron_jobs and CronJobDict go: nothing called them.

Version 4.26.0, so drivers can require it before the release is tagged.

Closes #86
2026-11-03 05:29:27 +01:00
32 changed files with 5901 additions and 83 deletions

No files matched your search

+125
View File
@@ -7,6 +7,131 @@ recorded here; their history is the git log.
## [Unreleased]
## [4.34.0] - 2026-11-10
### Added
- `RunningBackupsMixin.get_running_backups()`: the backups running on a device right now --
a hypervisor's backup of its guests (with the guests it is taking now), a backup server's
backup, sync, verify, garbage collection, prune or tape run, a host's own file-level backup.
Declared only; a driver mixes it in when it can tell, and a driver without it cannot tell.
`RunningBackupDict`, `RunningBackupKind` and `RUNNING_BACKUP_KINDS` are public. An
application asks right before it restarts a device (NetOrk/netork#2318).
## [4.33.0] - 2026-11-07
### Added
- `get_host_status()` tells an interrupted package manager on hosts without dpkg too, and
says which one it read in the new `package_manager` field (`dpkg`, `dnf`, `yum`, `apk`,
`opkg`, or None). dnf and yum: a package installed in two versions (not the kernels and
other packages dnf keeps side by side) as state `duplicate`, or a transaction that never
ended (dnf 4's history, yum's `transaction-all` journal). apk: a package apk marked with
broken files or a broken script (`broken-files`, `broken-script`), the ones `apk fix` would
take. opkg: a package left `unpacked`, `half-configured` or `half-installed`, or flagged for
reinstallation. All read without root; while one of them runs the answer is unknown
(NetOrk/netork#851).
## [4.32.0] - 2026-11-07
### Added
- `SshServerAuthMixin.get_ssh_server_auth()`: how an OpenSSH server lets people log in --
password, keyboard-interactive, public key, host-based and GSSAPI authentication,
`AuthenticationMethods` as lists of methods, `PermitRootLogin` and `UsePAM`, from `sshd -T`
as root, and sshd's PAM `auth` stack (each module with its control and arguments, includes
followed), from `/etc/pam.d` as the login user. `OpenSshServerMixin` carries it, so every
driver that can harden sshd has it. `parse_sshd_auth`, `parse_pam_auth`, `parse_pam_files`,
`pam_auth_stack` and `SSHD_EFFECTIVE_COMMAND` are public (NetOrk/netork#662).
## [4.31.1] - 2026-11-06
### Fixed
- `get_log_sources()` names as a file's writer only a process that holds it open for writing.
A log reader -- CrowdSec, Promtail, rsyslog's `imfile`, a `tail -F` -- was given the file
it reads as if it wrote it. A file only readers hold is now listed without a producer
(`unit` and `process` None); one only a reader inside a container holds is left out, as its
path is that container's view of a mount. The access mode comes from
`/proc/<pid>/fdinfo/<fd>`, read in the same command (NetOrk/netork#2092).
## [4.31.0] - 2026-11-06
### Added
- `get_log_sources()` also says where the logs go beyond the host: `forwarding` holds whether
rsyslog runs, its forwarding rules (legacy `@`/`@@`/`:omrelp:` and RainerScript
`omfwd`/`omrelp`, with selector, ruleset and whether they take every message), its `imfile`
inputs, and whether the journal reaches it (`imjournal`, or `imuxsock` with journald's
`ForwardToSyslog`). Read in the same command; `parse_log_forwarding` and `parse_rsyslog` are
public (NetOrk/netork#2053).
## [4.30.0] - 2026-11-04
### Added
- `LogSourcesMixin.get_log_sources()`: every log a Linux host writes. Sources are the units
journald holds entries for, the log files running processes hold open (with size, last write
and the service or container writing them), and the engine's file of each container's output.
A file a container writes is named by the container's own path. `LOG_SOURCES_COMMAND` and
`parse_log_sources` are public, and a host whose `find` cannot list open files (busybox)
raises `LogSourcesUnavailable`. A driver supplies `_run_log_sources_command(command,
privileged=)`, as for listening sockets (NetOrk/netork#2049).
## [4.29.0] - 2026-11-04
### Added
- `remove_local_account_keys(name, keys, *, clear_password=False)`: with `clear_password`
the account's password goes along with the keys, and so do its sudo rule and admin
groups, as sudo asks for that password. For a person who deleted their key and whose
password went with it. The account stays open; the steps are `keys`, `password` and
`sudo`, each run when another failed (NetOrk/netork#2033).
## [4.28.0] - 2026-11-04
### Added
- `LocalAccountsMixin.add_local_account(..., replaces=[...])`: the keys the new one takes
the place of go from the account, matched by their base64 whatever a line's options or
comment. Keys the person added on the host stay. The step is `old keys`; it runs even
when the new key could not be added (NetOrk/netork#2013).
- `add_local_account(..., clear_password=True)`: the account's password goes, and it logs
in with a key only. Step `password`. It cannot be combined with a `password_hash` or
with `admin=True`, as the sudo rule asks for that password. A driver supplies
`_account_clear_password`; one that does not fails the step and says so.
- `LocalAccountsMixin.remove_local_account_keys(name, keys)`: takes the named keys off an
account and leaves it open, its password and sudo as they are -- for a person who
deleted a key and has no new one. Step `keys`.
- A driver whose host keeps keys elsewhere overrides `_account_drop_keys` next to
`_account_add_key` and `_account_remove_keys`.
## [4.27.0] - 2026-11-03
### Added
- `ScheduledJobsMixin`: what a host runs on its own schedule as one shape,
`ScheduledJobDict` — cron lines, run-parts scripts, anacron, systemd timers, `periodic`,
a platform's own jobs, Windows tasks — with who owns each definition (`managed_by`) and
what the driver can do to it (`actions`). `get_scheduled_jobs` reads them with the host's
time zone; `manage_scheduled_job` enables, disables or runs one now; `put_scheduled_job`
and `delete_scheduled_job` create, change and delete a job an application writes. Every
write names the `digest` the caller saw (`job_digest`) and raises `ScheduledJobChanged`
when the host's definition differs; `ScheduledJobRefused` for a job that cannot take a
verb (NetOrk/netork#1933, #86).
- `PosixScheduledJobsMixin`: the implementation over the command channel. One framed report
as root reads `/etc/crontab`, `/etc/cron.d`, the user crontabs, the run-parts directories,
`/etc/anacrontab` and every systemd timer with the unit it starts, and which package owns
each definition. A cron line is switched off behind `#napalm-disabled: `; an
application's job is a timer where systemd runs, else a file in `/etc/cron.d` or marked
lines in the user's crontab.
- `napalm_device_types.cron`: cron expressions as Vixie cron reads them, their next runs in
a time zone, and as systemd `OnCalendar` lines that fire on exactly the same minutes.
### Removed
- `OSDriver.get_cron_jobs` and `CronJobDict`. No application called them; the readers in
napalm-linux and napalm-bsd go with them. `get_scheduled_jobs` replaces them.
## [4.26.0] - 2026-11-03
### Added
- `OpenSshServerMixin.SSHD_CONFIG_GENERATED`: a driver whose platform writes sshd_config
from its own database, but includes `/etc/ssh/sshd_config.d` (OpenMediaVault), sets it
and keeps `harden_ssh_server`. The hardening then goes only into the drop-in; a
sshd_config without that include is left alone and the result says why, since the
platform would write over a change to it on its next apply (#88, NetOrk/netork#1950).
## [4.25.0] - 2026-11-02
### Added
+47 -7
View File
@@ -174,9 +174,12 @@ A driver supplies `_run_apt_command`, `_run_apt_root_command` and `_write_apt_fi
`HostStatusMixin` (`get_host_status`) is mixed in the same way: whether a Linux host needs
a reboot to finish an update (`/var/run/reboot-required`, `needs-restarting -r`, or a newer
kernel of the running flavour installed), whether it patches itself (unattended-upgrades,
dnf-automatic) and when that last succeeded (apt's `upgrade-stamp`), and whether dpkg was
left halfway (a journal in `/var/lib/dpkg/updates` or a half-configured package; unknown
while a package manager is running), and how full `/`, `/boot` and `/var` are (`df -Pk`,
dnf-automatic) and when that last succeeded (apt's `upgrade-stamp`), and whether its package
manager was left halfway (unknown while one is running): dpkg (a journal in
`/var/lib/dpkg/updates` or a half-configured package), dnf or yum (two versions of a package
in the rpm database, a transaction that never ended), apk (a package marked with broken files
or a broken script) or opkg (a package left `unpacked`), with `package_manager` naming which,
and how full `/`, `/boot` and `/var` are (`df -Pk`,
each filesystem once, in bytes; how full is too full is the caller's decision). `package_updates` holds the shared apt and dnf parsers: apt's suites become
an update's `origin`, a `-security` suite makes it a security update, and dnf's security
advisories do the same.
@@ -191,6 +194,23 @@ without `ss` is read with `netstat -lntup` (OpenWrt's busybox, old net-tools); o
cgroup names the procd service (`/services/<name>/<instance>`). A host with neither raises
`ListeningSocketsUnavailable`.
`LogSourcesMixin` (`get_log_sources`) is mixed in the same way: every log the host writes, in
one round trip. The units journald holds entries for (`journalctl -F _SYSTEMD_UNIT`, services
only), the log files running processes hold open (`/proc/<pid>/fd`: `*.log`, `*.log.N` and
anything under `/var/log`) with size and last write from `stat -L` on the descriptor, and the
engine's own file of a container's output (`/var/lib/docker/containers/<id>/…`) as that
container's stream. Each file names the service or container of its first writer from
`/proc/<pid>/cgroup`; a file a container writes carries the container's own path. journald's
store, deleted files and a database's write-ahead log (LevelDB's `000004.log`) are left out; a
host whose `find` cannot list open files (busybox) raises `LogSourcesUnavailable`. A
driver supplies `_run_log_sources_command(command, privileged=)`; without root only the login
user's processes show their files, and the reading says so (`attributed: false`).
The same round trip reads where the logs go beyond the host (`forwarding`,
`napalm_device_types.log_forwarding`): whether rsyslog runs, its forwarding rules (`@`, `@@`,
`:omrelp:`, `action(type="omfwd"|"omrelp")`, each *complete* only for `*.*` with no condition
around it), the files it reads in (`imfile`), and whether the journal reaches it (`imjournal`,
or `imuxsock` with journald's `ForwardToSyslog`, the distribution's commented default counting).
LLDP neighbours are rows: `LldpNeighborRow` is one neighbour as its local port heard it
(chassis and port id with their type, name, description, management addresses,
capabilities in NAPALM's names), and `lldp_neighbors(rows)` / `lldp_neighbors_detail(rows,
@@ -278,6 +298,16 @@ A driver mixes it in and supplies `run_command`; a platform that regenerates ssh
itself (an appliance with its own database) must not, or opts out with
`harden_ssh_server = None`.
`SshServerAuthMixin` (`get_ssh_server_auth`) says how an OpenSSH server lets people log in:
password, keyboard-interactive, public key, host-based and GSSAPI authentication on or off,
`AuthenticationMethods` as lists of methods (empty for `any`), `PermitRootLogin` and
`UsePAM` (None where sshd has no PAM), from `sshd -T` as root -- the same fixed command the
hardening reads. With PAM on, it also names the modules of sshd's PAM `auth` stack
with each control and its arguments, read from `/etc/pam.d` in one listing as the
login user and followed through `@include`, `include` and `substack`. Which of them is a
second factor is the consumer's call. `Match` blocks are not read. `OpenSshServerMixin`
carries it; a driver can also mix it in alone and supply `run_command`.
`SudoHistoryMixin` (`get_history_sudo_lines`, `remove_history_sudo_lines`) deals with what
a driver left behind when it typed `echo '<password>' | sudo -S -p "" …` into an interactive
shell: that line, password and all, in the login user's shell history, and files its partial
@@ -301,6 +331,20 @@ trip (`list-unit-files` plus one `systemctl show` over every loaded unit) instea
`is-enabled` and a `show` per unit. A host without systemd raises `SystemdUnavailable`,
a `NotImplementedError`, so a driver can fall back to another init system.
`ScheduledJobsMixin` (`get_scheduled_jobs`, `manage_scheduled_job`, `put_scheduled_job`,
`delete_scheduled_job`) is what a host runs on its own schedule — cron lines, run-parts
scripts, anacron, systemd timers, `periodic`, a platform's own jobs, Windows tasks — as one
shape, `ScheduledJobDict`. Every job says who owns its definition (`managed_by`: package,
admin, platform, application) and what the driver can do to it (`actions`): every job a
driver can switch takes `enable`, `disable` and `run`; only a job an application wrote
through `put_scheduled_job` takes `edit` and `delete`. Every write names the `digest` the
caller saw and raises `ScheduledJobChanged` when the host's definition is no longer that.
`PosixScheduledJobsMixin` implements it over `run_command`: one framed report as root,
a cron line switched off behind `#napalm-disabled: `, a timer by `systemctl`, an
application's job as a timer where systemd runs, else in `/etc/cron.d` or the user's
crontab. A driver sets the paths its host keeps jobs in. `napalm_device_types.cron` parses
cron expressions, gives their next runs and writes them as `OnCalendar` lines.
### Access channels: why there is no container model here
`CommandChannelMixin` declares a driver's public channel to its host:
@@ -621,10 +665,6 @@ class LinuxDriver(OSDriver):
def get_processes(self):
# return List[ProcessDict]
...
def get_cron_jobs(self):
# return List[CronJobDict]
...
```
### Storage / NAS
+1
View File
@@ -151,6 +151,7 @@ the driver.
| Module | Knows |
|---|---|
| `host_status` | apt, dnf, dpkg, systemd timers, `df` |
| `package_managers` | rpm, dnf and yum history, apk's and opkg's databases, `/proc/<pid>/comm` |
| `kernel` | `/proc`, `/lib/modules`, gzip, base64 |
| `listening` | `ss`, `netstat`, `/proc/<pid>/cgroup` |
| `systemd` | `systemctl` |
+75
View File
@@ -50,14 +50,19 @@ instead of being restated on every role that happens to need it:
* :class:`~napalm_device_types.kernel.KernelFactsMixin`
* :class:`~napalm_device_types.listening.ListeningSocketsMixin`
* :class:`~napalm_device_types.local_accounts.LocalAccountsMixin`
* :class:`~napalm_device_types.log_sources.LogSourcesMixin`
* :class:`~napalm_device_types.host_firewall.HostFirewallMixin`
* :class:`~napalm_device_types.mac_acl.MacAclMixin`
* :class:`~napalm_device_types.nat_vpn.NatVpnMixin`
* :class:`~napalm_device_types.packages.PackageManagementMixin`
* :class:`~napalm_device_types.ping_sweep.PingSweepMixin`
* :class:`~napalm_device_types.running_backups.RunningBackupsMixin`
* :class:`~napalm_device_types.service_clusters.ServiceClustersMixin`
* :class:`~napalm_device_types.service_packages.ServicePackagesMixin`
* :class:`~napalm_device_types.ssh_server_auth.SshServerAuthMixin`
* :class:`~napalm_device_types.sudo_history.SudoHistoryMixin`
* :class:`~napalm_device_types.scheduled_jobs.ScheduledJobsMixin`
* :class:`~napalm_device_types.scheduled_jobs_posix.PosixScheduledJobsMixin`
* :class:`~napalm_device_types.services.ServiceControlMixin`
* :class:`~napalm_device_types.systemd.SystemdServicesMixin`
* :class:`~napalm_device_types.updates.UpdateMixin`
@@ -164,6 +169,21 @@ from napalm_device_types.local_accounts import (
LocalAccountDict,
LocalAccountsMixin,
)
from napalm_device_types.log_forwarding import (
FileInputDict,
ForwardRuleDict,
LogForwardingDict,
parse_rsyslog,
)
from napalm_device_types.log_sources import (
LOG_SOURCES_COMMAND,
LogSourceDict,
LogSourcesDict,
LogSourcesMixin,
LogSourcesUnavailable,
parse_log_forwarding,
parse_log_sources,
)
from napalm_device_types.mac_acl import MacAclMixin
from napalm_device_types.management_controller import ManagementControllerDriver
from napalm_device_types.media import MediaDriver
@@ -205,6 +225,23 @@ from napalm_device_types.ping_sweep import PingSweepMixin, driver_supports_ping
from napalm_device_types.provisioning import VMProvisionIncomplete
from napalm_device_types.residential_gateway import ResidentialGatewayDriver
from napalm_device_types.roles import primary_role_of, role_keys_of, roles_of
from napalm_device_types.running_backups import (
RUNNING_BACKUP_KINDS,
RunningBackupDict,
RunningBackupKind,
RunningBackupsMixin,
)
from napalm_device_types.scheduled_jobs import (
ScheduledJobChanged,
ScheduledJobDict,
ScheduledJobRefused,
ScheduledJobResultDict,
ScheduledJobsMixin,
ScheduledJobSpecDict,
ScheduledJobsReadingDict,
job_digest,
)
from napalm_device_types.scheduled_jobs_posix import PosixScheduledJobsMixin
from napalm_device_types.service_clusters import (
SERVICE_CLUSTERS_COMMAND,
ServiceClustersMixin,
@@ -242,6 +279,14 @@ from napalm_device_types.ssh_server import (
SshHardeningResult,
plan_ssh_hardening,
)
from napalm_device_types.ssh_server_auth import (
SSHD_EFFECTIVE_COMMAND,
PamAuthEntryDict,
SshServerAuthDict,
SshServerAuthMixin,
pam_auth_stack,
parse_sshd_auth,
)
from napalm_device_types.storage import StorageDriver
from napalm_device_types.sudo_history import (
HistoryCleanupDict,
@@ -385,6 +430,17 @@ __all__ = [
"parse_host_firewall",
"ListeningSocketsMixin",
"ListeningSocketsUnavailable",
"LOG_SOURCES_COMMAND",
"FileInputDict",
"ForwardRuleDict",
"LogForwardingDict",
"parse_log_forwarding",
"parse_rsyslog",
"LogSourceDict",
"LogSourcesDict",
"LogSourcesMixin",
"LogSourcesUnavailable",
"parse_log_sources",
"parse_listening_sockets",
"LLDPCTL_KEYVALUE_COMMAND",
"LldpNeighborDict",
@@ -400,6 +456,15 @@ __all__ = [
"PortSpec",
"ResidentialGatewayDriver",
"ServiceControlMixin",
"PosixScheduledJobsMixin",
"ScheduledJobChanged",
"ScheduledJobDict",
"ScheduledJobRefused",
"ScheduledJobResultDict",
"ScheduledJobSpecDict",
"ScheduledJobsMixin",
"ScheduledJobsReadingDict",
"job_digest",
"StorageDriver",
"SwitchDriver",
"SYSTEMD_SERVICES_COMMAND",
@@ -410,6 +475,10 @@ __all__ = [
"VIRTUAL_IPS_COMMAND",
"VirtualIpsMixin",
"VmStopRequestMixin",
"RUNNING_BACKUP_KINDS",
"RunningBackupDict",
"RunningBackupKind",
"RunningBackupsMixin",
"VMProvisionIncomplete",
"SERVICE_CLUSTERS_COMMAND",
"ServiceClustersMixin",
@@ -420,6 +489,12 @@ __all__ = [
"is_placeholder_identity",
"parse_system_identity",
"OpenSshServerMixin",
"PamAuthEntryDict",
"SshServerAuthDict",
"SshServerAuthMixin",
"SSHD_EFFECTIVE_COMMAND",
"pam_auth_stack",
"parse_sshd_auth",
"SSH_ALGORITHM_CLASSES",
"SshHardeningPlan",
"SshHardeningResult",
+222
View File
@@ -0,0 +1,222 @@
"""Crontabs and anacrontabs as text: the jobs in them, and changing one line of them.
Pure: no I/O. :mod:`napalm_device_types.scheduled_jobs_posix` reads the files and
writes them back; this module says what is in them and what the new text is.
**A line's key survives switching it off.** A job switched off is the same line
behind the marker :data:`DISABLED_MARKER`; the key is built from the line without
it, so the job keeps its key and the marker is all that changes. Two equal lines
in one file get ``:2``, ``:3`` after the first.
**A job an application wrote** is the line after ``# napalm-scheduled-job: <name>``
and is keyed ``job:<name>``, whatever its schedule.
**Everything else is left as it was**: comments, blank lines, variables, lines
cron would not take.
"""
from __future__ import annotations
import hashlib
import re
from dataclasses import dataclass, field
from shlex import quote
DISABLED_MARKER = "#napalm-disabled: "
JOB_MARKER = "# napalm-scheduled-job: "
_VARIABLE = re.compile(r"^([A-Za-z_][A-Za-z0-9_]*)\s*=\s*(.*)$")
_ANACRON_PERIOD = re.compile(r"^(\d+|@(daily|weekly|monthly|yearly|annually))$")
@dataclass(frozen=True)
class CronLine:
"""One job in a crontab: where it is, what it says, and what was set before it."""
index: int # the line's number in the text, from 0
key: str
schedule: str
run_as: str
command: str
enabled: bool
env: dict[str, str] = field(default_factory=dict)
name: str = "" # the name an application gave it
description: str = ""
def _unquote(value: str) -> str:
value = value.strip()
if len(value) >= 2 and value[0] == value[-1] and value[0] in "'\"":
return value[1:-1]
return value
def _digest(text: str) -> str:
return hashlib.sha256(" ".join(text.split()).encode()).hexdigest()[:8]
def _split(content: str, *, system: bool, user: str) -> tuple[str, str, str] | None:
"""(schedule, run_as, command) of a cron line; None for one cron would not take."""
fields = 1 if content.startswith("@") else 5
owner = 1 if system else 0
parts = content.split(None, fields + owner)
if len(parts) < fields + owner + 1:
return None
schedule = " ".join(parts[:fields])
run_as = parts[fields] if system else user
return schedule, run_as, parts[-1]
def _anacron_split(content: str) -> tuple[str, str, str] | None:
"""(period, job identifier, command) of an anacrontab line."""
parts = content.split(None, 3)
if len(parts) < 4 or not _ANACRON_PERIOD.match(parts[0]) or not parts[1].isdigit():
return None
return parts[0], parts[2], parts[3]
@dataclass
class _Scan:
"""What is set while a text is read line by line."""
env: dict[str, str] = field(default_factory=dict)
seen: dict[str, int] = field(default_factory=dict)
name: str = ""
description: str = ""
def key(self, base: str) -> str:
n = self.seen.get(base, 0) + 1
self.seen[base] = n
return base if n == 1 else f"{base}:{n}"
def take_name(self) -> tuple[str, str]:
named = self.name, self.description
self.name, self.description = "", ""
return named
def _content(raw: str) -> tuple[str, bool] | None:
"""The job text of *raw* and whether it is on; None for a comment or a blank."""
line = raw.strip()
if line.startswith(DISABLED_MARKER.strip()):
return line[len(DISABLED_MARKER.strip()) :].strip(), False
if not line or line.startswith("#"):
return None
return line, True
def _remember(scan: _Scan, raw: str) -> bool:
"""Take a variable or an application's marker from *raw*; whether it was one."""
line = raw.strip()
if line.startswith(JOB_MARKER.strip()):
name, _, description = line[len(JOB_MARKER.strip()) :].strip().partition(" ")
scan.name, scan.description = name, description.strip()
return True
variable = _VARIABLE.match(line)
if variable:
scan.env[variable[1]] = _unquote(variable[2])
return True
return False
def parse_crontab(path: str, text: str, *, system: bool, user: str = "") -> list[CronLine]:
"""The jobs of the crontab *text* at *path*.
*system*: a sixth field names the user (``/etc/crontab``, ``/etc/cron.d``);
otherwise every job runs as *user*.
"""
scan, jobs = _Scan(), []
for index, raw in enumerate(text.splitlines()):
if _remember(scan, raw):
continue
found = _content(raw)
split = _split(found[0], system=system, user=user) if found else None
if found is None or split is None:
continue
content, enabled = found
name, description = scan.take_name()
key = f"job:{name}" if name else scan.key(f"cron:{path}:{_digest(content)}")
schedule, run_as, command = split
jobs.append(
CronLine(
index, key, schedule, run_as, command, enabled, dict(scan.env), name, description
)
)
return jobs
def parse_anacrontab(text: str) -> list[CronLine]:
"""The jobs of an anacrontab: period, delay, identifier, command; all run as root."""
scan, jobs = _Scan(), []
for index, raw in enumerate(text.splitlines()):
if _remember(scan, raw):
continue
found = _content(raw)
split = _anacron_split(found[0]) if found else None
if found is None or split is None:
continue
period, identifier, command = split
key = scan.key(f"anacron:{identifier}")
jobs.append(CronLine(index, key, period, "root", command, found[1], dict(scan.env)))
return jobs
def _replace(text: str, index: int, lines: list[str]) -> str:
rows = text.splitlines()
rows[index : index + 1] = lines
return "\n".join(rows) + "\n"
def switched(text: str, line: CronLine, *, enabled: bool) -> str:
"""*text* with *line* switched on or off: the marker added or taken away, nothing else."""
raw = text.splitlines()[line.index]
found = _content(raw)
if found is None: # pragma: no cover - a CronLine always points at a job
raise ValueError(f"line {line.index} is no job")
content = found[0]
return _replace(text, line.index, [content if enabled else DISABLED_MARKER + content])
def marked_lines(name: str, description: str, line: str, *, enabled: bool) -> list[str]:
"""An application's job as crontab lines: its marker, then the job, maybe switched off."""
marker = f"{JOB_MARKER}{name} {description}".rstrip()
return [marker, line if enabled else DISABLED_MARKER + line]
def with_job(text: str, jobs: list[CronLine], name: str, lines: list[str]) -> str:
"""*text* with the application's job *name* as *lines*: replaced, or added at the end."""
current = next((j for j in jobs if j.name == name), None)
if current is None:
body = text if not text or text.endswith("\n") else text + "\n"
return body + "\n".join(lines) + "\n"
rows = text.splitlines()
rows[current.index - 1 : current.index + 1] = lines
return "\n".join(rows) + "\n"
def without_job(text: str, jobs: list[CronLine], name: str) -> str:
"""*text* without the application's job *name* and its marker."""
current = next(j for j in jobs if j.name == name)
rows = text.splitlines()
del rows[current.index - 1 : current.index + 1]
return "\n".join(rows) + ("\n" if rows else "")
def _unescape(part: str) -> str:
return part.replace("\\%", "%")
def run_script(command: str, env: dict[str, str]) -> str:
"""*command* as ``sh -c`` takes it, the way cron would run it.
The crontab's variables are exported first. The first ``%`` that is not
escaped ends the command; what follows is its stdin, with every further
``%`` a newline, and ``\\%`` is a ``%``.
"""
parts = re.split(r"(?<!\\)%", command)
exports = "".join(f"export {k}={quote(v)}; " for k, v in env.items() if k != "MAILTO")
head = _unescape(parts[0])
if len(parts) == 1:
return exports + head
stdin = "\n".join(_unescape(p) for p in parts[1:])
return f"{exports}printf '%s\\n' {quote(stdin)} | {{ {head}\n}}"
+417
View File
@@ -0,0 +1,417 @@
"""The scheduled-jobs report of a POSIX host: the command that prints it, and its parse.
:mod:`napalm_device_types.scheduled_jobs_posix` sends :func:`reading_command` and
hands what came back to :func:`parse_reading`. Pure: no I/O.
**One framed report, as root.** Every crontab, cron.d file, anacrontab and job
script is printed behind a header line and with ``|`` before each of its lines,
so no line of a file can pass for a header; run-parts scripts by their execute
bit and checksum; systemd's timers and the units they start by ``systemctl show``;
then the time zone and which package owns each definition file. A report whose
end marker is missing raises: a list cut short must never read as jobs that went
away.
"""
from __future__ import annotations
import re
from dataclasses import dataclass, field
from datetime import UTC, datetime
from shlex import quote
from napalm_device_types._crontab import JOB_MARKER, CronLine, parse_anacrontab, parse_crontab
from napalm_device_types.scheduled_jobs import DELETE, EDIT, job_digest
from napalm_device_types.terminal import strip_terminal_codes
_BEGIN = "SJ_BEGIN"
_END = "SJ_END"
SCHEDULE_HEADER = "# schedule: "
TIMER_PROPERTIES = (
"Id,Unit,Description,TimersCalendar,TimersMonotonic,NextElapseUSecRealtime,"
"LastTriggerUSec,UnitFileState,ActiveState,FragmentPath"
)
UNIT_PROPERTIES = "Id,ExecStart,User,Result,FragmentPath"
#: The names run-parts runs (Debian's ``--lsbsysinit``-less default).
_RUN_PARTS_NAME = re.compile(r"^[A-Za-z0-9_-]+$")
_SECTION = re.compile(r"^\[([a-z-]+)(?: (.*))?\]$")
#: Reads definition paths on stdin, prints ``<package>: <path>`` for each a package owns.
OWNERS_SH = (
"if command -v dpkg >/dev/null 2>&1; then xargs dpkg -S 2>/dev/null; "
"elif command -v rpm >/dev/null 2>&1; then while read -r p; do "
'o=$(rpm -qf --qf \'%{NAME}\' -- "$p" 2>/dev/null) && printf \'%s: %s\\n\' "$o" "$p"; done; '
"elif command -v apk >/dev/null 2>&1; then while read -r p; do "
'o=$(apk info -qW -- "$p" 2>/dev/null) && [ -n "$o" ] && printf \'%s: %s\\n\' "$o" "$p"; done; '
"else cat >/dev/null; fi"
)
#: Prints the host's time zone: an IANA name, or a path ending in ``zoneinfo/<name>``.
TIME_ZONE_SH = "cat /etc/timezone 2>/dev/null || readlink /etc/localtime 2>/dev/null"
@dataclass(frozen=True)
class Paths:
"""Where a host keeps its scheduled jobs."""
system_files: tuple[str, ...]
cron_d_dirs: tuple[str, ...]
spool_dirs: tuple[str, ...]
run_parts: tuple[tuple[str, str], ...] # (period, directory)
anacrontab: str
script_dir: str
systemd_marker: str = "/run/systemd/system"
owners_sh: str = OWNERS_SH
time_zone_sh: str = TIME_ZONE_SH
@dataclass
class Reading:
"""What one report said, beyond the jobs: what writes need to know."""
jobs: list[dict[str, object]] = field(default_factory=list)
time_zone: str | None = None
systemd: bool = False
#: per key: the unit a timer starts, the crontab line a cron job is
units: dict[str, str] = field(default_factory=dict)
lines: dict[str, CronLine] = field(default_factory=dict)
def _q(path: str) -> str:
return quote(path)
def _files(section: str, pattern: str) -> str:
return (
f'for f in {pattern}; do [ -f "$f" ] || continue; printf \'[%s %s]\\n\' {section} "$f"; '
# GNU sed keeps a missing last newline; without one of its own the next
# header would run on into the file's last line.
'sed \'s/^/|/\' "$f"; [ -n "$(tail -c 1 "$f")" ] && echo; P="$P\n$f"; done; '
)
def _run_parts(period: str, directory: str) -> str:
return (
f'for f in {_q(directory)}/*; do [ -f "$f" ] || continue; '
'[ -x "$f" ] && x=1 || x=0; c=$(cksum < "$f" | cut -d" " -f1); '
f'printf \'[run-parts %s %s %s %s]\\n\' {period} "$x" "$c" "$f"; P="$P\n$f"; done; '
)
def _timers(paths: Paths) -> str:
show = f"systemctl show --no-pager -p {TIMER_PROPERTIES}"
units = f"systemctl show --no-pager -p {UNIT_PROPERTIES}"
return (
f"if [ -d {_q(paths.systemd_marker)} ]; then echo '[systemd]'; "
"t=$(systemctl list-unit-files --type=timer --no-legend --no-pager 2>/dev/null "
"| awk '$1 !~ /@\\./ {print $1}'); "
'if [ -n "$t" ]; then '
f"s=$({show} --timestamp=unix $t 2>/dev/null || {show} $t 2>/dev/null); "
"echo '[timers]'; printf '%s\\n' \"$s\" | sed 's/^/|/'; "
"u=$(printf '%s\\n' \"$s\" | sed -n 's/^Unit=//p'); "
f"echo '[units]'; {units} $u 2>/dev/null | sed 's/^/|/'; "
"P=\"$P\n$(printf '%s\\n' \"$s\" | sed -n 's/^FragmentPath=//p')\"; fi; fi; "
)
def reading_command(paths: Paths) -> str:
"""One POSIX ``sh`` script that prints every scheduled job of the host; run it as root."""
parts = ["printf '%s%s\\n' SJ_ BEGIN; P=''; "]
parts += [_files("system", _q(f)) for f in paths.system_files]
parts += [_files("system", f"{_q(d)}/*") for d in paths.cron_d_dirs]
parts += [_files("user", f"{_q(d)}/*") for d in paths.spool_dirs]
parts.append(_files("anacron", _q(paths.anacrontab)))
parts += [_run_parts(period, d) for period, d in paths.run_parts]
parts.append(_files("script", f"{_q(paths.script_dir)}/*"))
parts.append(_timers(paths))
parts.append(
f"echo '[tz]'; {{ {paths.time_zone_sh}; }}; "
f"echo '[owners]'; printf '%s\\n' \"$P\" | sed '/^$/d' | {{ {paths.owners_sh}; }}; "
"printf '%s%s\\n' SJ_ END"
)
return "".join(parts)
# -- parse ------------------------------------------------------------------------------------
@dataclass
class _Section:
name: str
args: str
lines: list[str] = field(default_factory=list)
def _sections(output: str) -> list[_Section]:
text = strip_terminal_codes(output)
begin, end = text.rfind(_BEGIN), text.rfind(_END)
if begin < 0 or end < begin:
raise ValueError("the scheduled-jobs report is cut short (no end marker)")
sections: list[_Section] = []
for line in text[begin + len(_BEGIN) : end].splitlines():
header = _SECTION.match(line)
if header:
sections.append(_Section(header[1], header[2] or ""))
elif sections:
sections[-1].lines.append(line[1:] if line.startswith("|") else line)
return sections
def _owned(lines: list[str]) -> set[str]:
owned: set[str] = set()
for line in lines:
package, sep, path = line.partition(": ")
if sep and path.startswith("/") and not package.startswith("diversion"):
owned.add(path.strip())
return owned
def _time_zone(lines: list[str]) -> str | None:
value = next((ln.strip() for ln in lines if ln.strip()), "")
if "zoneinfo/" in value:
value = value.split("zoneinfo/", 1)[1]
return value or None
def _blocks(lines: list[str]) -> list[dict[str, list[str]]]:
blocks: list[dict[str, list[str]]] = [{}]
for line in lines:
if not line.strip():
blocks.append({})
continue
key, sep, value = line.partition("=")
if sep:
blocks[-1].setdefault(key, []).append(value)
return [b for b in blocks if b.get("Id")]
def _one(block: dict[str, list[str]], key: str) -> str:
return (block.get(key) or [""])[0].strip()
def _timestamp(value: str) -> str | None:
"""systemd's ``@<seconds>`` (or a UTC time) as ISO 8601; None when it never was."""
value = value.strip()
if value.startswith("@") and value[1:].isdigit() and int(value[1:]) > 0:
return datetime.fromtimestamp(int(value[1:]), UTC).isoformat()
match = re.search(r"(\d{4}-\d\d-\d\d \d\d:\d\d:\d\d) UTC$", value)
if match:
return datetime.strptime(match[1], "%Y-%m-%d %H:%M:%S").replace(tzinfo=UTC).isoformat()
return None
def _timer_entries(block: dict[str, list[str]], key: str) -> list[str]:
"""``{ OnCalendar=daily ; next_elapse=… }`` → ``OnCalendar=daily``, one per entry.
``systemctl show`` names a monotonic setting by its property (``OnUnitActiveUSec``);
it is given as the unit file says it (``OnUnitActiveSec``).
"""
entries = (v.strip(" {}").split(" ; ")[0].strip() for v in block.get(key, []) if v.strip())
return [re.sub(r"^(On\w+)USec=", r"\1Sec=", e) for e in entries]
def _argv(exec_start: list[str]) -> str:
commands = []
for value in exec_start:
match = re.search(r"argv\[\]=(.*?) ; ", value)
if match:
commands.append(match[1].strip())
return "; ".join(commands)
@dataclass
class _Script:
name: str
description: str
schedule: str
command: str
def _script(path: str, lines: list[str]) -> _Script | None:
"""An application's job script: its marker, schedule and command; None for any other file."""
marker = next((ln for ln in lines if ln.startswith(JOB_MARKER)), None)
schedule = next((ln for ln in lines if ln.startswith(SCHEDULE_HEADER)), None)
if marker is None or schedule is None:
return None
name, _, description = marker[len(JOB_MARKER) :].partition(" ")
body = [ln for ln in lines if ln not in (marker, schedule) and not ln.startswith("#!")]
return _Script(
name, description.strip(), schedule[len(SCHEDULE_HEADER) :], "\n".join(body).strip()
)
def _job(
*,
key: str,
kind: str,
source: str,
run_as: str,
schedule: str,
schedule_kind: str,
command: str,
enabled: bool,
managed_by: str,
actions: list[str],
content: str = "",
**extra: object,
) -> dict[str, object]:
job: dict[str, object] = {
"key": key,
"kind": kind,
"source": source,
"run_as": run_as,
"schedule": schedule,
"schedule_kind": schedule_kind,
"command": command,
"enabled": enabled,
"managed_by": managed_by,
"actions": actions,
}
job["digest"] = job_digest(job, content=content)
job.update({k: v for k, v in extra.items() if v not in (None, "")})
return job
_SWITCH = ["enable", "disable", "run"]
_OWN = [*_SWITCH, EDIT, DELETE]
def _cron_jobs(section: _Section, owned: set[str], reading: Reading) -> None:
path = section.args
text = "\n".join(section.lines)
if section.name == "anacron":
found = parse_anacrontab(text)
else:
user = path.rsplit("/", 1)[-1]
found = parse_crontab(path, text, system=section.name == "system", user=user)
kind = "anacron" if section.name == "anacron" else "cron"
for line in found:
own = bool(line.name)
reading.lines[line.key] = line
reading.jobs.append(
_job(
key=line.key,
kind=kind,
source=path,
run_as=line.run_as,
schedule=line.schedule,
schedule_kind="period" if kind == "anacron" else "cron",
command=line.command,
enabled=line.enabled,
managed_by="application" if own else ("package" if path in owned else "admin"),
actions=_OWN if own else _SWITCH,
name=line.name,
description=line.description,
)
)
def _run_parts_job(section: _Section, owned: set[str], reading: Reading) -> None:
period, executable, checksum, path = section.args.split(" ", 3)
if not _RUN_PARTS_NAME.match(path.rsplit("/", 1)[-1]):
return
reading.jobs.append(
_job(
key=f"run-parts:{path}",
kind="run_parts",
source=path,
run_as="root",
schedule=period,
schedule_kind="period",
command=path,
enabled=executable == "1",
managed_by="package" if path in owned else "admin",
actions=_SWITCH,
content=checksum,
)
)
def _timer_job(
timer: dict[str, list[str]],
unit: dict[str, list[str]],
scripts: dict[str, _Script],
owned: set[str],
) -> dict[str, object]:
fragment = _one(timer, "FragmentPath")
calendar = _timer_entries(timer, "TimersCalendar")
schedule = "; ".join(calendar + _timer_entries(timer, "TimersMonotonic"))
command = _argv(unit.get("ExecStart", []))
script = scripts.get(command.split(" ", 1)[0])
last_run = _timestamp(_one(timer, "LastTriggerUSec"))
switchable = _one(timer, "UnitFileState") not in ("static", "generated", "transient")
common = {
"kind": "systemd_timer",
"source": fragment,
"run_as": _one(unit, "User") or "root",
"enabled": _one(timer, "ActiveState") == "active",
"last_run": last_run,
"last_result": _one(unit, "Result") or None if last_run else None,
"next_run": _timestamp(_one(timer, "NextElapseUSecRealtime")),
}
if script is not None:
return _job(
key=f"job:{script.name}",
schedule=script.schedule,
schedule_kind="cron",
command=script.command,
managed_by="application",
actions=_OWN,
name=script.name,
description=script.description,
**common, # type: ignore[arg-type]
)
packaged = fragment in owned or fragment.startswith(("/usr/lib/", "/lib/"))
return _job(
key=f"timer:{_one(timer, 'Id')}",
schedule=schedule,
schedule_kind="calendar" if calendar else "monotonic",
command=command,
managed_by="package" if packaged else "admin",
actions=_SWITCH if switchable else ["run"],
description=_one(timer, "Description"),
**common, # type: ignore[arg-type]
)
def _timer_jobs(
sections: dict[str, _Section], scripts: dict[str, _Script], owned: set[str], reading: Reading
) -> None:
timers = _blocks(sections["timers"].lines) if "timers" in sections else []
units = (
{_one(b, "Id"): b for b in _blocks(sections["units"].lines)} if "units" in sections else {}
)
for timer in timers:
unit = units.get(_one(timer, "Unit"), {})
job = _timer_job(timer, unit, scripts, owned)
reading.units[str(job["key"])] = _one(timer, "Unit")
reading.jobs.append(job)
def parse_reading(output: str) -> Reading:
"""The jobs, time zone and write context of a report from :func:`reading_command`.
:raises ValueError: when the report is cut short.
"""
sections = _sections(output)
named = {s.name: s for s in sections if s.name in ("timers", "units", "tz", "owners")}
owned = _owned(named["owners"].lines) if "owners" in named else set()
reading = Reading(
time_zone=_time_zone(named["tz"].lines) if "tz" in named else None,
systemd=any(s.name == "systemd" for s in sections),
)
for section in sections:
if section.name in ("system", "user", "anacron"):
_cron_jobs(section, owned, reading)
elif section.name == "run-parts":
_run_parts_job(section, owned, reading)
_timer_jobs(named, _scripts(sections), owned, reading)
return reading
def _scripts(sections: list[_Section]) -> dict[str, _Script]:
"""An application's job scripts, by path."""
scripts = ((s.args, _script(s.args, s.lines)) for s in sections if s.name == "script")
return {path: script for path, script in scripts if script is not None}
+203
View File
@@ -0,0 +1,203 @@
"""Cron expressions: what they mean, when they fire next, and as systemd ``OnCalendar``.
A scheduled job is given as a cron expression on every host, and a host with
systemd runs it as a timer. So the expression has to mean the same in three
places -- on the host's cron, in a preview of the next runs, and in the
``OnCalendar`` lines a timer is written with -- and the meaning lives here once.
**What cron means is Vixie cron's.** Five fields (minute, hour, day of month,
month, day of week), each ``*``, a value, a range ``a-b``, a step ``*/n`` or
``a-b/n``, or a list of those; month and weekday names in any case; ``7`` is
Sunday as well as ``0``; the ``@hourly`` … ``@yearly`` aliases. When both day
fields are restricted -- neither starts with ``*`` -- a day matches if *either*
does; otherwise both must (``*/2`` counts as starting with ``*``).
**``OnCalendar`` by expansion.** Every field is written as its values, so a range
with a step needs no systemd syntax of its own, and a timer fires on exactly the
minutes cron would. The *either* rule is two ``OnCalendar`` lines, which systemd
ORs. ``@reboot`` has no calendar; a timer takes it as ``OnBootSec``.
**Next runs are in the host's time zone.** A local time the clock skips when
summer time starts is left out; cron implementations disagree on it, and a
preview that promised it would promise too much.
"""
from __future__ import annotations
from datetime import date, datetime, timedelta
from typing import NamedTuple
from zoneinfo import ZoneInfo
REBOOT = "@reboot"
ALIASES = {
"@yearly": "0 0 1 1 *",
"@annually": "0 0 1 1 *",
"@monthly": "0 0 1 * *",
"@weekly": "0 0 * * 0",
"@daily": "0 0 * * *",
"@midnight": "0 0 * * *",
"@hourly": "0 * * * *",
}
_MONTHS = ("jan", "feb", "mar", "apr", "may", "jun", "jul", "aug", "sep", "oct", "nov", "dec")
_WEEKDAYS = ("sun", "mon", "tue", "wed", "thu", "fri", "sat")
_SYSTEMD_WEEKDAYS = ("Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat")
#: (lowest, highest, names) per field; a name stands for its index plus the lowest.
_FIELDS = (
(0, 59, ()),
(0, 23, ()),
(1, 31, ()),
(1, 12, _MONTHS),
(0, 7, _WEEKDAYS),
)
#: How far ahead a next run is looked for: a 29 February comes within eight years.
_HORIZON_DAYS = 8 * 366
class CronSchedule(NamedTuple):
"""An expression's values per field, and whether a day matches on either day field."""
minutes: frozenset[int]
hours: frozenset[int]
days: frozenset[int]
months: frozenset[int]
weekdays: frozenset[int] # 0 = Sunday
either_day: bool
def is_reboot(expr: str) -> bool:
"""Whether *expr* is ``@reboot``: once at boot, no calendar."""
return expr.strip().lower() == REBOOT
def _value(token: str, low: int, names: tuple[str, ...]) -> int:
if token.isdigit():
return int(token)
if token.lower() in names:
return names.index(token.lower()) + low
raise ValueError(f"not a value: {token!r}")
def _part(part: str, low: int, high: int, names: tuple[str, ...]) -> set[int]:
span, _, step_text = part.partition("/")
step = 1
if step_text:
if not step_text.isdigit() or int(step_text) == 0:
raise ValueError(f"not a step: {part!r}")
step = int(step_text)
if span == "*":
first, last = low, high
elif "-" in span:
a, _, b = span.partition("-")
first, last = _value(a, low, names), _value(b, low, names)
else:
first = _value(span, low, names)
last = high if step_text else first
if not low <= first <= last <= high:
raise ValueError(f"out of range {low}-{high}: {part!r}")
return set(range(first, last + 1, step))
def _field(text: str, low: int, high: int, names: tuple[str, ...]) -> frozenset[int]:
values: set[int] = set()
for part in text.split(","):
if not part:
raise ValueError(f"empty list item in {text!r}")
values |= _part(part, low, high, names)
return frozenset(values)
def parse_cron(expr: str) -> CronSchedule:
"""The values *expr* stands for.
:raises ValueError: for anything Vixie cron would not take, and for
``@reboot``, which names no time.
"""
text = expr.strip()
if is_reboot(text):
raise ValueError("@reboot names no time; it runs once at boot")
if text.startswith("@"):
if text.lower() not in ALIASES:
raise ValueError(f"unknown alias {text!r}")
text = ALIASES[text.lower()]
fields = text.split()
if len(fields) != len(_FIELDS):
raise ValueError(f"a cron expression has five fields, not {len(fields)}: {expr!r}")
minutes, hours, days, months, weekdays = (
_field(f, low, high, names) for f, (low, high, names) in zip(fields, _FIELDS, strict=True)
)
if 7 in weekdays:
weekdays = (weekdays - {7}) | {0}
either = not fields[2].startswith("*") and not fields[4].startswith("*")
return CronSchedule(minutes, hours, days, months, frozenset(weekdays), either)
def _day_matches(s: CronSchedule, day: date) -> bool:
if day.month not in s.months:
return False
on_day = day.day in s.days
on_weekday = (day.isoweekday() % 7) in s.weekdays
return (on_day or on_weekday) if s.either_day else (on_day and on_weekday)
def _local(day: date, hour: int, minute: int, tz: ZoneInfo) -> datetime | None:
"""*day* at *hour*:*minute* in *tz*; None when the clock skips that time."""
naive = datetime(day.year, day.month, day.day, hour, minute)
aware = naive.replace(tzinfo=tz)
if aware.astimezone(ZoneInfo("UTC")).astimezone(tz).replace(tzinfo=None) != naive:
return None
return aware
def next_runs(expr: str, after: datetime, tz: ZoneInfo, count: int = 5) -> list[datetime]:
"""The next *count* times *expr* fires after *after*, in *tz*.
``@reboot`` and a day that never comes (31 February) give none.
:raises ValueError: for an expression :func:`parse_cron` refuses.
"""
if is_reboot(expr):
return []
s = parse_cron(expr)
start = after.astimezone(tz)
runs: list[datetime] = []
for offset in range(_HORIZON_DAYS):
day = start.date() + timedelta(days=offset)
if not _day_matches(s, day):
continue
for hour in sorted(s.hours):
for minute in sorted(s.minutes):
at = _local(day, hour, minute, tz)
if at is None or at <= after:
continue
runs.append(at)
if len(runs) == count:
return runs
return runs
def _values(values: frozenset[int], low: int, high: int) -> str:
if values == frozenset(range(low, high + 1)):
return "*"
return ",".join(f"{v:02d}" for v in sorted(values))
def to_on_calendar(expr: str) -> list[str]:
"""``OnCalendar`` lines that fire exactly when *expr* does.
:raises ValueError: for an expression :func:`parse_cron` refuses, ``@reboot``
among them -- a timer takes that as ``OnBootSec``.
"""
s = parse_cron(expr)
time = f"{_values(s.hours, 0, 23)}:{_values(s.minutes, 0, 59)}:00"
months = _values(s.months, 1, 12)
days = _values(s.days, 1, 31)
weekdays = ",".join(_SYSTEMD_WEEKDAYS[d] for d in (1, 2, 3, 4, 5, 6, 0) if d in s.weekdays)
every_weekday = s.weekdays == frozenset(range(7))
if s.either_day:
return [f"*-{months}-{days} {time}", f"{weekdays} *-{months}-* {time}"]
prefix = "" if every_weekday else f"{weekdays} "
return [f"{prefix}*-{months}-{days} {time}"]
+30 -8
View File
@@ -43,7 +43,9 @@ exits quietly every day (NetOrk/netork#738). While dpkg, apt, aptitude or
unattended-upgrade is running, packages are halfway on purpose -- also between
two of apt's dpkg calls -- so the answer is then ``None``, as on a host without
dpkg. unattended-upgrade is told from Ubuntu's idle shutdown helper by its
command line (:data:`BUSY_CHECK`).
command line (:data:`BUSY_CHECK`). On a host without dpkg, dnf or yum, apk and opkg
are read the same way (:mod:`napalm_device_types.package_managers`,
NetOrk/netork#851), and ``package_manager`` names the one the answer is about.
**Filesystems** are how full ``/``, ``/boot`` and ``/var`` are, from POSIX
``df -Pk``: where an upgrade unpacks a kernel, builds its initrd and keeps the
@@ -65,6 +67,11 @@ from napalm_device_types.models import (
HostStatusDict,
InterruptedPackageDict,
)
from napalm_device_types.package_managers import (
PACKAGE_MANAGERS_COMMAND,
PackageManagerReading,
parse_package_managers,
)
from napalm_device_types.terminal import strip_terminal_codes
_BEGIN = "HSTAT_BEGIN"
@@ -117,6 +124,7 @@ HOST_STATUS_COMMAND = (
f"echo '[dpkg-journal]'; ls -1 {_DPKG_JOURNAL} 2>/dev/null | head -n 20; "
"echo '[dpkg-audit]'; dpkg-query -W -f='${db:Status-Abbrev} ${Package}\\n' 2>/dev/null "
"| awk 'substr($0, 2, 1) ~ /[HUFWt]/ || substr($0, 3, 1) == \"R\"' | head -n 50; fi; "
f"{PACKAGE_MANAGERS_COMMAND}"
f"echo '[filesystems]'; df -Pk {_FILESYSTEMS}; "
f"echo '[timers]'; for u in {_APT_TIMER} {' '.join(_DNF_TIMERS)}; do "
'printf \'%s %s\\n\' "$u" "$(systemctl is-enabled "$u" 2>/dev/null)"; done; '
@@ -287,12 +295,24 @@ def _halfway(line: str) -> InterruptedPackageDict | None:
return {"name": name, "state": state} if name and state else None
def _dpkg(sections: dict[str, list[str]]) -> tuple[bool | None, list[InterruptedPackageDict]]:
if "dpkg-audit" not in sections or "dpkg-busy" in sections:
return None, []
def _dpkg(sections: dict[str, list[str]]) -> PackageManagerReading | None:
if "dpkg-audit" not in sections:
return None
if "dpkg-busy" in sections:
return PackageManagerReading("dpkg", None, [])
packages = [p for p in map(_halfway, sections["dpkg-audit"]) if p is not None]
journal = any(_JOURNAL_ENTRY.fullmatch(f) for f in sections.get("dpkg-journal") or [])
return bool(packages) or journal, packages
return PackageManagerReading("dpkg", bool(packages) or journal, packages)
def _package_manager(sections: dict[str, list[str]]) -> PackageManagerReading:
"""The reading that says most: one left halfway, then one that is not, then a busy one."""
readings = [r for r in (_dpkg(sections), *parse_package_managers(sections)) if r is not None]
for interrupted in (True, False):
found = next((r for r in readings if r.interrupted is interrupted), None)
if found is not None:
return found
return readings[0] if readings else PackageManagerReading(None, None, [])
def _filesystems(sections: dict[str, list[str]]) -> list[FilesystemUsageDict] | None:
@@ -318,15 +338,16 @@ def parse_host_status(output: str) -> HostStatusDict:
"""
sections = _sections(output)
required, reason = _reboot(sections)
interrupted, packages = _dpkg(sections)
manager = _package_manager(sections)
return {
"reboot_required": required,
"reboot_reason": reason,
"auto_updates": _auto_updates(sections),
"auto_updates_last_success": _last_success(sections),
"auto_updates_interval": _interval(sections),
"package_manager_interrupted": interrupted,
"interrupted_packages": packages,
"package_manager": manager.manager,
"package_manager_interrupted": manager.interrupted,
"interrupted_packages": manager.packages,
"filesystems": _filesystems(sections),
}
@@ -356,6 +377,7 @@ class HostStatusMixin:
* auto_updates (bool or None)
* auto_updates_last_success (Unix time or None)
* auto_updates_interval (seconds or None)
* package_manager (dpkg, dnf, yum, apk, opkg, or None)
* package_manager_interrupted (bool or None)
* interrupted_packages (list of {name, state})
* filesystems (list of {mountpoint, total, used, available} in bytes, or None)
+182 -18
View File
@@ -5,6 +5,14 @@ with their public key and, where they are to administer the host, may use
sudo with their own password. When they leave, the account is locked, not
deleted: their key goes, sudo goes, the login is refused, and the home stays.
**What the person changes follows them** (NetOrk/netork#2013). A key they
replace goes from the account as the new one comes in (``replaces``); a key
they delete goes without a new one (``remove_local_account_keys``). Either way
only the keys named go: one the person added on the host stays, and the
account stays open. A password they remove goes from the account too
(``clear_password``): it logs in with a key only and has no sudo, as the sudo
rule asks for that password.
:class:`LocalAccountsMixin` holds what is the same on every POSIX host -- the
checks, reading an account back, the order of the steps, the key file and the
sudo rule. A driver supplies the commands that differ between operating
@@ -48,7 +56,7 @@ from typing import TYPE_CHECKING, ClassVar
from typing_extensions import TypedDict
from napalm_device_types.fixes import run_fix_steps
from napalm_device_types.fixes import FixStep, run_fix_steps
from napalm_device_types.models import FixResultDict, FixStepDict, FixStepOutcomeDict
if TYPE_CHECKING: # pragma: no cover - for type checkers only
@@ -179,6 +187,18 @@ def check_public_key(key: str) -> None:
raise ValueError("not a public key: expected '<type> <base64> [comment]' on one line")
def _checked_keys(keys: Sequence[str]) -> list[str]:
"""*keys*, each one stripped.
:raises ValueError: when *keys* is one string, or one of them no public key.
"""
if isinstance(keys, str):
raise ValueError("expected a list of public keys, not one string")
for key in keys:
check_public_key(key)
return [key.strip() for key in keys]
def _command(argv: Sequence[str]) -> str:
return shlex.join(argv)
@@ -225,9 +245,11 @@ class LocalAccountsMixin:
# -- what a driver supplies ------------------------------------------------------
#
# ``_account_*`` are the driver's: the five below it must supply, the key
# store (``_account_key_files``, ``_account_read_keys``, ``_account_add_key``,
# ``_account_remove_keys``) it may replace. ``_accounts_*`` are this mixin's own.
# ``_account_*`` are the driver's: the five below it must supply, the two after
# them (``_account_create_with_uid``, ``_account_clear_password``) it may, and
# the key store (``_account_key_files``, ``_account_read_keys``,
# ``_account_add_key``, ``_account_drop_keys``, ``_account_remove_keys``) it may
# replace. ``_accounts_*`` are this mixin's own.
def _account_create(self, name: str, password_hash: str | None) -> list[list[str]]:
"""The commands that create *name* with a home and :attr:`ACCOUNT_SHELL`.
@@ -265,6 +287,16 @@ class LocalAccountsMixin:
"""
raise NotImplementedError
def _account_clear_password(self, name: str) -> list[list[str]]:
"""The commands that take *name*'s password away; its key login stays as it is.
Not an empty password -- PAM's ``nullok`` lets anyone in without one --
and not a lock or an expiry, which sshd or PAM apply to a key too.
Only ``add_local_account(clear_password=True)`` and
``remove_local_account_keys(clear_password=True)`` ask for it.
"""
raise NotImplementedError
# -- reading ---------------------------------------------------------------------
def get_local_accounts(self, names: Iterable[str]) -> list[LocalAccountDict]:
@@ -339,6 +371,8 @@ class LocalAccountsMixin:
*,
password_hash: str | None = None,
admin: bool = False,
clear_password: bool = False,
replaces: Sequence[str] = (),
) -> LocalAccountChangeDict:
"""Give *name* an account that logs in with *public_key*, or open it again.
@@ -347,30 +381,82 @@ class LocalAccountsMixin:
:attr:`ACCOUNT_PASSWORD_SCHEME`) becomes its password; *admin* gives it
the sudo rule, which asks for that password -- so an administrator
needs one. Without *admin*, the rule and :attr:`ACCOUNT_ADMIN_GROUPS`
are taken away.
are taken away. Without *password_hash* the account keeps the password
it has, unless *clear_password* takes it away: the account then logs
in with a key only, and can have no sudo.
Steps: ``account``, then ``sudo`` and ``keys``, each of which runs when
the other failed.
*replaces* are keys *public_key* takes the place of -- the person's
old ones. They go from the account, matched by their base64 whatever
a line's options or comment; every other key stays.
:raises ValueError: for a name, key or hash no host takes, the login
user's own account, or a system account.
Steps: ``account``, then ``sudo``, ``password`` (with
*clear_password*), ``keys`` and ``old keys`` (with *replaces*), each of
which runs when another failed: an old key goes even when the new one
could not be added.
:raises ValueError: for a name, key or hash no host takes, a replaced
key that is none, *clear_password* with a *password_hash* or with
*admin*, the login user's own account, or a system account.
"""
check_account_name(name)
check_public_key(public_key)
if password_hash is not None:
self._accounts_check_hash(password_hash)
new = _key_blob(public_key)
old = [key for key in _checked_keys(replaces) if _key_blob(key) != new]
self._accounts_check_password(password_hash, admin=admin, clear=clear_password)
existing = self._accounts_changeable(name)
first = run_fix_steps(
[("account", lambda: self._accounts_open_or_create(name, existing, password_hash))]
)
parts = [first]
if first["success"]:
parts += [
run_fix_steps([("sudo", lambda: self._accounts_set_sudo(name, admin=admin))]),
run_fix_steps([("keys", lambda: self._accounts_add_key(name, public_key))]),
]
steps = self._accounts_after_open(name, public_key, admin, clear_password, old)
parts += [run_fix_steps([step]) for step in steps]
return self._accounts_result(name, parts)
def _accounts_after_open(
self, name: str, public_key: str, admin: bool, clear_password: bool, old: list[str]
) -> list[FixStep]:
"""The steps of ``add_local_account`` once the account is there, each on its own."""
steps: list[FixStep] = [("sudo", lambda: self._accounts_set_sudo(name, admin=admin))]
if clear_password:
steps.append(("password", lambda: self._accounts_clear_password(name)))
steps.append(("keys", lambda: self._accounts_add_key(name, public_key)))
if old:
steps.append(("old keys", lambda: self._accounts_drop_keys(name, old)))
return steps
def remove_local_account_keys(
self, name: str, keys: Sequence[str], *, clear_password: bool = False
) -> LocalAccountChangeDict:
"""Take *keys* off *name*'s account, for a person who deleted them.
Only those keys go, matched by their base64 whatever a line's options
or comment; the other keys stay, and the account stays open -- to get
a new key once there is one. A locked account has none left to
remove: the lock took them all. Without *clear_password* the password
and sudo stay too; with it the password goes, and so do the sudo rule
and :attr:`ACCOUNT_ADMIN_GROUPS`, as sudo asks for that password -- for
a person whose password went with their key.
Steps: ``keys``, then ``password`` and ``sudo`` (with *clear_password*),
each of which runs when another failed.
:raises ValueError: for a name no host takes, no keys or one that is
none, the login user's own account, or a system account.
"""
check_account_name(name)
wanted = _checked_keys(keys)
if not wanted:
raise ValueError("no public key to remove")
if self._accounts_changeable(name) is None:
step: FixStepDict = {"step": "keys", "ok": False, "output": f"no account {name}"}
return {**_joined([step]), "account": None}
steps: list[FixStep] = [("keys", lambda: self._accounts_drop_keys(name, wanted))]
if clear_password:
steps.append(("password", lambda: self._accounts_clear_password(name)))
steps.append(("sudo", lambda: self._accounts_set_sudo(name, admin=False)))
return self._accounts_result(name, [run_fix_steps([step]) for step in steps])
def lock_local_account(self, name: str) -> LocalAccountChangeDict:
"""Lock *name*: its keys go, every login is refused, sudo goes; its home stays.
@@ -490,6 +576,20 @@ class LocalAccountsMixin:
raise RuntimeError(f"cannot ask who has uid {uid}: {_said(result)}")
return result.stdout.split(":", 1)[0].strip() or None
def _accounts_check_password(
self, password_hash: str | None, *, admin: bool, clear: bool
) -> None:
""":raises ValueError: for a hash the host does not take, or a password to
clear that is to be set, or that sudo would ask for."""
if clear and password_hash is not None:
raise ValueError(
"a password to clear cannot be set too: clear_password or password_hash"
)
if clear and admin:
raise ValueError("an account without a password cannot use sudo: its rule asks for it")
if password_hash is not None:
self._accounts_check_hash(password_hash)
def _accounts_check_hash(self, password_hash: str) -> None:
prefixes = PASSWORD_SCHEMES[self.ACCOUNT_PASSWORD_SCHEME]
if not _CRYPT_RE.fullmatch(password_hash) or not password_hash.startswith(prefixes):
@@ -550,6 +650,16 @@ class LocalAccountsMixin:
return {"ok": False, "output": _said(failed)}
return {"ok": True, "output": done + (", password set" if password_hash else "")}
def _accounts_clear_password(self, name: str) -> FixStepOutcomeDict:
try:
commands = self._account_clear_password(name)
except NotImplementedError:
return {"ok": False, "output": "this driver cannot remove a password"}
failed = self._accounts_run(commands)
if failed is not None:
return {"ok": False, "output": _said(failed)}
return {"ok": True, "output": f"{name} has no password: it logs in with a key only"}
def _accounts_lock(self, name: str) -> FixStepOutcomeDict:
passwd = self._accounts_passwd(name)
if passwd is None:
@@ -683,9 +793,23 @@ class LocalAccountsMixin:
mine = self._accounts_key_file(name, path)
if mine is None:
return {"ok": False, "output": f"cannot read {path}"}
staged = self._accounts_stage("".join(f"{line}\n" for line in [*mine, public_key.strip()]))
failed = self._accounts_write_key_file(name, home, path, [*mine, public_key.strip()])
if failed is not None:
return {"ok": False, "output": f"{path}: {_said(failed)}"}
return {"ok": True, "output": f"key added to {path}"}
def _accounts_write_key_file(
self, name: str, home: str, path: str, lines: list[str]
) -> CommandResult | None:
"""*lines* as *name*'s key file *path*, written as *name*; None, or what failed.
Without lines the file goes: nothing is staged from an empty stdin.
"""
if not lines:
return self._accounts_run_as(name, [["rm", "-f", "--", path]])
staged = self._accounts_stage("".join(f"{line}\n" for line in lines))
try:
failed = self._accounts_run_as(
return self._accounts_run_as(
name,
[
["mkdir", "-p", "-m", "700", "--", f"{home}/.ssh"],
@@ -694,9 +818,49 @@ class LocalAccountsMixin:
)
finally:
self._accounts_unstage(staged)
def _accounts_drop_from(
self, name: str, home: str, path: str, blobs: set[str | None]
) -> FixStepOutcomeDict:
"""The keys of *blobs* out of *name*'s key file *path*; no output when none was in it."""
lines = self._accounts_key_file(name, path)
if lines is None:
return {"ok": False, "output": f"cannot read {path}"}
kept = [line for line in lines if _key_blob(line) not in blobs]
if len(kept) == len(lines):
return {"ok": True, "output": ""}
failed = self._accounts_write_key_file(name, home, path, kept)
if failed is not None:
return {"ok": False, "output": f"{path}: {_said(failed)}"}
return {"ok": True, "output": f"key added to {path}"}
return {"ok": True, "output": f"{len(lines) - len(kept)} removed from {path}"}
def _accounts_drop_keys(self, name: str, keys: list[str]) -> FixStepOutcomeDict:
home = self._accounts_home(name)
if home is None:
return {"ok": False, "output": f"no account {name}"}
return self._account_drop_keys(name, home, keys)
def _account_drop_keys(self, name: str, home: str, keys: list[str]) -> FixStepOutcomeDict:
"""Remove exactly *keys* from *name*'s key files in *home*, as *name*.
A key is matched by its base64, whatever a line's options or comment;
every other line stays. Afterwards none of *keys* may be left in
:meth:`_account_key_files`. A driver whose host keeps keys elsewhere
overrides this, with :meth:`_account_add_key`.
"""
blobs = {_key_blob(key) for key in keys}
said: list[str] = []
for path in authorized_keys_paths(home):
outcome = self._accounts_drop_from(name, home, path, blobs)
if not outcome["ok"]:
return outcome
said += [outcome["output"]] if outcome["output"] else []
left = self._account_read_keys(name, home)
if left is None:
return {"ok": False, "output": f"cannot read the key files of {name}"}
if any(_key_blob(line) in blobs for line in left):
return {"ok": False, "output": f"{name} still has a key that was to go"}
return {"ok": True, "output": "\n".join(said) or f"none of the keys is in {home}/.ssh"}
def _accounts_key_file(self, name: str, path: str) -> list[str] | None:
there = self.run_command(_command(["test", "-e", path]), privileged=True, timeout=30)
+318
View File
@@ -0,0 +1,318 @@
"""Where a host's logs go beyond it: what rsyslog forwards, which files it reads
in, and whether it gets the journal (NetOrk/netork#2053).
A log source (:mod:`napalm_device_types.log_sources`) says who writes a log;
this says who reads it on. It is read in the same command, so a reading of
the logs a host writes also says which of them leave it.
**rsyslog's configuration, read as text.** Both of its syntaxes are read:
the legacy one (``authpriv.* @@logs:514``, ``& @host``, ``:programname, isequal,
"sshd" @host``, ``$InputFileName``) and RainerScript (``action(type="omfwd" …)``,
``input(type="imfile" …)``, ``ruleset(…) { … }``, ``if … then``). A rule is
*complete* when it takes every message of its ruleset -- ``*.*`` with no
condition around it. Anything narrower is partial, and keeps its selector or
condition for the reader to see: which facility a unit logs to is not
computed (decided in #2053).
**The journal reaches rsyslog** through ``imjournal``, or through ``imuxsock``
when journald forwards to syslog (``ForwardToSyslog``). With no setting of its
own a host runs on its distribution's default, which ``journald.conf`` ships
commented out: ``#ForwardToSyslog=yes``.
"""
from __future__ import annotations
import re
from typing import TypedDict
class ForwardRuleDict(TypedDict):
"""One rule sending messages to another host."""
protocol: str # "udp", "tcp" or "relp"
target: str # the host, as configured
port: int | None # None: the protocol's default
selector: str # "*.*", "authpriv.*", or the condition around it
complete: bool # every message of its ruleset goes
ruleset: str | None # None: the default ruleset, which gets the journal and syslog
class FileInputDict(TypedDict):
"""A file rsyslog reads in (``imfile``), and the ruleset it hands it to."""
path: str # as configured: may hold a glob
ruleset: str | None
class RsyslogDict(TypedDict):
rules: list[ForwardRuleDict]
file_inputs: list[FileInputDict]
imjournal: bool
imuxsock: bool
class LogForwardingDict(TypedDict):
"""Where a host's logs go beyond it.
``journal`` is None when it cannot be told whether the journal reaches
rsyslog: ``imuxsock`` is loaded but ``journald.conf`` says nothing.
"""
rsyslog: bool # rsyslogd runs
journal: bool | None # the journal reaches rsyslog
rules: list[ForwardRuleDict]
file_inputs: list[FileInputDict]
#: Selectors that take every message.
_EVERYTHING = {"*.*", "*.debug"}
_PARAM_RE = re.compile(r'([A-Za-z][\w.]*)\s*=\s*"([^"]*)"')
_CALL_RE = re.compile(
r"^(?P<before>.*?)\b(?P<name>action|input|module|ruleset)\s*\((?P<args>.*)\)\s*;?$", re.S
)
_IF_RE = re.compile(r"^(?:else\s+)?if\s+(?P<cond>.*?)\s+then\b\s*(?P<rest>.*)$", re.S)
_PROPERTY_RE = re.compile(r'^(?P<sel>:[^,]+,\s*!?[\w-]+,\s*"(?:[^"\\]|\\.)*")\s+(?P<action>.+)$')
_LEGACY_RE = re.compile(r"^(?P<sel>\S+)\s+(?P<action>.+)$")
_FORWARD_RE = re.compile(
r"^(?:(?P<at>@@?)(?:\([^)]*\))?|:omrelp:)"
r"(?:\[(?P<v6>[^\]]+)\]|(?P<host>[^:;\s]+))(?::(?P<port>\d+))?"
)
_FORWARD_TYPES = {"omfwd", "omrelp"}
_TRUE = {"yes", "true", "1", "on"}
def _uncomment(text: str) -> str:
"""*text* without its comments: ``#`` to the end of the line, outside quotes."""
out: list[str] = []
quoted = commented = False
for char in text:
if char == "\n":
quoted = commented = False
elif commented:
continue
elif char == '"':
quoted = not quoted
elif char == "#" and not quoted:
commented = True
continue
out.append(char)
return "".join(out)
class _Pieces:
"""Cuts a configuration into statements, ``{`` and ``}``: a line runs on
while a bracket is open, and a brace outside one stands as a piece of its own."""
def __init__(self) -> None:
self.pieces: list[str] = []
self.current: list[str] = []
self.depth = 0
self.quoted = False
def close(self) -> None:
piece = "".join(self.current).strip()
if piece:
self.pieces.append(piece)
self.current.clear()
def bracket(self, char: str) -> None:
if char in "([":
self.depth += 1
elif char in ")]":
self.depth = max(self.depth - 1, 0)
def feed(self, char: str) -> None:
if char == '"':
self.quoted = not self.quoted
elif not self.quoted:
self.bracket(char)
if self.depth == 0 and char in "{}\n":
self.close()
if char != "\n":
self.pieces.append(char)
return
self.current.append(char)
def _pieces(text: str) -> list[str]:
"""Statements, ``{`` and ``}`` in order (:class:`_Pieces`)."""
cutter = _Pieces()
for char in text:
cutter.feed(char)
cutter.close()
return cutter.pieces
def _params(args: str) -> dict[str, str]:
return {key.lower(): value for key, value in _PARAM_RE.findall(args)}
def _port(value: str | None) -> int | None:
return int(value) if value and value.isdigit() else None
class _Reader:
"""Walks the pieces of a configuration, keeping the blocks it is in."""
def __init__(self) -> None:
self.parsed: RsyslogDict = {
"rules": [],
"file_inputs": [],
"imjournal": False,
"imuxsock": False,
}
self.stack: list[tuple[str, str | None]] = []
self.pending: tuple[str, str | None] | None = None
self.legacy_ruleset: str | None = None
self.last_selector = "*.*"
def ruleset(self) -> str | None:
names = [value for kind, value in self.stack if kind == "ruleset"]
return names[-1] if names else self.legacy_ruleset
def condition(self, once: str | None) -> str | None:
conditions = [value for kind, value in self.stack if kind == "if"]
return once or (conditions[-1] if conditions else None)
def forward(
self, protocol: str, target: str, port: int | None, selector: str, once: str | None
) -> None:
condition = self.condition(once)
self.parsed["rules"].append(
{
"protocol": protocol,
"target": target,
"port": port,
"selector": condition or selector,
"complete": condition is None and selector in _EVERYTHING,
"ruleset": self.ruleset(),
}
)
def piece(self, piece: str) -> None:
if piece == "{":
self.stack.append(self.pending or ("block", None))
self.pending = None
return
if piece == "}":
if self.stack:
self.stack.pop()
return
if piece == "else":
self.pending = ("if", "else")
return
# A condition whose action follows on its own: the action is under it.
once = self.pending[1] if self.pending and self.pending[0] == "if" else None
self.pending = None
match = _IF_RE.match(piece)
if match:
if match["rest"]:
self.statement(match["rest"], once=match["cond"])
else:
self.pending = ("if", match["cond"])
return
self.statement(piece, once=once)
def statement(self, text: str, once: str | None) -> None:
call = _CALL_RE.match(text)
if call:
self.call(call["name"], _params(call["args"]), call["before"].strip(), once)
elif text.startswith("$"):
self.directive(text)
else:
self.legacy(text, once)
def call(self, name: str, params: dict[str, str], before: str, once: str | None) -> None:
if name == "module":
self.module(params.get("load", ""))
elif name == "ruleset":
self.pending = ("ruleset", params.get("name"))
elif name == "input" and params.get("type") == "imfile" and params.get("file"):
self.parsed["file_inputs"].append(
{"path": params["file"], "ruleset": params.get("ruleset")}
)
elif name == "action":
self.action(params, before, once)
def action(self, params: dict[str, str], before: str, once: str | None) -> None:
selector = self.last_selector if before == "&" else (before or "*.*")
self.last_selector = selector
kind = params.get("type", "")
if kind in _FORWARD_TYPES and params.get("target"):
protocol = "relp" if kind == "omrelp" else params.get("protocol", "udp").lower()
self.forward(protocol, params["target"], _port(params.get("port")), selector, once)
def module(self, name: str) -> None:
if name == "imjournal":
self.parsed["imjournal"] = True
elif name == "imuxsock":
self.parsed["imuxsock"] = True
def directive(self, text: str) -> None:
name, _, value = text.partition(" ")
name, value = name.lower(), value.strip()
if name == "$modload":
self.module(value)
elif name == "$inputfilename" and value:
self.parsed["file_inputs"].append({"path": value, "ruleset": self.legacy_ruleset})
elif name == "$ruleset":
self.legacy_ruleset = None if value == "RSYSLOG_DefaultRuleset" else value
def legacy(self, text: str, once: str | None) -> None:
match = _PROPERTY_RE.match(text) or _LEGACY_RE.match(text)
if not match:
return
selector = self.last_selector if match["sel"] == "&" else match["sel"]
self.last_selector = selector
forward = _FORWARD_RE.match(match["action"].strip())
if forward:
protocol = {"@": "udp", "@@": "tcp"}.get(forward["at"] or "", "relp")
self.forward(
protocol, forward["v6"] or forward["host"], _port(forward["port"]), selector, once
)
def parse_rsyslog(text: str) -> RsyslogDict:
"""rsyslog's configuration -- ``rsyslog.conf`` and ``rsyslog.d`` as one text --
read for its forwarding rules, its file inputs, and the inputs that take
the journal."""
reader = _Reader()
for piece in _pieces(_uncomment(text)):
reader.piece(piece)
return reader.parsed
def journal_to_syslog(lines: list[str]) -> bool | None:
"""Whether journald forwards to syslog, from its ``ForwardToSyslog`` lines in
the order journald reads them: the last setting wins; with none, the
default the distribution ships commented out; None when nothing says."""
setting = shipped = None
for line in lines:
commented = line.lstrip().startswith("#")
_, _, value = line.lstrip("# \t").partition("=")
if commented:
shipped = value.strip().lower() in _TRUE
else:
setting = value.strip().lower() in _TRUE
return setting if setting is not None else shipped
def log_forwarding(
running: bool, rsyslog: RsyslogDict, journald_lines: list[str]
) -> LogForwardingDict:
"""Where a host's logs go: *rsyslog* is its configuration, read whether it runs or not."""
if not running:
return {"rsyslog": False, "journal": False, "rules": [], "file_inputs": []}
journal: bool | None
if rsyslog["imjournal"]:
journal = True
elif rsyslog["imuxsock"]:
journal = journal_to_syslog(journald_lines)
else:
journal = False
return {
"rsyslog": True,
"journal": journal,
"rules": rsyslog["rules"],
"file_inputs": rsyslog["file_inputs"],
}
+365
View File
@@ -0,0 +1,365 @@
"""Log sources: which logs a host writes, and who writes each of them.
Before anything can watch a host's logs -- a person, CrowdSec, a syslog
forwarder -- it has to know which logs there are. That is read the same way on
every Linux host, so the command and its parse live here once, and a driver
only carries the command across (NetOrk/netork#2049).
**Three kinds.** journald names every unit it holds entries for
(``journalctl -F _SYSTEMD_UNIT``); only services count, not login sessions.
``/proc/<pid>/fd`` shows the files running processes hold open, a log being a
``*.log`` / ``*.log.N`` file or anything under ``/var/log``; ``stat -L`` on the
descriptor gives its size and last write, also for a file inside a container,
whose path is the container's own. The engine's own file of a container's
output (``/var/lib/docker/containers/<id>/…``) is that container's stream, not
a file of ``dockerd``.
**Who writes it.** Only a process holding the file open for writing writes
it: the access mode in ``/proc/<pid>/fdinfo/<fd>``'s ``flags`` is ``O_WRONLY`` or
``O_RDWR``. A reader -- CrowdSec, Promtail, rsyslog's ``imfile``, a ``tail -F``
-- holds it too, and is no producer (NetOrk/netork#2092). ``/proc/<pid>/cgroup``
says which systemd service or container the first writer runs in, as for
listening sockets (:mod:`napalm_device_types.listening`). A file only readers
hold is listed without a producer; one only readers in a container hold is left
out, its path being the reader's view of a mount. journald's own files, files
already deleted and a database's write-ahead log (LevelDB's ``000004.log``) are
left out.
**Where they go** is read in the same round trip
(:mod:`napalm_device_types.log_forwarding`, NetOrk/netork#2053): whether
rsyslog runs, its configuration -- each line behind ``| ``, so that none reads
as a section -- and journald's ``ForwardToSyslog`` lines in the order journald
reads them.
**One round trip, root, and without it.** As for listening sockets the script
goes as one ``sh -c`` argument. Without root only the login user's processes
show their files; the command then runs again unprivileged, and the reading
says it is not ``attributed``. A report whose end is missing raises: a list cut
short must never read as logs that went away, and so must a host whose
``find`` cannot list open files (busybox): it raises
:class:`LogSourcesUnavailable`.
"""
from __future__ import annotations
import re
from datetime import UTC, datetime
from shlex import quote
from typing import TYPE_CHECKING, TypedDict
from napalm_device_types.listening import _cgroup_paths, _container, _sections, _unit
from napalm_device_types.log_forwarding import LogForwardingDict, log_forwarding, parse_rsyslog
from napalm_device_types.terminal import strip_terminal_codes
class LogSourceDict(TypedDict):
"""A log the host writes: a unit's journal, a file a process holds open, or a
container's stream as the engine keeps it.
``path`` is as its writer sees it: a file a container writes is named inside
that container. Whether anything reads the log is the consumer's question.
"""
kind: str # "journald", "file" or "container"
path: str | None # the file; None for a journal
unit: str | None # the journal's unit, or the file writer's service; without ".service"
container_id: str | None # the full container ID: the writer's, or the stream's
process: str | None # the first process writing the file; None when only readers hold it
size: int | None # bytes; None for a journal, or when the file could not be read
modified: str | None # the last write, ISO 8601 UTC; None as for size
class LogSourcesDict(TypedDict):
"""What ``LogSourcesMixin.get_log_sources`` read.
``attributed`` is False when the reading ran without root: only the login
user's own processes show their files then, so a missing log means "not
told", not "not there". ``journald`` is False on a host without systemd's
journal, where no unit has one.
"""
attributed: bool
journald: bool
sources: list[LogSourceDict]
forwarding: LogForwardingDict | None # where the logs go beyond the host
_BEGIN = "LOGS_BEGIN"
_END = "LOGS_END"
#: One line, POSIX ``sh`` with GNU ``find`` (``-lname``, ``-printf``), read-only.
#: The frame markers are printed in two halves so that a transport which echoes
#: the command does not show them early. ``-printf '%p %l'`` puts the descriptor
#: before its target, so a target with a space in it stays whole.
LOG_SOURCES_COMMAND = (
"PATH=$PATH:/usr/sbin:/sbin; "
"printf '%s%s\\n' LOGS_ BEGIN; "
"if [ -d /run/systemd/system ] && command -v journalctl >/dev/null 2>&1; then "
"j=journalctl; command -v timeout >/dev/null 2>&1 && j='timeout 30 journalctl'; "
"echo '[journald]'; $j -F _SYSTEMD_UNIT 2>/dev/null; fi; "
"if find / -maxdepth 0 -lname x -printf '' >/dev/null 2>&1; then "
"o=$(find /proc/[0-9]*/fd -mindepth 1 -maxdepth 1 -type l "
"\\( -lname '*.log' -o -lname '*.log.[0-9]*' -o -lname '/var/log/*' \\) "
"-printf '%p %l\\n' 2>/dev/null); "
"echo '[open]'; printf '%s\\n' \"$o\"; "
"echo '[stat]'; printf '%s\\n' \"$o\" | cut -d' ' -f1 "
"| xargs -r stat -L -c '%n %s %Y' 2>/dev/null; "
"echo '[flags]'; printf '%s\\n' \"$o\" | cut -d' ' -f1 | sed 's|/fd/|/fdinfo/|' "
"| xargs -r grep -H '^flags:' 2>/dev/null; "
"pids=$(printf '%s\\n' \"$o\" | cut -d/ -f3 | sort -un); "
"echo '[cgroups]'; for p in $pids; do sed \"s|^|$p |\" /proc/$p/cgroup 2>/dev/null; done; "
"echo '[comm]'; for p in $pids; do c=$(cat /proc/$p/comm 2>/dev/null) && echo \"$p $c\"; done; "
"else echo '[no-find]'; fi; "
"echo '[rsyslog]'; { pidof rsyslogd || pgrep -x rsyslogd; } >/dev/null 2>&1 && echo running; "
"echo '[rsyslog-conf]'; cat /etc/rsyslog.conf /etc/rsyslog.d/*.conf 2>/dev/null "
"| sed 's/^/| /'; "
"echo '[journald-conf]'; cat /etc/systemd/journald.conf "
"/usr/lib/systemd/journald.conf.d/*.conf /run/systemd/journald.conf.d/*.conf "
"/etc/systemd/journald.conf.d/*.conf 2>/dev/null "
"| grep -i '^[#[:space:]]*ForwardToSyslog'; "
"printf '%s%s\\n' LOGS_ END"
)
#: ``/proc/900/fd/5`` -- the process and its descriptor.
_FD_RE = re.compile(r"^/proc/(\d+)/fd/\d+$")
#: ``/proc/900/fdinfo/5:flags:<tab>0102001`` -- the descriptor's open flags, octal.
_FLAGS_RE = re.compile(r"^/proc/(\d+)/fdinfo/(\d+):flags:\s*([0-7]+)$")
#: The access mode bits of the open flags: 0 is O_RDONLY, 1 O_WRONLY, 2 O_RDWR.
_ACCESS_MODE = 0o3
#: The engine's file of a container's output: json-file and local drivers.
_STREAM_RE = re.compile(r"^/var/lib/docker/containers/([0-9a-f]{64})/")
#: Files that are no log to read: journald's store, a file already deleted, and
#: the write-ahead log LevelDB and RocksDB name ``000004.log``.
_NOT_A_SOURCE = re.compile(r"^/var/log/journal/|\.journal~?$| \(deleted\)$|/\d{6}\.log$")
_SERVICE_SUFFIX = ".service"
_NO_FIND = "no-find"
class LogSourcesUnavailable(NotImplementedError):
"""The host's ``find`` cannot list open files (busybox); nothing to read, nothing to retry."""
def _frame(output: str) -> list[str]:
lines = [line.strip() for line in strip_terminal_codes(output).splitlines()]
try:
start = lines.index(_BEGIN)
end = lines.index(_END, start)
except ValueError:
raise ValueError("no intact log source report in the output") from None
return [line for line in lines[start + 1 : end] if line]
def _stats(lines: list[str]) -> dict[str, tuple[int, str]]:
"""Each descriptor's size and last write: ``/proc/900/fd/5 2048 1762250200``."""
stats: dict[str, tuple[int, str]] = {}
for line in lines:
fd, _, rest = line.partition(" ")
size, _, mtime = rest.partition(" ")
if size.isdigit() and mtime.isdigit():
stats[fd] = (int(size), datetime.fromtimestamp(int(mtime), UTC).isoformat())
return stats
def _names(lines: list[str]) -> dict[int, str]:
names: dict[int, str] = {}
for line in lines:
pid, _, name = line.partition(" ")
if pid.isdigit() and name:
names[int(pid)] = name
return names
def _journals(units: list[str]) -> list[LogSourceDict]:
services = {u[: -len(_SERVICE_SUFFIX)] for u in units if u.endswith(_SERVICE_SUFFIX)}
return [
{
"kind": "journald",
"path": None,
"unit": unit,
"container_id": None,
"process": None,
"size": None,
"modified": None,
}
for unit in sorted(services)
]
def _held(lines: list[str]) -> list[tuple[int, str, str]]:
"""(pid, descriptor, target) for every log file a process holds, by pid."""
held = []
for line in lines:
fd, _, target = line.partition(" ")
match = _FD_RE.match(fd)
if match and target.startswith("/") and not _NOT_A_SOURCE.search(target):
held.append((int(match.group(1)), fd, target))
return sorted(held)
def _writes(lines: list[str]) -> dict[str, bool]:
"""Whether each descriptor is open for writing, by its ``/proc/<pid>/fd/<fd>``."""
writes: dict[str, bool] = {}
for line in lines:
match = _FLAGS_RE.match(line)
if match:
pid, fd, flags = match.groups()
writes[f"/proc/{pid}/fd/{fd}"] = int(flags, 8) & _ACCESS_MODE != 0
return writes
_Key = tuple[str, str | None, str]
def _holders(lines: list[str], paths: dict[int, str]) -> dict[_Key, list[tuple[int, str]]]:
"""The (pid, descriptor) pairs holding each log, by kind, container and path."""
holders: dict[_Key, list[tuple[int, str]]] = {}
for pid, fd, target in _held(lines):
stream = _STREAM_RE.match(target)
kind, container = (
("container", stream.group(1)) if stream else ("file", _container(paths.get(pid)))
)
holders.setdefault((kind, container, target), []).append((pid, fd))
return holders
def _writer(held: list[tuple[int, str]], writes: dict[str, bool] | None) -> int | None:
"""The first process writing; the first holding for a report without flags,
as an older command printed it."""
if writes is None:
return held[0][0]
return next((pid for pid, fd in held if writes.get(fd)), None)
def _producer(
kind: str, writer: int | None, paths: dict[int, str], names: dict[int, str]
) -> tuple[str | None, str | None]:
"""The writer's service and name; a stream's service is its container."""
if writer is None:
return None, None
return (_unit(paths.get(writer)) if kind == "file" else None), names.get(writer)
def _files(sections: dict[str, list[str]]) -> list[LogSourceDict]:
stats = _stats(sections.get("stat", []))
paths = _cgroup_paths(sections.get("cgroups", []))
names = _names(sections.get("comm", []))
writes = _writes(sections["flags"]) if "flags" in sections else None
found: list[LogSourceDict] = []
for (kind, container, target), held in _holders(sections.get("open", []), paths).items():
writer = _writer(held, writes)
if writer is None and kind == "file" and container is not None:
continue # a reader's view of a mount, inside its container
unit, process = _producer(kind, writer, paths, names)
size, modified = next((stats[fd] for _, fd in held if fd in stats), (None, None))
found.append(
{
"kind": kind,
"path": target,
"unit": unit,
"container_id": container,
"process": process,
"size": size,
"modified": modified,
}
)
return found
def _order(source: LogSourceDict) -> tuple[str, str, str]:
return (source["kind"], source["container_id"] or "", source["path"] or source["unit"] or "")
def parse_log_sources(output: str) -> tuple[bool, list[LogSourceDict]]:
"""Parse what :data:`LOG_SOURCES_COMMAND` printed: whether the host has a
journal, and every source, sorted by kind, container and name.
:raises LogSourcesUnavailable: when the host has no GNU ``find``.
:raises ValueError: when the output carries no intact report.
"""
sections = _sections(_frame(output))
if _NO_FIND in sections:
raise LogSourcesUnavailable("the host's find cannot list open files (no GNU find)")
journald = "journald" in sections
sources = _journals(sections.get("journald", [])) + _files(sections)
return journald, sorted(sources, key=_order)
def _reading(output: str, *, attributed: bool) -> LogSourcesDict:
journald, sources = parse_log_sources(output)
forwarding = parse_log_forwarding(output)
return {
"attributed": attributed,
"journald": journald,
"sources": sources,
"forwarding": forwarding,
}
def parse_log_forwarding(output: str) -> LogForwardingDict | None:
"""Parse where the logs go from what :data:`LOG_SOURCES_COMMAND` printed;
None for a report without the sections, as an older command printed it.
:raises ValueError: when the output carries no intact report.
"""
sections = _sections(_frame(output))
if "rsyslog" not in sections:
return None
config = "\n".join(
line[2:] if line.startswith("| ") else line[1:] for line in sections.get("rsyslog-conf", [])
)
running = "running" in sections["rsyslog"]
return log_forwarding(running, parse_rsyslog(config), sections.get("journald-conf", []))
class LogSourcesMixin:
"""Adds :meth:`get_log_sources` to a driver that can run a command on a Linux host.
The template form (README, "Function classes"): the command and its parse
are the same everywhere, so they are concrete here, and a driver supplies
only :meth:`_run_log_sources_command` -- how a command reaches its host, and
how it gains root there. Mixed in by the drivers that can, so
``hasattr(driver, "get_log_sources")`` stays truthful.
"""
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
def _run_log_sources_command(self, command: str, *, privileged: bool) -> str:
"""Run *command* on the host and return what it printed; as root
when *privileged*. The command is a single ``sh -c`` invocation."""
...
def get_log_sources(self) -> LogSourcesDict:
"""
Returns every log the host writes, with the service or container writing it.
* attributed (bool) - read as root, so every process's files are listed
* journald (bool) - the host keeps systemd's journal
* sources (list) - see :class:`~napalm_device_types.models.LogSourceDict`
* forwarding (dict) - where the logs go beyond the host, see
:class:`~napalm_device_types.log_forwarding.LogForwardingDict`
Example::
{
"attributed": True,
"journald": True,
"sources": [
{"kind": "journald", "path": None, "unit": "nginx",
"container_id": None, "process": None, "size": None, "modified": None},
{"kind": "file", "path": "/var/log/nginx/access.log", "unit": "nginx",
"container_id": None, "process": "nginx", "size": 2048,
"modified": "2026-11-04T09:56:40+00:00"},
],
}
:raises LogSourcesUnavailable: if the host has no GNU ``find``.
:raises ValueError: if neither reading carried an intact report.
"""
command = f"sh -c {quote(LOG_SOURCES_COMMAND)}"
try:
output = self._run_log_sources_command(command, privileged=True)
return _reading(output, attributed=True)
except ValueError:
pass
output = self._run_log_sources_command(command, privileged=False)
return _reading(output, attributed=False)
+5 -14
View File
@@ -85,8 +85,8 @@ class InterruptedPackageDict(TypedDict):
"""A package an interrupted package manager run left halfway."""
name: str
#: dpkg's word for it: "half-installed", "unpacked", "half-configured",
#: "triggers-awaited", "triggers-pending", or "reinstall-required".
#: The manager's word for it: dpkg's and opkg's ("unpacked", "half-configured", ...),
#: rpm's "duplicate", apk's "broken-files" and "broken-script" (``package_managers``).
state: str
@@ -111,9 +111,9 @@ class HostStatusDict(TypedDict):
reboot_reason: str | None
#: True when the host installs updates on its own (unattended-upgrades, dnf-automatic).
auto_updates: bool | None
#: True when a package manager run was cut off and has to be finished
#: (``dpkg --configure -a``) before anything else installs; None when it
#: cannot tell, or a package manager is running right now.
#: The package manager read (dpkg, dnf, yum, apk, opkg; None: none) and whether a run
#: of it was cut off, left halfway (None: cannot tell, or one is running right now).
package_manager: NotRequired[str | None]
package_manager_interrupted: NotRequired[bool | None]
#: The packages that run left halfway; empty when only its journal says so.
interrupted_packages: NotRequired[list[InterruptedPackageDict]]
@@ -1038,15 +1038,6 @@ class ProcessDict(TypedDict):
command: str # full command line
class CronJobDict(TypedDict):
"""A scheduled cron task."""
user: str # owner of the crontab entry
schedule: str # five-field cron expression, e.g. "0 * * * *"
command: str # shell command to execute
description: NotRequired[str] # optional inline comment
class ApplyUpdatesResultDict(TypedDict):
"""Result of an ``apply_updates()`` call."""
-29
View File
@@ -15,7 +15,6 @@ from typing import TYPE_CHECKING
from napalm_device_types.base import DeviceTypeDriver
from napalm_device_types.health_metrics import HealthMetricsMixin
from napalm_device_types.models import (
CronJobDict,
DeviceActionResultDict,
ProcessDict,
SNMPConfigDict,
@@ -131,34 +130,6 @@ class OSDriver(
"""
...
# ------------------------------------------------------------------
# Cron jobs
# ------------------------------------------------------------------
def get_cron_jobs(self) -> list[CronJobDict]:
"""
Returns scheduled cron tasks from all user crontabs and ``/etc/cron.d``.
Each entry contains:
* user (string) - owner of the crontab entry
* schedule (string) - five-field cron expression (e.g. ``"0 * * * *"``)
* command (string) - shell command to execute
* description (string, optional) - inline comment text if present
Example::
[
{
"user": "root",
"schedule": "0 4 * * *",
"command": "/usr/local/bin/backup.sh",
"description": "nightly backup",
},
]
"""
...
# ------------------------------------------------------------------
# SNMP
# ------------------------------------------------------------------
+194
View File
@@ -0,0 +1,194 @@
"""Whether dnf or yum, apk or opkg was left halfway: the host status's answer for
a host without dpkg (NetOrk/netork#851).
Read the same way as dpkg's state (:mod:`napalm_device_types.host_status`): in the
one host status command, without root, and changing nothing. Each manager is asked
only where it exists, and while one of its processes runs -- from
``/proc/<pid>/comm``, which every Linux has and BusyBox's ``ps`` cannot narrow to --
its packages are halfway on purpose and the answer is unknown.
**dnf and yum** (rpm). A transaction cut off after rpm installed the new version
of a package and before it erased the old one leaves both in the rpm database:
what ``dnf check --duplicates`` and ``package-cleanup --dupes`` report. Packages
dnf installs side by side on purpose do not count: its default ``installonlypkgs``
(``kernel``, ``installonlypkg(kernel)`` and the rest), plus any the host adds in
``dnf.conf``, asked of rpm by what they provide; and ``gpg-pubkey``, one entry
per imported key. Also interrupted is a transaction that never ended: dnf 4's
history (libdnf's ``trans`` table) holds its row with ``state`` 0 until
``finish()`` writes 1 (done) or 2 (error), so the newest row at 0 is one cut off;
yum leaves a ``transaction-all.*`` journal in ``/var/lib/yum``, which
``yum-complete-transaction`` reads (one renamed ``.disabled`` is given up). The
history is read with Python's sqlite3 module in read-only mode, as a stock EL host
has no ``sqlite3`` program and a ``dnf`` command loads every plugin and may reach
for its repositories. dnf 5's history lives elsewhere and is not read; its
duplicates are.
**apk** (Alpine, OpenWrt from 25.12). apk marks an installed package whose files
or scripts failed with ``f:`` in ``/lib/apk/db/installed`` -- ``f`` broken files,
``s`` a broken script; ``x`` (xattrs) and ``S`` (the checksum kind) are no fault --
and those are exactly the packages ``apk fix`` without arguments picks. A run
killed outright leaves no such mark; its next run usually repeats what it left.
**opkg** (OpenWrt before 25.12). ``/usr/lib/opkg/status`` holds
``Status: <want> <flags> <state>`` per package. A run cut off, or a postinst that
failed, leaves ``unpacked`` (or ``half-configured``, ``half-installed``), and the
``reinstreq`` flag asks for a reinstall; ``opkg configure`` finishes every unpacked
package.
What to do about it is not decided here.
"""
from __future__ import annotations
from typing import NamedTuple
from napalm_device_types.models import InterruptedPackageDict
#: dnf's default ``installonlypkgs``: names and what such packages provide.
_INSTALLONLY = (
"kernel",
"kernel-PAE",
"installonlypkg(kernel)",
"installonlypkg(kernel-module)",
"installonlypkg(vm)",
"multiversion(kernel)",
)
#: Where a host adds its own installonly packages, dnf's then yum's.
_DNF_CONF = "/etc/dnf/dnf.conf /etc/yum.conf"
DNF_HISTORY = "/var/lib/dnf/history.sqlite"
YUM_DIR = "/var/lib/yum"
APK_INSTALLED = "/lib/apk/db/installed"
OPKG_STATUS = "/usr/lib/opkg/status"
#: manager -> the process names that leave its packages halfway while they run.
_RUNNING = {
"rpm": frozenset({"rpm", "dnf", "dnf5", "yum", "dnf-automatic", "packagekitd"}),
"apk": frozenset({"apk"}),
"opkg": frozenset({"opkg"}),
}
_UNFINISHED = "0"
_YUM_JOURNAL = "transaction-all."
_YUM_GIVEN_UP = ".disabled"
_APK_FLAGS = (("f", "broken-files"), ("s", "broken-script"))
_OPKG_HALFWAY = frozenset({"unpacked", "half-configured", "half-installed"})
_OPKG_REINSTALL = "reinstreq"
#: The newest transaction in dnf's history as ``<id> <state>``; nothing when unreadable.
#: One literal, with :data:`DNF_HISTORY` spelt out: a fixed query, nothing put into it.
_DNF_HISTORY_PY = (
"import sqlite3; "
"c = sqlite3.connect('file:/var/lib/dnf/history.sqlite?mode=ro', uri=True); "
"r = c.execute('SELECT id, state FROM trans ORDER BY id DESC LIMIT 1').fetchone(); "
"r and print(*r)"
)
_RUNNING_PART = (
"echo '[pm-running]'; cat /proc/[0-9]*/comm 2>/dev/null | grep -xE '"
+ "|".join(sorted(set().union(*_RUNNING.values())))
+ "' | sort -u; "
)
_RPM_PART = (
"if command -v rpm >/dev/null 2>&1 && { command -v dnf || command -v yum; } >/dev/null 2>&1; "
"then echo '[rpm-frontend]'; { command -v dnf || command -v yum; } | sed 's#.*/##'; "
f"echo '[rpm-installonly]'; io=\"{' '.join(_INSTALLONLY)} "
f"$(sed -n 's/^[[:space:]]*installonlypkgs[[:space:]]*=//p' {_DNF_CONF} 2>/dev/null "
"| tr ',' ' ')\"; rpm -q --whatprovides $io --qf '%{NAME}\\n' 2>/dev/null "
"| grep -v ' ' | sort -u; "
"echo '[rpm-dupes]'; rpm -qa --qf '%{NAME} %{ARCH}\\n' 2>/dev/null "
"| grep -v '^gpg-pubkey ' | sort | uniq -d | head -n 50; "
f"echo '[yum-journal]'; ls -1 {YUM_DIR} 2>/dev/null | grep '^transaction-all\\.' "
"| head -n 20; "
"echo '[dnf-history]'; for py in /usr/libexec/platform-python python3; do "
f'if command -v $py >/dev/null 2>&1; then $py -c "{_DNF_HISTORY_PY}" 2>/dev/null; '
"break; fi; done; fi; "
)
_APK_PART = (
f"if command -v apk >/dev/null 2>&1 && [ -r {APK_INSTALLED} ]; then echo '[apk-audit]'; "
"awk '/^P:/ {p = substr($0, 3)} /^f:/ {f = substr($0, 3)} "
'/^$/ {if (f ~ /[fs]/) print f, p; p = ""; f = ""} END {if (f ~ /[fs]/) print f, p}\' '
f"{APK_INSTALLED} | head -n 50; fi; "
)
_OPKG_PART = (
f"if command -v opkg >/dev/null 2>&1 && [ -r {OPKG_STATUS} ]; then echo '[opkg-audit]'; "
"awk '/^Package:/ {p = $2} /^Status:/ && "
"($4 ~ /^(unpacked|half-configured|half-installed)$/ || $3 ~ /reinstreq/) "
f"{{print $3, $4, p}}' {OPKG_STATUS} | head -n 50; fi; "
)
#: The part of the host status command that reads them; POSIX ``sh``, BusyBox too.
PACKAGE_MANAGERS_COMMAND = _RUNNING_PART + _RPM_PART + _APK_PART + _OPKG_PART
class PackageManagerReading(NamedTuple):
"""One package manager's state: halfway or not, None while it is running."""
manager: str | None
interrupted: bool | None
packages: list[InterruptedPackageDict]
def _unfinished_transaction(sections: dict[str, list[str]]) -> bool:
newest = (sections.get("dnf-history") or [""])[0].split()
if len(newest) == 2 and newest[1] == _UNFINISHED:
return True
return any(
name.startswith(_YUM_JOURNAL) and not name.endswith(_YUM_GIVEN_UP)
for name in sections.get("yum-journal") or []
)
def _rpm(sections: dict[str, list[str]], running: set[str]) -> PackageManagerReading | None:
if "rpm-dupes" not in sections:
return None
manager = (sections.get("rpm-frontend") or ["dnf"])[0]
if running & _RUNNING["rpm"]:
return PackageManagerReading(manager, None, [])
installonly = set(sections.get("rpm-installonly") or [])
names = dict.fromkeys(line.split()[0] for line in sections["rpm-dupes"])
packages: list[InterruptedPackageDict] = [
{"name": name, "state": "duplicate"} for name in names if name not in installonly
]
return PackageManagerReading(
manager, bool(packages) or _unfinished_transaction(sections), packages
)
def _apk(sections: dict[str, list[str]], running: set[str]) -> PackageManagerReading | None:
if "apk-audit" not in sections:
return None
if running & _RUNNING["apk"]:
return PackageManagerReading("apk", None, [])
packages: list[InterruptedPackageDict] = []
for line in sections["apk-audit"]:
flags, _, name = line.partition(" ")
name = name.strip()
packages += [
{"name": name, "state": state} for flag, state in _APK_FLAGS if name and flag in flags
]
return PackageManagerReading("apk", bool(packages), packages)
def _opkg_package(line: str) -> InterruptedPackageDict | None:
parts = line.split()
if len(parts) != 3:
return None
flags, state, name = parts
if _OPKG_REINSTALL in flags.split(","):
return {"name": name, "state": "reinstall-required"}
return {"name": name, "state": state} if state in _OPKG_HALFWAY else None
def _opkg(sections: dict[str, list[str]], running: set[str]) -> PackageManagerReading | None:
if "opkg-audit" not in sections:
return None
if running & _RUNNING["opkg"]:
return PackageManagerReading("opkg", None, [])
packages = [p for p in map(_opkg_package, sections["opkg-audit"]) if p is not None]
return PackageManagerReading("opkg", bool(packages), packages)
def parse_package_managers(sections: dict[str, list[str]]) -> list[PackageManagerReading]:
"""What :data:`PACKAGE_MANAGERS_COMMAND` found, one reading per manager it asked."""
running = set(sections.get("pm-running") or [])
readings = (read(sections, running) for read in (_rpm, _apk, _opkg))
return [reading for reading in readings if reading is not None]
+69
View File
@@ -0,0 +1,69 @@
"""The backups running on a device right now.
A restart aborts every backup that runs on the device it restarts: a
hypervisor's backup of its guests, a backup server's backup, sync, verify or
garbage collection, a host's own file-level backup (restic, borg and the like).
A guest's restart breaks the backup its hypervisor is taking of it. So an
application that restarts a device asks first, right before the restart.
Declared under ``if TYPE_CHECKING``: a driver mixes this in when it can tell;
``hasattr`` tells a caller whether it did. No role base carries it, and a driver
without it cannot tell -- which is not the same as "nothing runs".
"""
from __future__ import annotations
from typing import TYPE_CHECKING, Literal, TypedDict, get_args
#: What a running backup does: the backup itself, or one of a backup server's
#: jobs (``napalm_device_types.backups.BackupJobKind``).
RunningBackupKind = Literal["backup", "sync", "verify", "gc", "prune", "tape"]
RUNNING_BACKUP_KINDS: tuple[str, ...] = get_args(RunningBackupKind)
class RunningBackupDict(TypedDict):
"""One backup running on the device (``get_running_backups``)."""
kind: RunningBackupKind
job: str | None # the job it runs for, as the platform names it; None: started by hand
task: str | None # this run's id on the platform: a task id, a unit, a process id
guests: list[str] # the guests (get_vms' ids) it is backing up now; [] for the host's own
since: int | None # unix seconds it started; None when the platform does not say
class RunningBackupsMixin:
if TYPE_CHECKING:
def get_running_backups(self) -> list[RunningBackupDict]:
"""
Returns the backups running on the device now; ``[]`` when none is.
Every entry runs on this device: restarting the device aborts it.
A hypervisor names in ``guests`` the guests a backup is taking right
now -- a guest that is locked for its backup counts, with or without
a task the platform still shows -- so the application can tell a
guest's own restart not to break it.
Example::
[
{
"kind": "backup",
"job": "backup-3f2a9c1e-77d1",
"task": "UPID:pve1:0001F2A3:00C4B1D2:6731A0C0:vzdump::root@pam:",
"guests": ["101"],
"since": 1793660400,
},
{
"kind": "gc",
"job": "store1",
"task": "UPID:localhost:000004D2:...:garbage_collection:store1:root@pam:",
"guests": [],
"since": 1793664000,
},
]
:raises Exception: when the device could not be read; never an
empty list for a failed read.
"""
...
+196
View File
@@ -0,0 +1,196 @@
"""Scheduled jobs: what a host runs on its own schedule, and switching it on, off and now.
A host runs work on its own: cron lines, run-parts scripts, anacron, systemd
timers, FreeBSD's ``periodic``, a firewall's or a NAS's own scheduled jobs, the
Windows Task Scheduler. :class:`ScheduledJobsMixin` reads all of them as one
shape, :class:`~napalm_device_types.models.ScheduledJobDict`, and acts on them.
Declared under ``if TYPE_CHECKING`` like a role: nothing exists at runtime until
a driver implements it. :class:`~napalm_device_types.scheduled_jobs_posix.PosixScheduledJobsMixin`
is the implementation for a host reached through the command channel.
**Who owns a definition** is on every job (``managed_by``): ``package`` (installed
by one), ``admin`` (written on the host), ``platform`` (generated by the device's
own configuration -- OPNsense, OpenMediaVault, QTS, Windows), ``application`` (written
through :meth:`put_scheduled_job`, under the ``name`` the application gave it).
What the application makes of that is the application's.
**What may be done is on every job too** (``actions``): every job a driver can
switch takes ``enable``, ``disable`` and ``run``; only an ``application`` job takes
``edit`` and ``delete``. A driver never edits or removes a job it did not write.
**Never over a change it did not see.** Every write names the ``digest`` the caller
saw. The driver reads the job again right before it writes and raises
:class:`ScheduledJobChanged` when the definition differs.
"""
from __future__ import annotations
import hashlib
import json
import re
from typing import TYPE_CHECKING, Any, NotRequired, TypedDict
from napalm_device_types.cron import is_reboot, parse_cron
#: What :meth:`ScheduledJobsMixin.manage_scheduled_job` takes.
ACTIONS = ("enable", "disable", "run")
#: What a job's ``actions`` may list beyond :data:`ACTIONS`.
EDIT = "edit"
DELETE = "delete"
KINDS = ("cron", "run_parts", "anacron", "systemd_timer", "periodic", "task")
MANAGED_BY = ("package", "admin", "platform", "application")
#: Seconds a run now may take on the host before ``timeout`` stops it.
RUN_TIMEOUT = 300
#: A job's name becomes a file and a unit name: lower case, digits, dashes.
_NAME = re.compile(r"[a-z0-9][a-z0-9-]{0,62}")
#: A user name as useradd and the BSDs take it.
_USER = re.compile(r"[A-Za-z_][A-Za-z0-9_.-]{0,31}\$?")
#: The parts of a job that are its definition; what the host says of its runs is not.
_DEFINITION = ("kind", "source", "run_as", "schedule", "command", "enabled")
class ScheduledJobDict(TypedDict):
"""A job the host runs on its own schedule (``get_scheduled_jobs``).
A cron line, a run-parts script, an anacron job, a systemd timer, a FreeBSD
``periodic`` script, a platform's own job, a Windows task -- one shape for all
of them (``napalm_device_types.scheduled_jobs``).
"""
key: str # stable name on this host: "timer:apt-daily.timer", "cron:/etc/crontab:1a2b3c4d"
kind: str # cron | run_parts | anacron | systemd_timer | periodic | task
source: str # the file, unit file or configuration it is defined in
run_as: str # the user it runs as
schedule: str # as the host has it: "*/5 * * * *", "OnCalendar=daily", "daily"
schedule_kind: str # cron | calendar | monotonic | period | triggers
command: str # as the host has it
enabled: bool
managed_by: str # package | admin | platform | application
digest: str # job_digest(): the definition as it is now
actions: list[str] # what the driver can do to this job: enable, disable, run, edit, delete
name: NotRequired[str] # the name an application gave a job it wrote (put_scheduled_job)
description: NotRequired[str]
last_run: NotRequired[str | None] # ISO 8601, UTC; where the host knows it
last_result: NotRequired[str | None] # "success", or what the host says went wrong
next_run: NotRequired[str | None] # ISO 8601, UTC; where the host knows it
class ScheduledJobsReadingDict(TypedDict):
"""Every scheduled job of a host, and the time zone its schedules are in."""
jobs: list[ScheduledJobDict]
time_zone: str | None # IANA name ("Europe/Berlin"); None when the host does not say
class ScheduledJobSpecDict(TypedDict):
"""A job an application wants on the host (``put_scheduled_job``)."""
name: str # the job's name on the host, chosen by the application: [a-z0-9][a-z0-9-]*
schedule: str # a cron expression (napalm_device_types.cron), "@reboot" included
run_as: str
command: str # one or more lines of POSIX sh; a platform may take its own (OPNsense: configd)
enabled: bool
description: NotRequired[str]
class ScheduledJobResultDict(TypedDict):
"""What an action on a scheduled job did."""
success: bool
output: str # what the host said; for a run, what the job printed
class ScheduledJobChanged(Exception): # noqa: N818 - a state, as the callers word it
"""The job's definition differs from the digest the caller saw: nothing was written."""
class ScheduledJobRefused(ValueError):
"""The job cannot take what was asked of it, or the request itself is not valid."""
def job_digest(job: dict[str, Any], *, content: str = "") -> str:
"""The definition of *job* in 16 hex characters.
Over kind, source, run_as, schedule, command and enabled -- so enabling a job
changes it -- and *content*: what the definition points at that the job's
fields do not show, such as a run-parts script's checksum.
"""
parts = [job.get(field) for field in _DEFINITION] + [content]
return hashlib.sha256(json.dumps(parts).encode()).hexdigest()[:16]
def check_job_name(name: str) -> None:
""":raises ScheduledJobRefused: for a name a file or a unit could not carry safely."""
if not _NAME.fullmatch(name or ""):
raise ScheduledJobRefused(f"not a job name: {name!r} (a-z, 0-9 and -, at most 63)")
def check_spec(spec: ScheduledJobSpecDict) -> None:
""":raises ScheduledJobRefused: for a spec no host could be given as it is."""
check_job_name(spec["name"])
if not _USER.fullmatch(spec.get("run_as") or ""):
raise ScheduledJobRefused(f"not a user name: {spec.get('run_as')!r}")
if not (spec.get("command") or "").strip():
raise ScheduledJobRefused("a job needs a command")
if "\n" in (spec.get("description") or ""):
raise ScheduledJobRefused("a description is one line")
if not is_reboot(spec["schedule"]):
try:
parse_cron(spec["schedule"])
except ValueError as exc:
raise ScheduledJobRefused(str(exc)) from exc
class ScheduledJobsMixin:
"""Declares the scheduled-jobs contract; a driver that can implements it.
A verb a driver cannot do at all is ``None`` on the driver, as ``manage_service``
is; a verb one job cannot take is missing from that job's ``actions``.
"""
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
def get_scheduled_jobs(self) -> ScheduledJobsReadingDict:
"""Every scheduled job of the host.
:raises ValueError: when the reading came back cut short -- never a
shorter list.
"""
...
def manage_scheduled_job(
self, key: str, action: str, *, digest: str
) -> ScheduledJobResultDict:
"""Enable, disable or run the job *key* now.
:raises ScheduledJobChanged: when the job is no longer as *digest* says.
:raises ScheduledJobRefused: when the job does not take *action*.
:raises LookupError: when the host has no job *key*.
"""
...
def put_scheduled_job(
self, spec: ScheduledJobSpecDict, *, digest: str | None = None
) -> ScheduledJobResultDict:
"""Create the job ``spec["name"]`` (*digest* None) or change it.
:raises ScheduledJobChanged: when the job is no longer as *digest*
says, or when a job of that name is there although *digest* is None.
:raises ScheduledJobRefused: for a spec :func:`check_spec` refuses, or
one the host cannot take.
"""
...
def delete_scheduled_job(self, key: str, *, digest: str) -> ScheduledJobResultDict:
"""Delete the job *key*, which an application wrote.
:raises ScheduledJobChanged: when the job is no longer as *digest* says.
:raises ScheduledJobRefused: when the job is not one an application wrote.
:raises LookupError: when the host has no job *key*.
"""
...
+493
View File
@@ -0,0 +1,493 @@
"""Scheduled jobs on a POSIX host, over the command channel: cron and systemd timers.
:class:`PosixScheduledJobsMixin` implements
:class:`~napalm_device_types.scheduled_jobs.ScheduledJobsMixin` for a driver that has
:meth:`~napalm_device_types.channel.CommandChannelMixin.run_command`. The driver
mixes it in and sets the paths its host keeps jobs in; the defaults are Debian's.
**Reading** is one framed report as root (:mod:`napalm_device_types._scheduled_reading`).
**Switching** a cron or anacron line comments it out behind ``#napalm-disabled: ``
and takes exactly that marker away again; a run-parts script loses or gets its
execute bit; a timer is ``systemctl disable --now`` / ``enable --now``. A timer that
cannot be switched (``static``) only runs.
**Writing** stages the new text as the login user and installs it as root with one
fixed command: ``crontab -u <user> <file>`` for a user's crontab, ``install`` with
the file's own mode and owner for a system file. Root never reads stdin. Before any
write the job is read again and compared with the digest the caller saw.
**An application's own job** (:meth:`put_scheduled_job`) is a timer on a host with
systemd -- ``<name>.timer`` and ``<name>.service`` in :attr:`UNIT_DIR`, the command
in :attr:`JOB_SCRIPT_DIR` ``/<name>`` behind a marker and its cron expression --
else a file in :attr:`OWN_CRON_D_DIR`, else marked lines in the user's crontab.
**Running now** is bounded by :data:`~napalm_device_types.scheduled_jobs.RUN_TIMEOUT`:
a timer's unit by ``systemctl start`` and its journal since then, a cron line's
command as its user the way cron runs it (``_crontab.run_script``).
"""
from __future__ import annotations
import fnmatch
import shlex
from collections.abc import Sequence
from typing import TYPE_CHECKING, ClassVar
from napalm_device_types._crontab import (
JOB_MARKER,
CronLine,
marked_lines,
parse_anacrontab,
parse_crontab,
run_script,
switched,
with_job,
without_job,
)
from napalm_device_types._scheduled_reading import (
OWNERS_SH,
SCHEDULE_HEADER,
TIME_ZONE_SH,
Paths,
Reading,
parse_reading,
reading_command,
)
from napalm_device_types.cron import is_reboot, to_on_calendar
from napalm_device_types.scheduled_jobs import (
ACTIONS,
RUN_TIMEOUT,
ScheduledJobChanged,
ScheduledJobRefused,
ScheduledJobResultDict,
ScheduledJobsMixin,
ScheduledJobSpecDict,
ScheduledJobsReadingDict,
check_spec,
)
if TYPE_CHECKING:
from napalm_device_types.channel import CommandResult
_STAGE = 't=$(mktemp) && chmod 644 "$t" && cat > "$t" && printf %s "$t"'
#: What a run keeps of the job's output.
_OUTPUT_LIMIT = 64 * 1024
_READ_TIMEOUT = 120
def _command(argv: Sequence[str]) -> str:
return shlex.join(argv)
def _said(result: CommandResult) -> str:
return "\n".join(p for p in (result.stdout.strip(), result.stderr.strip()) if p)[
-_OUTPUT_LIMIT:
]
def _result(result: CommandResult, done: str) -> ScheduledJobResultDict:
if result.exit_code != 0:
return {"success": False, "output": _said(result) or f"exit {result.exit_code}"}
return {"success": True, "output": _said(result) or done}
class PosixScheduledJobsMixin(ScheduledJobsMixin):
"""Scheduled jobs of a host reached through ``run_command``; the paths are the driver's."""
CRON_SYSTEM_FILES: ClassVar[tuple[str, ...]] = ("/etc/crontab",)
CRON_D_DIRS: ClassVar[tuple[str, ...]] = ("/etc/cron.d",)
CRON_SPOOL_DIRS: ClassVar[tuple[str, ...]] = (
"/var/spool/cron/crontabs",
"/var/spool/cron",
"/etc/crontabs",
)
RUN_PARTS_DIRS: ClassVar[tuple[tuple[str, str], ...]] = (
("hourly", "/etc/cron.hourly"),
("daily", "/etc/cron.daily"),
("weekly", "/etc/cron.weekly"),
("monthly", "/etc/cron.monthly"),
)
ANACRONTAB: ClassVar[str] = "/etc/anacrontab"
JOB_SCRIPT_DIR: ClassVar[str] = "/usr/local/libexec/scheduled-jobs"
UNIT_DIR: ClassVar[str] = "/etc/systemd/system"
#: Where a host without systemd takes an application's job as a file, if the directory
#: is there; else, or None, the user's crontab.
OWN_CRON_D_DIR: ClassVar[str | None] = "/etc/cron.d"
ROOT_GROUP: ClassVar[str] = "root"
#: Prints "<octal mode> <owner> <group>" of the path appended.
STAT_ARGV: ClassVar[tuple[str, ...]] = ("stat", "-c", "%a %U %G", "--")
#: ``install`` and how it makes missing directories (FreeBSD's ``-D`` is a destdir).
INSTALL: ClassVar[tuple[str, ...]] = ("install", "-D")
#: ``sh`` that reads definition paths on stdin and prints ``<package>: <path>``.
OWNERS_SH: ClassVar[str] = OWNERS_SH
#: ``sh`` that prints the time zone: an IANA name, or a path ending in ``zoneinfo/<name>``.
TIME_ZONE_SH: ClassVar[str] = TIME_ZONE_SH
#: ``fnmatch`` patterns on a job's ``source``: what the platform generates is ``platform``.
PLATFORM_SOURCES: ClassVar[tuple[str, ...]] = ()
#: Run as root after a user's crontab was written; for a host that keeps it elsewhere too.
CRONTAB_AFTER_WRITE: ClassVar[tuple[str, ...] | None] = None
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
def run_command(
self,
command: str,
*,
privileged: bool = False,
timeout: float = 60,
stdin: bytes | None = None,
) -> CommandResult: ...
# -- reading ---------------------------------------------------------------------------------
def _sj_paths(self) -> Paths:
return Paths(
system_files=self.CRON_SYSTEM_FILES,
cron_d_dirs=self.CRON_D_DIRS,
spool_dirs=self.CRON_SPOOL_DIRS,
run_parts=self.RUN_PARTS_DIRS,
anacrontab=self.ANACRONTAB,
script_dir=self.JOB_SCRIPT_DIR,
owners_sh=self.OWNERS_SH,
time_zone_sh=self.TIME_ZONE_SH,
)
def _sj_read(self) -> Reading:
result = self.run_command(
reading_command(self._sj_paths()), privileged=True, timeout=_READ_TIMEOUT
)
reading = parse_reading(result.stdout)
for job in reading.jobs:
source = str(job["source"])
if job["managed_by"] != "application" and any(
fnmatch.fnmatchcase(source, p) for p in self.PLATFORM_SOURCES
):
job["managed_by"] = "platform"
return reading
def get_scheduled_jobs(self) -> ScheduledJobsReadingDict:
"""Every scheduled job of the host, in one round trip.
:raises ValueError: when the report came back cut short.
"""
reading = self._sj_read()
return {"jobs": reading.jobs, "time_zone": reading.time_zone} # type: ignore[typeddict-item]
def _sj_job(self, reading: Reading, key: str, digest: str) -> dict[str, object]:
job = next((j for j in reading.jobs if j["key"] == key), None)
if job is None:
raise LookupError(f"no scheduled job {key}")
if job["digest"] != digest:
raise ScheduledJobChanged(f"{key} changed on the host since it was read")
return job
# -- commands --------------------------------------------------------------------------------
def _sj_root(self, argv: Sequence[str], timeout: float = 60) -> CommandResult:
return self.run_command(_command(argv), privileged=True, timeout=timeout)
def _sj_checked(self, argv: Sequence[str]) -> CommandResult:
result = self._sj_root(argv)
if result.exit_code != 0:
raise RuntimeError(f"{_command(argv)}: {_said(result) or result.exit_code}")
return result
def _sj_stage(self, content: str) -> str:
result = self.run_command(_STAGE, timeout=30, stdin=content.encode())
path = result.stdout.strip().splitlines()[-1:] or [""]
if result.exit_code != 0 or not path[0].startswith("/"):
raise RuntimeError(f"cannot stage a file: {_said(result)}")
return path[0]
def _sj_install(self, content: str, path: str, mode: str, owner: str, group: str) -> None:
staged = self._sj_stage(content)
try:
self._sj_checked(
[*self.INSTALL, "-m", mode, "-o", owner, "-g", group, "--", staged, path]
)
finally:
self.run_command(_command(["rm", "-f", "--", staged]), timeout=30)
# -- crontab files ---------------------------------------------------------------------------
def _sj_spool_user(self, path: str) -> str | None:
"""The user whose crontab *path* is; None for a system file."""
directory, _, user = path.rpartition("/")
return user if directory in self.CRON_SPOOL_DIRS else None
def _sj_crontab(self, user: str) -> str:
result = self._sj_root(["crontab", "-u", user, "-l"])
if result.exit_code != 0:
if "no crontab" in (result.stderr + result.stdout).lower():
return ""
raise RuntimeError(f"cannot read the crontab of {user}: {_said(result)}")
return result.stdout
def _sj_text(self, path: str) -> str:
user = self._sj_spool_user(path)
if user is not None:
return self._sj_crontab(user)
return self._sj_checked(["cat", "--", path]).stdout
def _sj_write_crontab(self, user: str, text: str) -> None:
staged = self._sj_stage(text)
try:
self._sj_checked(["crontab", "-u", user, staged])
finally:
self.run_command(_command(["rm", "-f", "--", staged]), timeout=30)
if self.CRONTAB_AFTER_WRITE:
self._sj_checked(self.CRONTAB_AFTER_WRITE)
def _sj_write(self, path: str, text: str) -> None:
user = self._sj_spool_user(path)
if user is not None:
self._sj_write_crontab(user, text)
return
# Through a symbolic link to the file it names, with that file's mode: the
# link's own mode is 777, and cron ignores a file anyone may write.
real = self._sj_checked(["readlink", "-f", "--", path]).stdout.strip() or path
mode, owner, group = self._sj_checked([*self.STAT_ARGV, real]).stdout.split()[:3]
self._sj_install(text, real, mode, owner, group)
def _sj_line(self, job: dict[str, object], text: str) -> CronLine:
"""The job's line in *text* as it is now, read again for the write."""
path = str(job["source"])
if job["kind"] == "anacron":
found = parse_anacrontab(text)
else:
user = self._sj_spool_user(path)
found = parse_crontab(path, text, system=user is None, user=user or "")
line = next((ln for ln in found if ln.key == job["key"]), None)
if line is None:
raise ScheduledJobChanged(f"{job['key']} is no longer in {path}")
return line
# -- switching and running -------------------------------------------------------------------
def _sj_timer_unit(self, job: dict[str, object]) -> str:
key = str(job["key"])
return f"{job['name']}.timer" if key.startswith("job:") else key.removeprefix("timer:")
def _sj_switch(self, job: dict[str, object], enabled: bool) -> ScheduledJobResultDict:
word = "enabled" if enabled else "disabled"
if job["kind"] == "systemd_timer":
verb = "enable" if enabled else "disable"
return _result(
self._sj_root(["systemctl", verb, "--now", "--", self._sj_timer_unit(job)]), word
)
if job["kind"] == "run_parts":
return _result(
self._sj_root(["chmod", "+x" if enabled else "-x", "--", str(job["source"])]), word
)
path = str(job["source"])
text = self._sj_text(path)
self._sj_write(path, switched(text, self._sj_line(job, text), enabled=enabled))
return {"success": True, "output": f"{job['key']} {word}"}
def _sj_as(self, user: str, script: str) -> list[str]:
"""*script* run by ``sh`` as *user*, from root."""
if user == "root":
return ["sh", "-c", script]
return ["su", "-s", "/bin/sh", user, "-c", script]
def _sj_run(self, job: dict[str, object], reading: Reading) -> ScheduledJobResultDict:
bound = ["timeout", str(RUN_TIMEOUT)]
wait = RUN_TIMEOUT + 30
if job["kind"] == "systemd_timer":
unit = reading.units[str(job["key"])]
since = self.run_command("date +%s", timeout=30).stdout.strip()
started = self._sj_root([*bound, "systemctl", "start", "--", unit], timeout=wait)
journal = self._sj_root(
[
"journalctl",
"-u",
unit,
"--since",
f"@{since}",
"--no-pager",
"-o",
"cat",
"-n",
"200",
]
)
output = "\n".join(p for p in (_said(started), journal.stdout.strip()) if p)
return {"success": started.exit_code == 0, "output": output or f"{unit} ran"}
if job["kind"] == "run_parts":
return _result(self._sj_root([*bound, str(job["source"])], timeout=wait), "ran")
path = str(job["source"])
line = self._sj_line(job, self._sj_text(path))
script = run_script(line.command, line.env)
return _result(
self._sj_root([*bound, *self._sj_as(line.run_as, script)], timeout=wait), "ran"
)
def manage_scheduled_job(self, key: str, action: str, *, digest: str) -> ScheduledJobResultDict:
"""Enable, disable or run the job *key* now; see :class:`ScheduledJobsMixin`."""
if action not in ACTIONS:
raise ScheduledJobRefused(f"unknown action {action!r}; allowed: {list(ACTIONS)}")
reading = self._sj_read()
job = self._sj_job(reading, key, digest)
if action not in job["actions"]: # type: ignore[operator]
raise ScheduledJobRefused(f"{key} cannot be told to {action} on this host")
if action == "run":
return self._sj_run(job, reading)
return self._sj_switch(job, action == "enable")
# -- an application's own jobs ---------------------------------------------------------------
def _sj_own(
self, reading: Reading, spec: ScheduledJobSpecDict, digest: str | None
) -> dict[str, object] | None:
current = next((j for j in reading.jobs if j["key"] == f"job:{spec['name']}"), None)
if digest is None:
if current is not None:
raise ScheduledJobChanged(f"a job {spec['name']} is there already")
return None
if current is None:
raise LookupError(f"no scheduled job job:{spec['name']}")
if current["digest"] != digest:
raise ScheduledJobChanged(f"job:{spec['name']} changed on the host since it was read")
return current
def _sj_group(self, user: str) -> str:
result = self.run_command(_command(["id", "-gn", "--", user]), timeout=30)
if result.exit_code != 0:
raise ScheduledJobRefused(f"the host has no user {user}")
return result.stdout.strip()
def _sj_calendar(self, schedule: str) -> list[str]:
if is_reboot(schedule):
return ["OnBootSec=1min"]
lines = to_on_calendar(schedule)
for line in lines:
checked = self.run_command(
_command(["systemd-analyze", "calendar", "--", line]), timeout=30
)
if checked.exit_code != 0:
raise ScheduledJobRefused(f"systemd does not take {line!r}: {_said(checked)}")
return [f"OnCalendar={line}" for line in lines]
def _sj_put_timer(self, spec: ScheduledJobSpecDict) -> None:
name, description = spec["name"], spec.get("description") or spec["name"]
script = f"{self.JOB_SCRIPT_DIR}/{name}"
unit_description = description.replace("%", "%%")
timer = "\n".join(
[
"[Unit]",
f"Description={unit_description}",
"",
"[Timer]",
*self._sj_calendar(spec["schedule"]),
f"Unit={name}.service",
"",
"[Install]",
"WantedBy=timers.target",
"",
]
)
service = "\n".join(
[
"[Unit]",
f"Description={unit_description}",
"",
"[Service]",
"Type=oneshot",
f"User={spec['run_as']}",
f"ExecStart={script}",
"",
]
)
body = f"#!/bin/sh\n{JOB_MARKER}{name} {spec.get('description') or ''}".rstrip()
body += f"\n{SCHEDULE_HEADER}{spec['schedule']}\n{spec['command'].strip()}\n"
self._sj_install(body, script, "750", "root", self._sj_group(spec["run_as"]))
self._sj_install(service, f"{self.UNIT_DIR}/{name}.service", "644", "root", self.ROOT_GROUP)
self._sj_install(timer, f"{self.UNIT_DIR}/{name}.timer", "644", "root", self.ROOT_GROUP)
self._sj_checked(["systemctl", "daemon-reload"])
verb = "enable" if spec["enabled"] else "disable"
self._sj_checked(["systemctl", verb, "--now", "--", f"{name}.timer"])
def _sj_cron_text(self, spec: ScheduledJobSpecDict, *, system: bool) -> list[str]:
if "\n" in spec["command"].strip():
raise ScheduledJobRefused("a cron line holds one line of command")
owner = f" {spec['run_as']}" if system else ""
line = f"{spec['schedule']}{owner} {spec['command'].strip()}"
return marked_lines(
spec["name"], spec.get("description") or "", line, enabled=spec["enabled"]
)
def _sj_cron_d(self) -> str | None:
""":attr:`OWN_CRON_D_DIR` where the host has it; BusyBox crond reads none."""
directory = self.OWN_CRON_D_DIR
if directory is None:
return None
found = self.run_command(_command(["test", "-d", directory]), timeout=30)
return directory if found.exit_code == 0 else None
def _sj_put_cron(self, spec: ScheduledJobSpecDict, current: dict[str, object] | None) -> None:
if self._sj_cron_d() is not None:
self._sj_group(spec["run_as"])
text = "\n".join(self._sj_cron_text(spec, system=True)) + "\n"
self._sj_install(
text, f"{self.OWN_CRON_D_DIR}/{spec['name']}", "644", "root", self.ROOT_GROUP
)
return
if current is not None and current["run_as"] != spec["run_as"]:
self._sj_delete_cron(current)
user = spec["run_as"]
text = self._sj_crontab(user)
jobs = parse_crontab("", text, system=False, user=user)
self._sj_write_crontab(
user, with_job(text, jobs, spec["name"], self._sj_cron_text(spec, system=False))
)
def put_scheduled_job(
self, spec: ScheduledJobSpecDict, *, digest: str | None = None
) -> ScheduledJobResultDict:
"""Create or change the application's own job; see :class:`ScheduledJobsMixin`."""
check_spec(spec)
reading = self._sj_read()
current = self._sj_own(reading, spec, digest)
if reading.systemd:
self._sj_put_timer(spec)
else:
self._sj_put_cron(spec, current)
return {
"success": True,
"output": f"job:{spec['name']} {'changed' if current else 'created'}",
}
def _sj_delete_cron(self, job: dict[str, object]) -> None:
path = str(job["source"])
if self._sj_spool_user(path) is None:
self._sj_checked(["rm", "-f", "--", path])
return
text = self._sj_text(path)
jobs = parse_crontab(path, text, system=False, user=str(job["run_as"]))
self._sj_write(path, without_job(text, jobs, str(job["name"])))
def delete_scheduled_job(self, key: str, *, digest: str) -> ScheduledJobResultDict:
"""Delete the application's job *key*; see :class:`ScheduledJobsMixin`."""
job = self._sj_job(self._sj_read(), key, digest)
if job["managed_by"] != "application":
raise ScheduledJobRefused(
f"{key} was not written by an application; it is only switched"
)
name = str(job["name"])
if job["kind"] != "systemd_timer":
self._sj_delete_cron(job)
return {"success": True, "output": f"{key} deleted"}
self._sj_root(["systemctl", "disable", "--now", "--", f"{name}.timer"])
self._sj_checked(
[
"rm",
"-f",
"--",
f"{self.UNIT_DIR}/{name}.timer",
f"{self.UNIT_DIR}/{name}.service",
f"{self.JOB_SCRIPT_DIR}/{name}",
]
)
self._sj_checked(["systemctl", "daemon-reload"])
return {"success": True, "output": f"{key} deleted"}
+25 -4
View File
@@ -15,7 +15,9 @@ accident.
``/etc/ssh/sshd_config.d/*.conf`` -- Debian and Ubuntu do, at the top -- the
lists go to :data:`DROPIN_PATH`, which sorts first and so wins. sshd_config
stays as the distribution shipped it, and removing the file undoes the change.
Only without such an ``Include`` is sshd_config itself rewritten, after a copy.
Only without such an ``Include`` is sshd_config itself rewritten, after a copy --
and not at all on a platform that generates it
(:attr:`OpenSshServerMixin.SSHD_CONFIG_GENERATED`).
**Nobody is locked out, by three checks.**
@@ -51,6 +53,8 @@ from typing import (
TYPE_CHECKING,
)
from napalm_device_types.ssh_server_auth import SSHD_EFFECTIVE_COMMAND, SshServerAuthMixin
if TYPE_CHECKING: # pragma: no cover - for type checkers only
from napalm_device_types.channel import CommandResult
@@ -418,7 +422,7 @@ def _unchanged(
return result
class OpenSshServerMixin:
class OpenSshServerMixin(SshServerAuthMixin):
"""Adds :meth:`harden_ssh_server` to a driver that can run commands as root on an OpenSSH host.
Mixed in by the drivers whose host runs a stock OpenSSH, not by a base class,
@@ -426,9 +430,19 @@ class OpenSshServerMixin:
whose platform writes sshd_config itself (an appliance that regenerates it
from its own database) must not inherit this, or must opt out with
``harden_ssh_server = None``: its file would be overwritten on the next
apply.
apply. Where that generated file still includes the drop-in directory, the
driver sets :attr:`SSHD_CONFIG_GENERATED` instead and keeps the hardening.
It carries :class:`~napalm_device_types.ssh_server_auth.SshServerAuthMixin`:
a host whose sshd can be hardened can also say how it lets people log in.
"""
#: True where the platform writes sshd_config from its own database
#: (OpenMediaVault): only the drop-in directory is written, and a
#: sshd_config that does not include it is left alone, since the platform
#: would write over a change to it on its next apply.
SSHD_CONFIG_GENERATED: bool = False
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
def run_command(
@@ -504,7 +518,7 @@ class OpenSshServerMixin:
return parse_sshd_effective(self._sshd_effective_text())
def _sshd_effective_text(self) -> str:
result = self._ssh_run("PATH=$PATH:/usr/sbin:/sbin sshd -T")
result = self._ssh_run(SSHD_EFFECTIVE_COMMAND)
if result.exit_code != 0:
raise RuntimeError(f"sshd -T failed: {(result.stderr or result.stdout).strip()}")
return result.stdout
@@ -565,6 +579,13 @@ class OpenSshServerMixin:
main = self._ssh_read(MAIN_CONFIG)
if main is None:
raise RuntimeError(f"cannot read {MAIN_CONFIG}")
if self.SSHD_CONFIG_GENERATED and not has_dropin_include(main):
result.message = (
f"Not changed: {MAIN_CONFIG} does not include {DROPIN_DIR}, and this"
" platform writes it from its own settings."
)
say(result.message)
return result
restore = self._write_change(plan, main, result, say)
return self._apply_or_restore(
result,
+281
View File
@@ -0,0 +1,281 @@
"""How an OpenSSH server lets people log in: the effective settings and sshd's PAM stack.
Whether a host takes a password, only a key, or a key *and* a second factor is
the same question on every host that runs OpenSSH, so the reading lives here,
and a driver only mixes in :class:`SshServerAuthMixin` and supplies
``run_command``.
**What sshd uses, not what a file says.** ``sshd -T`` prints the settings after
sshd's defaults, ``sshd_config``, every ``Include`` and first-match-wins -- the
same reading :class:`~napalm_device_types.ssh_server.OpenSshServerMixin` plans
its hardening on, by the same fixed command (:data:`SSHD_EFFECTIVE_COMMAND`). It
needs root: without it sshd cannot load its host keys and exits.
**PAM, as far as it can be read.** With ``UsePAM yes``, keyboard-interactive
asks whatever PAM's ``auth`` stack for sshd asks: a password from
``pam_unix.so``, a one-time code from an OTP module, or both. The stack is read
from ``/etc/pam.d`` in one listing as the login user (the files are
world-readable), and followed through ``@include``, ``include`` and
``substack``. Each module is reported with its control and arguments as
written; which of them is a second factor, and whether ``nullok`` lets a user
without one through, is the consumer's call.
Not read: ``Match`` blocks (``sshd -T`` without ``-C`` prints the global
settings), and what a PAM module's own configuration asks for.
"""
from __future__ import annotations
import re
from collections.abc import Mapping
from typing import TYPE_CHECKING, TypedDict
if TYPE_CHECKING: # pragma: no cover - for type checkers only
from napalm_device_types.channel import CommandResult
#: sshd's effective configuration. ``sshd`` lives in ``/usr/sbin``, outside a login
#: user's ``PATH``. One fixed command, run as root.
SSHD_EFFECTIVE_COMMAND = "PATH=$PATH:/usr/sbin:/sbin sshd -T"
PAM_DIR = "/etc/pam.d"
#: Every file in ``/etc/pam.d``, each after a ``==> path <==`` header (GNU, BSD and
#: busybox ``tail`` alike). One fixed command, as the login user.
PAM_FILES_COMMAND = f"tail -n +1 -- {PAM_DIR}/*"
#: The PAM service sshd runs as.
PAM_SERVICE = "sshd"
_TIMEOUT = 30
_HEADER_RE = re.compile(rf"^==> {re.escape(PAM_DIR)}/(?P<name>[^/]+) <==$")
#: A service name an include may name: a file in ``/etc/pam.d``, never a path out of it.
_SERVICE_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]*$")
_INCLUDE_CONTROLS = ("include", "substack")
class PamAuthEntryDict(TypedDict):
"""One line of a PAM ``auth`` stack: ``auth required pam_unix.so nullok``."""
#: The module's file name, without its directory: ``pam_unix.so``.
module: str
#: ``required``, ``requisite``, ``sufficient``, ``optional``, or a bracketed
#: ``[value=action ...]`` as written.
control: str
args: list[str]
class SshServerAuthDict(TypedDict):
"""How an OpenSSH server lets people log in, as ``get_ssh_server_auth`` read it.
The global settings ``sshd -T`` prints. A ``Match`` block can change them for
some users or addresses; ``sshd -T`` without ``-C`` does not show that.
``authentication_methods`` is ``AuthenticationMethods``: each inner list is one
way in, every method of it required in turn (``publickey,keyboard-interactive``
-> ``["publickey", "keyboard-interactive"]``); empty for ``any``, where any one
enabled method is enough. A method keeps its submethod
(``keyboard-interactive:pam``).
``use_pam`` is None where sshd has no PAM (OpenBSD). ``pam_auth_stack`` is the
``auth`` stack PAM runs for sshd, includes and substacks followed; None when
PAM is off or its files gave nothing for sshd.
"""
password_authentication: bool
#: ``KbdInteractiveAuthentication``; before OpenSSH 8.7, either it or
#: ``ChallengeResponseAuthentication``.
kbd_interactive_authentication: bool
pubkey_authentication: bool
hostbased_authentication: bool
#: False also where sshd was built without GSSAPI.
gssapi_authentication: bool
authentication_methods: list[list[str]]
#: ``yes``, ``prohibit-password`` (``without-password``), ``forced-commands-only`` or ``no``.
permit_root_login: str
use_pam: bool | None
pam_auth_stack: list[PamAuthEntryDict] | None
def _flag(settings: Mapping[str, str], keyword: str) -> bool:
return settings.get(keyword, "no") == "yes"
def _settings(text: str) -> dict[str, str]:
"""``keyword value`` per line, the first one of each keyword."""
found: dict[str, str] = {}
for raw in text.splitlines():
parts = raw.strip().split(None, 1)
if len(parts) == 2:
found.setdefault(parts[0].lower(), parts[1].strip())
return found
def _methods(value: str) -> list[list[str]]:
if value == "any":
return []
return [[m for m in alternative.split(",") if m] for alternative in value.split()]
def parse_sshd_auth(text: str) -> SshServerAuthDict:
"""The authentication settings from ``sshd -T`` output; ``pam_auth_stack`` left None.
:raises ValueError: if a setting every OpenSSH prints is missing -- not sshd -T
output, or cut short.
"""
settings = _settings(text)
missing = [
k
for k in ("passwordauthentication", "pubkeyauthentication", "permitrootlogin")
if k not in settings
]
if missing:
raise ValueError(f"sshd -T printed no {', '.join(missing)}")
use_pam = settings.get("usepam")
return {
"password_authentication": _flag(settings, "passwordauthentication"),
"kbd_interactive_authentication": _flag(settings, "kbdinteractiveauthentication")
or _flag(settings, "challengeresponseauthentication"),
"pubkey_authentication": _flag(settings, "pubkeyauthentication"),
"hostbased_authentication": _flag(settings, "hostbasedauthentication"),
"gssapi_authentication": _flag(settings, "gssapiauthentication"),
"authentication_methods": _methods(settings.get("authenticationmethods", "any")),
"permit_root_login": settings["permitrootlogin"],
"use_pam": None if use_pam is None else use_pam == "yes",
"pam_auth_stack": None,
}
def _pam_tokens(line: str) -> list[str]:
"""A PAM line's tokens, a bracketed control (``[success=1 default=ignore]``) as one."""
tokens: list[str] = []
for part in line.split():
if tokens and tokens[-1].startswith("[") and not tokens[-1].endswith("]"):
tokens[-1] += " " + part
else:
tokens.append(part)
return tokens
def _pam_entry(line: str) -> PamAuthEntryDict | None:
tokens = _pam_tokens(line.split("#", 1)[0])
if len(tokens) >= 2 and tokens[0] == "@include":
tokens = ["auth", "include", tokens[1]]
if len(tokens) < 3 or tokens[0].lstrip("-") != "auth":
return None
control, target, args = tokens[1], tokens[2], tokens[3:]
if control in _INCLUDE_CONTROLS:
if not _SERVICE_RE.match(target):
return None
return {"module": target, "control": control, "args": []}
return {"module": target.rsplit("/", 1)[-1], "control": control, "args": args}
def parse_pam_auth(text: str) -> list[PamAuthEntryDict]:
"""One PAM file's ``auth`` stack, in order.
A module by its file name (``pam_unix.so``), with its control and arguments.
An include is an entry whose control is ``include`` or ``substack`` and whose
module is the service it names: ``@include common-auth`` brings in a whole
file, ``auth include`` and ``auth substack`` its ``auth`` lines; both are
followed by :func:`pam_auth_stack`. An include that names a path rather than
a service is left out.
"""
return [entry for line in text.splitlines() if (entry := _pam_entry(line.strip()))]
def parse_pam_files(text: str) -> dict[str, str]:
""":data:`PAM_FILES_COMMAND`'s output, by file name."""
files: dict[str, list[str]] = {}
current: list[str] | None = None
for line in text.splitlines():
header = _HEADER_RE.match(line.strip())
if header:
current = files.setdefault(header["name"], [])
elif current is not None:
current.append(line)
return {name: "\n".join(lines).strip("\n") + "\n" for name, lines in files.items()}
def pam_auth_stack(
files: Mapping[str, str], service: str = PAM_SERVICE
) -> list[PamAuthEntryDict] | None:
"""*service*'s ``auth`` modules, includes and substacks followed, in stack order.
A file included twice -- or in a loop -- counts once; an include of a file
that is not there adds nothing, as with PAM. None without a file for *service*.
"""
if service not in files:
return None
stack: list[PamAuthEntryDict] = []
seen: set[str] = set()
def walk(name: str) -> None:
if name in seen or name not in files:
return
seen.add(name)
for entry in parse_pam_auth(files[name]):
if entry["control"] in _INCLUDE_CONTROLS:
walk(entry["module"])
else:
stack.append(entry)
walk(service)
return stack
class SshServerAuthMixin:
"""Adds :meth:`get_ssh_server_auth` to a driver that can run commands on an OpenSSH host.
Mixed in by the drivers whose host runs a stock OpenSSH --
:class:`~napalm_device_types.ssh_server.OpenSshServerMixin` carries it -- so
``hasattr(driver, "get_ssh_server_auth")`` stays a truthful answer. Reading,
not writing: a platform that generates sshd_config itself can carry it too.
"""
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
def run_command(
self,
command: str,
*,
privileged: bool = False,
timeout: float = 60,
stdin: bytes | None = None,
) -> CommandResult: ...
def get_ssh_server_auth(self) -> SshServerAuthDict:
"""
Returns how sshd lets people log in, from ``sshd -T`` and PAM's sshd stack.
Example::
{
"password_authentication": False,
"kbd_interactive_authentication": True,
"pubkey_authentication": True,
"hostbased_authentication": False,
"gssapi_authentication": False,
"authentication_methods": [["publickey", "keyboard-interactive"]],
"permit_root_login": "no",
"use_pam": True,
"pam_auth_stack": [
{"module": "pam_google_authenticator.so", "control": "required", "args": []}
],
}
:raises RuntimeError: if ``sshd -T`` fails -- no OpenSSH, or no root.
"""
result = self.run_command(SSHD_EFFECTIVE_COMMAND, privileged=True, timeout=_TIMEOUT)
if result.exit_code != 0:
raise RuntimeError(f"sshd -T failed: {(result.stderr or result.stdout).strip()}")
auth = parse_sshd_auth(result.stdout)
if auth["use_pam"]:
auth["pam_auth_stack"] = self._pam_auth_stack()
return auth
def _pam_auth_stack(self) -> list[PamAuthEntryDict] | None:
"""sshd's PAM auth stack; None when ``/etc/pam.d`` gave nothing for sshd.
``tail`` goes on past a file it cannot open and exits 1, so what it did
print is used whatever the exit status.
"""
result = self.run_command(PAM_FILES_COMMAND, privileged=False, timeout=_TIMEOUT)
return pam_auth_stack(parse_pam_files(result.stdout))
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "napalm-device-types"
version = "4.25.0"
version = "4.34.0"
description = "Abstract device-type base classes for NAPALM drivers"
readme = "README.md"
requires-python = ">=3.12"
+1 -1
View File
@@ -32,7 +32,7 @@
},
"mypy_relaxed": [],
"python_file_lines": {
"napalm_device_types/models.py": 1463
"napalm_device_types/models.py": 1454
},
"untyped_public": []
}
+141
View File
@@ -0,0 +1,141 @@
"""Cron expressions: what they mean, when they fire next, and as systemd ``OnCalendar``.
One module answers all three, because a job netOrk writes as a timer must fire
exactly when the cron expression the user gave says, and the preview must show
the same runs the host will make (NetOrk/netork#1933).
"""
from __future__ import annotations
from datetime import UTC, datetime
from zoneinfo import ZoneInfo
import pytest
from napalm_device_types.cron import is_reboot, next_runs, parse_cron, to_on_calendar
BERLIN = ZoneInfo("Europe/Berlin")
class TestParse:
def test_every_field_is_expanded_to_its_values(self):
s = parse_cron("*/15 6-8 1,15 * *")
assert s.minutes == frozenset({0, 15, 30, 45})
assert s.hours == frozenset({6, 7, 8})
assert s.days == frozenset({1, 15})
assert s.months == frozenset(range(1, 13))
assert s.weekdays == frozenset(range(7))
def test_a_range_takes_a_step(self):
assert parse_cron("1-30/10 * * * *").minutes == frozenset({1, 11, 21})
def test_names_are_months_and_weekdays_in_any_case(self):
s = parse_cron("0 0 * Jan,MAR mon-Fri")
assert s.months == frozenset({1, 3})
assert s.weekdays == frozenset({1, 2, 3, 4, 5})
def test_seven_is_sunday_too(self):
assert parse_cron("0 0 * * 7").weekdays == frozenset({0})
@pytest.mark.parametrize(
("alias", "expanded"),
[
("@hourly", "0 * * * *"),
("@daily", "0 0 * * *"),
("@midnight", "0 0 * * *"),
("@weekly", "0 0 * * 0"),
("@monthly", "0 0 1 * *"),
("@yearly", "0 0 1 1 *"),
("@annually", "0 0 1 1 *"),
],
)
def test_an_alias_is_its_expression(self, alias, expanded):
assert parse_cron(alias) == parse_cron(expanded)
def test_reboot_is_no_calendar(self):
assert is_reboot("@reboot")
assert not is_reboot("0 0 * * *")
with pytest.raises(ValueError, match="reboot"):
parse_cron("@reboot")
@pytest.mark.parametrize(
"bad",
[
"",
"* * * *",
"* * * * * *",
"60 * * * *",
"* 24 * * *",
"* * 0 * *",
"* * * 13 *",
"* * * * 8",
"*/0 * * * *",
"5-1 * * * *",
"a * * * *",
"@often",
"1,,2 * * * *",
],
)
def test_what_cron_would_not_take_is_refused(self, bad):
with pytest.raises(ValueError):
parse_cron(bad)
def test_day_of_month_and_weekday_are_or_only_when_both_are_restricted(self):
assert parse_cron("0 0 1 * 1").either_day
assert not parse_cron("0 0 */2 * 1").either_day
assert not parse_cron("0 0 1 * *").either_day
class TestNextRuns:
def test_runs_after_the_instant_in_the_hosts_time(self):
after = datetime(2026, 11, 3, 5, 50, tzinfo=UTC) # 06:50 in Berlin
runs = next_runs("0 7 * * *", after, BERLIN, count=2)
assert runs == [
datetime(2026, 11, 3, 7, 0, tzinfo=BERLIN),
datetime(2026, 11, 4, 7, 0, tzinfo=BERLIN),
]
def test_the_instant_itself_is_not_a_next_run(self):
after = datetime(2026, 11, 3, 6, 0, tzinfo=UTC)
assert next_runs("0 6 * * *", after, ZoneInfo("UTC"), count=1)[0].day == 4
def test_either_day_fires_on_both(self):
after = datetime(2026, 11, 30, 12, 0, tzinfo=UTC) # a Monday
runs = next_runs("0 0 1 * 1", after, ZoneInfo("UTC"), count=3)
assert [r.day for r in runs] == [1, 7, 14]
def test_a_day_that_never_comes_gives_no_runs(self):
assert next_runs("0 0 31 2 *", datetime(2026, 1, 1, tzinfo=UTC), BERLIN) == []
def test_a_time_the_clock_skips_is_left_out(self):
# 2026-03-29 02:30 does not exist in Berlin.
after = datetime(2026, 3, 28, 12, 0, tzinfo=UTC)
runs = next_runs("30 2 * * *", after, BERLIN, count=2)
assert [r.day for r in runs] == [30, 31]
def test_reboot_has_no_next_run(self):
assert next_runs("@reboot", datetime(2026, 1, 1, tzinfo=UTC), BERLIN) == []
class TestOnCalendar:
def test_every_field_as_its_values(self):
assert to_on_calendar("*/15 6 * * *") == ["*-*-* 06:00,15,30,45:00"]
def test_a_restricted_weekday_leads(self):
assert to_on_calendar("30 4 * * 1-5") == ["Mon,Tue,Wed,Thu,Fri *-*-* 04:30:00"]
def test_months_and_days(self):
assert to_on_calendar("0 0 1,15 1,7 *") == ["*-01,07-01,15 00:00:00"]
def test_either_day_is_two_lines(self):
assert to_on_calendar("0 0 1 * 0") == ["*-*-01 00:00:00", "Sun *-*-* 00:00:00"]
def test_both_days_with_a_star_are_one_line(self):
assert to_on_calendar("0 0 */10 * 6") == ["Sat *-*-01,11,21,31 00:00:00"]
def test_an_alias(self):
assert to_on_calendar("@daily") == ["*-*-* 00:00:00"]
def test_reboot_is_refused(self):
with pytest.raises(ValueError, match="reboot"):
to_on_calendar("@reboot")
+145
View File
@@ -0,0 +1,145 @@
"""Crontabs as text: the jobs in them, and changing one line (NetOrk/netork#1933)."""
from __future__ import annotations
import subprocess
import pytest
from napalm_device_types._crontab import (
DISABLED_MARKER,
marked_lines,
parse_anacrontab,
parse_crontab,
run_script,
switched,
with_job,
without_job,
)
SYSTEM = """\
# /etc/crontab: system-wide crontab
SHELL=/bin/sh
PATH="/usr/local/sbin:/usr/bin:/bin"
17 * * * * root cd / && run-parts --report /etc/cron.hourly
@reboot root /usr/local/bin/warm-cache # keeps the # in the command
#napalm-disabled: 0 4 * * * backup /usr/local/bin/backup
garbage line
0 4 * * * backup /usr/local/bin/backup
"""
USER = """\
MAILTO=ops@example.org
*/5 * * * * /usr/bin/php /var/www/cron.php
# napalm-scheduled-job: netork-cleanup tidy the cache
0 3 * * * find /tmp -mtime +7 -delete
"""
class TestParse:
def test_a_system_crontab_names_the_user(self):
jobs = parse_crontab("/etc/crontab", SYSTEM, system=True)
first = jobs[0]
assert (first.schedule, first.run_as) == ("17 * * * *", "root")
assert first.command == "cd / && run-parts --report /etc/cron.hourly"
assert first.env == {"SHELL": "/bin/sh", "PATH": "/usr/local/sbin:/usr/bin:/bin"}
assert first.key.startswith("cron:/etc/crontab:")
def test_an_alias_and_a_hash_inside_the_command(self):
reboot = parse_crontab("/etc/crontab", SYSTEM, system=True)[1]
assert reboot.schedule == "@reboot"
assert reboot.command == "/usr/local/bin/warm-cache # keeps the # in the command"
def test_a_line_switched_off_is_a_job_that_is_off(self):
jobs = parse_crontab("/etc/crontab", SYSTEM, system=True)
off = jobs[2]
assert not off.enabled
assert off.run_as == "backup"
def test_equal_lines_get_their_own_keys_and_switching_off_keeps_the_key(self):
jobs = parse_crontab("/etc/crontab", SYSTEM, system=True)
assert jobs[3].key == jobs[2].key + ":2"
assert len(jobs) == 4 # the garbage line is left out
def test_a_user_crontab_runs_as_its_user(self):
jobs = parse_crontab("/var/spool/cron/crontabs/www", USER, system=False, user="www-data")
assert jobs[0].run_as == "www-data"
assert jobs[0].env == {"MAILTO": "ops@example.org"}
def test_an_applications_job_is_keyed_by_its_name(self):
job = parse_crontab("/x", USER, system=False, user="root")[1]
assert (job.key, job.name, job.description) == (
"job:netork-cleanup",
"netork-cleanup",
"tidy the cache",
)
def test_an_anacrontab(self):
text = "SHELL=/bin/sh\n1\t5\tcron.daily\trun-parts --report /etc/cron.daily\n@monthly 15 cron.monthly run-parts /etc/cron.monthly\nbad line here\n"
jobs = parse_anacrontab(text)
assert [(j.key, j.schedule, j.run_as) for j in jobs] == [
("anacron:cron.daily", "1", "root"),
("anacron:cron.monthly", "@monthly", "root"),
]
assert jobs[0].command == "run-parts --report /etc/cron.daily"
class TestEdit:
def test_switching_off_and_on_again_gives_the_text_back(self):
jobs = parse_crontab("/etc/crontab", SYSTEM, system=True)
off = switched(SYSTEM, jobs[0], enabled=False)
assert DISABLED_MARKER + "17 *\t* * *\troot" in off
again = parse_crontab("/etc/crontab", off, system=True)
assert again[0].key == jobs[0].key and not again[0].enabled
assert switched(off, again[0], enabled=True) == SYSTEM.replace(
"17 *\t* * *\troot\tcd", "17 *\t* * *\troot\tcd"
)
def test_an_applications_job_is_added_replaced_and_removed(self):
lines = marked_lines("netork-x", "a job", "0 1 * * * /bin/true", enabled=True)
added = with_job("A=1\n", [], "netork-x", lines)
assert added == "A=1\n# napalm-scheduled-job: netork-x a job\n0 1 * * * /bin/true\n"
jobs = parse_crontab("/x", added, system=False, user="root")
changed = with_job(
added,
jobs,
"netork-x",
marked_lines("netork-x", "", "0 2 * * * /bin/true", enabled=False),
)
assert (
changed
== "A=1\n# napalm-scheduled-job: netork-x\n#napalm-disabled: 0 2 * * * /bin/true\n"
)
jobs = parse_crontab("/x", changed, system=False, user="root")
assert without_job(changed, jobs, "netork-x") == "A=1\n"
def test_adding_to_a_text_without_a_last_newline(self):
assert with_job("A=1", [], "n", ["# m", "x"]) == "A=1\n# m\nx\n"
def test_removing_the_only_job_leaves_nothing(self):
text = "# napalm-scheduled-job: n\n0 1 * * * x\n"
jobs = parse_crontab("/x", text, system=False, user="root")
assert without_job(text, jobs, "n") == ""
class TestRunScript:
def _run(self, script: str) -> str:
return subprocess.run(
["sh", "-c", script], capture_output=True, text=True, check=True
).stdout
def test_the_crontabs_variables_are_set(self):
assert (
self._run(run_script('echo "$GREETING"', {"GREETING": "hi there", "MAILTO": "x"}))
== "hi there\n"
)
def test_a_percent_starts_stdin_and_further_ones_are_newlines(self):
assert self._run(run_script("cat%one%two", {})) == "one\ntwo\n"
def test_an_escaped_percent_is_a_percent(self):
assert self._run(run_script("date -d @0 -u +\\%Y", {})) == "1970\n"
@pytest.mark.parametrize("command", ["echo plain", "echo 'quoted'"])
def test_without_percent_the_command_is_as_it_is(self, command):
assert run_script(command, {}) == command
+376
View File
@@ -13,6 +13,8 @@ from __future__ import annotations
import os
import re
import shutil
import sqlite3
import subprocess
import pytest
@@ -25,6 +27,12 @@ from napalm_device_types.host_status import (
kernel_reboot_pending,
parse_host_status,
)
from napalm_device_types.package_managers import (
APK_INSTALLED,
DNF_HISTORY,
OPKG_STATUS,
PACKAGE_MANAGERS_COMMAND,
)
def _wire(
@@ -570,3 +578,371 @@ class TestFilesystems:
mountpoints = [fs["mountpoint"] for fs in parse_host_status(out.stdout)["filesystems"]]
assert "/" in mountpoints
def _other(*sections: tuple[str, tuple[str, ...]], running: tuple[str, ...] = ()) -> str:
"""A report from a host without dpkg: the given sections before ``[timers]``."""
lines = ["[pm-running]", *running]
for name, body in sections:
lines += [f"[{name}]", *body]
return _wire().replace("[timers]", "\n".join([*lines, "[timers]"]))
class TestInterruptedRpm:
"""dnf and yum hosts: a transaction rpm was cut off in leaves the old and the
new version of a package both installed (``dnf check --duplicates``,
``package-cleanup --dupes``), and dnf's history records a transaction that
never ended. netOrk only reports it: there is no one repair that is safe on
every host (NetOrk/netork#851, decided 2026-11-07).
The outputs are shaped after the documented formats -- ``rpm -qa --qf``,
libdnf's ``trans`` table (``state`` 0 = UNKNOWN until ``finish()`` sets DONE
1 or ERROR 2), yum's ``transaction-all.*`` journal that
``yum-complete-transaction`` reads -- and still have to be checked against a
real Oracle Linux 9 host.
"""
INSTALLONLY = ("kernel-core", "kernel-modules", "kernel-uek-core")
#: An upgrade of glibc and openssl-libs cut off after the new versions were in.
CUT_OFF = ("glibc x86_64", "kernel-uek-core x86_64", "openssl-libs x86_64")
def _rpm(self, dupes=(), history=("42 1",), journal=(), frontend="dnf", running=()):
return _other(
("rpm-frontend", (frontend,)),
("rpm-installonly", self.INSTALLONLY),
("rpm-dupes", dupes),
("yum-journal", journal),
("dnf-history", history),
running=running,
)
def test_a_clean_rpm_database_is_not_interrupted(self):
status = parse_host_status(self._rpm())
assert status["package_manager_interrupted"] is False
assert status["interrupted_packages"] == []
assert status["package_manager"] == "dnf"
def test_two_versions_of_a_package_left_by_a_cut_off_transaction(self):
status = parse_host_status(self._rpm(dupes=self.CUT_OFF))
assert status["package_manager_interrupted"] is True
assert status["interrupted_packages"] == [
{"name": "glibc", "state": "duplicate"},
{"name": "openssl-libs", "state": "duplicate"},
]
def test_kernels_are_installed_side_by_side_on_purpose(self):
"""dnf's installonly packages: several versions are the rule, not a fault."""
status = parse_host_status(self._rpm(dupes=("kernel-uek-core x86_64",)))
assert status["package_manager_interrupted"] is False
def test_a_package_duplicated_in_two_arches_is_named_once(self):
status = parse_host_status(self._rpm(dupes=("glibc i686", "glibc x86_64")))
assert status["interrupted_packages"] == [{"name": "glibc", "state": "duplicate"}]
def test_the_last_transaction_never_finished(self):
status = parse_host_status(self._rpm(history=("43 0",)))
assert status["package_manager_interrupted"] is True
assert status["interrupted_packages"] == []
@pytest.mark.parametrize("history", [("44 1",), ("45 2",), ()])
def test_a_finished_or_unreadable_history_says_nothing(self, history):
"""A failed transaction (ERROR) did end; an unreadable history is no answer."""
assert parse_host_status(self._rpm(history=history))["package_manager_interrupted"] is False
def test_yum_left_its_transaction_journal(self):
journal = ("transaction-all.2026-11-01.10:12.33", "transaction-done.2026-11-01.10:12.33")
status = parse_host_status(self._rpm(frontend="yum", history=(), journal=journal))
assert status["package_manager_interrupted"] is True
assert status["package_manager"] == "yum"
def test_a_journal_yum_gave_up_on_is_none(self):
"""yum-complete-transaction skips a journal it renamed ``.disabled``."""
journal = ("transaction-all.2026-11-01.10:12.33.disabled",)
status = parse_host_status(self._rpm(frontend="yum", history=(), journal=journal))
assert status["package_manager_interrupted"] is False
@pytest.mark.parametrize("name", ["rpm", "dnf", "dnf5", "yum", "dnf-automatic", "packagekitd"])
def test_a_running_transaction_makes_it_unknown(self, name):
status = parse_host_status(self._rpm(dupes=self.CUT_OFF, running=(name,)))
assert status["package_manager_interrupted"] is None
assert status["interrupted_packages"] == []
assert status["package_manager"] == "dnf"
def test_another_managers_run_does_not(self):
status = parse_host_status(self._rpm(dupes=self.CUT_OFF, running=("opkg",)))
assert status["package_manager_interrupted"] is True
def test_the_command_asks_rpm_only_where_dnf_or_yum_exists(self):
assert "command -v rpm" in HOST_STATUS_COMMAND
assert "rpm -qa --qf" in HOST_STATUS_COMMAND
assert f"file:{DNF_HISTORY}?mode=ro" in HOST_STATUS_COMMAND
def test_it_never_repairs(self):
for word in ("distro-sync", "--cleandupes", "remove", "yum-complete-transaction", "rpm -e"):
assert word not in HOST_STATUS_COMMAND
class TestInterruptedApk:
"""Alpine and OpenWrt 25.12: apk marks a package whose files or scripts failed
in ``/lib/apk/db/installed`` (``f:`` with ``f`` broken files, ``s`` a broken
script; ``x`` xattrs and ``S`` the checksum kind are no fault), and ``apk fix``
with no arguments is what picks exactly those (apk-tools ``app_fix.c``).
netOrk only reports it (NetOrk/netork#851). Shaped after apk-tools'
``database.c``; to be checked against a real OpenWrt 25.12 host."""
def _apk(self, audit=(), running=()):
return _other(("apk-audit", audit), running=running)
def test_a_clean_database_is_not_interrupted(self):
status = parse_host_status(self._apk())
assert status["package_manager_interrupted"] is False
assert status["package_manager"] == "apk"
@pytest.mark.parametrize(
("flags", "states"),
[
("s", ["broken-script"]),
("f", ["broken-files"]),
("fsS", ["broken-files", "broken-script"]),
],
)
def test_a_package_apk_marked_broken(self, flags, states):
status = parse_host_status(self._apk(audit=(f"{flags} busybox",)))
assert status["package_manager_interrupted"] is True
assert status["interrupted_packages"] == [
{"name": "busybox", "state": state} for state in states
]
def test_a_running_apk_makes_it_unknown(self):
status = parse_host_status(self._apk(audit=("s busybox",), running=("apk",)))
assert status["package_manager_interrupted"] is None
def test_the_command_reads_the_installed_database(self):
assert "/lib/apk/db/installed" in HOST_STATUS_COMMAND
assert "apk fix" not in HOST_STATUS_COMMAND
class TestInterruptedOpkg:
"""OpenWrt before 25.12: opkg writes ``Status: <want> <flags> <state>`` per
package to ``/usr/lib/opkg/status``; a package whose run was cut off or whose
postinst failed stays ``unpacked``, and ``opkg configure`` finishes every
unpacked package (opkg-lede ``opkg_cmd.c``, ``pkg.c``). netOrk's update run
does that first (NetOrk/netork#851). To be checked against a real router."""
def _opkg(self, audit=(), running=()):
return _other(("opkg-audit", audit), running=running)
def test_a_clean_status_file_is_not_interrupted(self):
status = parse_host_status(self._opkg())
assert status["package_manager_interrupted"] is False
assert status["package_manager"] == "opkg"
@pytest.mark.parametrize(
("line", "state"),
[
("user unpacked luci-app-sqm", "unpacked"),
("ok half-configured luci-app-sqm", "half-configured"),
("ok half-installed luci-app-sqm", "half-installed"),
("reinstreq,user installed luci-app-sqm", "reinstall-required"),
],
)
def test_every_state_an_interrupted_run_leaves(self, line, state):
status = parse_host_status(self._opkg(audit=(line,)))
assert status["package_manager_interrupted"] is True
assert status["interrupted_packages"] == [{"name": "luci-app-sqm", "state": state}]
def test_a_running_opkg_makes_it_unknown(self):
status = parse_host_status(self._opkg(audit=("ok unpacked x",), running=("opkg",)))
assert status["package_manager_interrupted"] is None
def test_the_command_reads_the_status_file_and_never_configures(self):
assert "/usr/lib/opkg/status" in HOST_STATUS_COMMAND
assert "opkg configure" not in HOST_STATUS_COMMAND
class TestPackageManager:
def test_dpkg_says_so(self):
assert parse_host_status(_wire(dpkg=()))["package_manager"] == "dpkg"
def test_a_busy_dpkg_still_says_which(self):
status = parse_host_status(_wire(dpkg=(), busy=True))
assert (status["package_manager"], status["package_manager_interrupted"]) == (
"dpkg",
None,
)
def test_no_package_manager_read_is_none(self):
status = parse_host_status(_wire())
assert status["package_manager"] is None
assert status["package_manager_interrupted"] is None
#: An rpm that answers the two queries the command asks, as on an EL host whose
#: glibc upgrade was cut off; and a dnf that only has to exist.
_RPM_STUB = """#!/bin/sh
case "$*" in
*--whatprovides*) printf '%s\\n' kernel-core kernel-core 'no package provides kernel-PAE' ;;
-qa*) printf '%s\\n' 'glibc x86_64' 'kernel-core x86_64' 'gpg-pubkey (none)' \\
'glibc x86_64' 'kernel-core x86_64' 'gpg-pubkey (none)' 'bash x86_64' ;;
esac
"""
#: /lib/apk/db/installed shaped after apk-tools' database.c: busybox's script failed.
_APK_DB = """C:Q1abc=
P:alpine-baselayout
V:3.6.8-r1
A:x86_64
F:etc
R:fstab
C:Q1def=
P:busybox
V:1.37.0-r12
A:x86_64
f:s
F:bin
R:busybox
C:Q1ghi=
P:musl
V:1.2.5-r9
f:S
"""
#: /usr/lib/opkg/status as opkg writes it: luci-app-sqm's postinst never ran.
_OPKG_STATUS = """Package: libc
Version: 1.2.4-4
Depends: libgcc1
Status: install hold installed
Architecture: mipsel_24kc
Installed-Time: 1717000000
Package: luci-app-sqm
Version: 1.5.3-r1
Depends: libc, sqm-scripts
Status: install user unpacked
Architecture: all
Installed-Time: 1730000000
"""
def _stubs(tmp_path, *names: str, rpm: bool = False) -> dict:
bin_dir = tmp_path / "bin"
bin_dir.mkdir()
for name in names:
stub = bin_dir / name
stub.write_text(_RPM_STUB if name == "rpm" else "#!/bin/sh\nexit 0\n")
stub.chmod(0o755)
return {**os.environ, "PATH": f"{bin_dir}:/usr/bin:/bin"}
def _run(command: str, env: dict, shell: tuple[str, ...]) -> str:
report = f"printf '%s%s\\n' HSTAT_ BEGIN; {command} printf '%s%s\\n' HSTAT_ END"
program = shutil.which(shell[0]) or shell[0]
return subprocess.run(
[program, *shell[1:], report], capture_output=True, text=True, env=env, timeout=30
).stdout
_SHELLS = [("sh", "-c")] + ([("busybox", "sh", "-c")] if shutil.which("busybox") else [])
def _skip_busybox_rpm(shell: tuple[str, ...]) -> None:
"""A BusyBox built with its own ``rpm`` applet runs that before any stand-in."""
if shell[0] == "busybox":
applets = subprocess.run(["busybox", "--list"], capture_output=True, text=True).stdout
if "rpm" in applets.split():
pytest.skip("this busybox has an rpm applet of its own")
@pytest.mark.parametrize("shell", _SHELLS)
class TestThePackageManagerCommandOnARealShell:
"""The command part run for real, with stand-in programs and files."""
def test_rpm_duplicates_without_the_kernel_and_the_keys(self, tmp_path, shell):
_skip_busybox_rpm(shell)
env = _stubs(tmp_path, "rpm", "dnf")
out = _run(PACKAGE_MANAGERS_COMMAND, env, shell)
status = parse_host_status(out)
assert status["package_manager"] == "dnf"
assert status["interrupted_packages"] == [{"name": "glibc", "state": "duplicate"}]
def test_dnfs_last_transaction_that_never_ended(self, tmp_path, shell):
_skip_busybox_rpm(shell)
history = tmp_path / "history.sqlite"
with sqlite3.connect(history) as db:
db.execute("CREATE TABLE trans (id INTEGER PRIMARY KEY, state INTEGER NOT NULL)")
db.executemany("INSERT INTO trans VALUES (?, ?)", [(41, 1), (42, 0)])
env = _stubs(tmp_path, "rpm", "dnf")
out = _run(PACKAGE_MANAGERS_COMMAND.replace(DNF_HISTORY, str(history)), env, shell)
assert "[dnf-history]\n42 0\n" in out
def test_apk_names_the_package_it_marked(self, tmp_path, shell):
installed = tmp_path / "installed"
installed.write_text(_APK_DB)
env = _stubs(tmp_path, "apk")
out = _run(PACKAGE_MANAGERS_COMMAND.replace(APK_INSTALLED, str(installed)), env, shell)
status = parse_host_status(out)
assert status["package_manager"] == "apk"
assert status["interrupted_packages"] == [{"name": "busybox", "state": "broken-script"}]
def test_opkg_names_the_unpacked_package(self, tmp_path, shell):
status_file = tmp_path / "status"
status_file.write_text(_OPKG_STATUS)
env = _stubs(tmp_path, "opkg")
out = _run(PACKAGE_MANAGERS_COMMAND.replace(OPKG_STATUS, str(status_file)), env, shell)
status = parse_host_status(out)
assert status["package_manager"] == "opkg"
assert status["interrupted_packages"] == [{"name": "luci-app-sqm", "state": "unpacked"}]
def test_a_host_with_none_of_them_reads_none(self, tmp_path, shell):
env = _stubs(tmp_path)
env["PATH"] = str(tmp_path / "bin")
out = _run(PACKAGE_MANAGERS_COMMAND, env, shell)
assert parse_host_status(out)["package_manager"] is None
def test_the_running_processes_come_from_proc(self, tmp_path, shell):
"""The test runner's own process is there under its own name; no manager is."""
out = _run(PACKAGE_MANAGERS_COMMAND, _stubs(tmp_path), shell)
assert "[pm-running]\n" in out
class TestTheWholeCommand:
def test_it_runs_on_busybox(self):
"""OpenWrt's and Alpine's shell: still one intact report."""
if not shutil.which("busybox"):
pytest.skip("no busybox here")
out = subprocess.run(
["busybox", "sh", "-c", HOST_STATUS_COMMAND], capture_output=True, text=True, timeout=30
).stdout
assert parse_host_status(out)["filesystems"]
def test_the_package_managers_are_part_of_it(self):
assert PACKAGE_MANAGERS_COMMAND in HOST_STATUS_COMMAND
+358 -1
View File
@@ -23,6 +23,8 @@ from napalm_device_types.local_accounts import (
from napalm_device_types.models import FixStepOutcomeDict
ALICE_KEY = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAlice alice@laptop"
#: The key Alice had before, on the laptop she lost.
OLD_KEY = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOld alice@lost-laptop"
OWN_KEY = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQOwn own@desktop"
HASH = "$6$salt$" + "x" * 86
#: Shell syntax a command for root must not hold: each is one fixed command.
@@ -61,6 +63,10 @@ class _Host:
self.ran: list[tuple[str, bool]] = []
self.staged = 0
def ran_words(self, first: str) -> list[str]:
"""The words of the last command run whose first word is *first*."""
return next(w for w in (shlex.split(c) for c, _ in reversed(self.ran)) if w[0] == first)
def is_admin(self, name: str) -> bool:
rule = self.files.get(sudoers_rule_path(_Driver.SUDOERS_DIR, name))
return name in self.other_rules or (self.includes_dropins and rule is not None)
@@ -200,6 +206,11 @@ class _Host:
self.accounts[args[0]].groups.remove(args[1])
return CommandResult("", "", 0)
def _clearpw(self, args: list[str], privileged: bool) -> CommandResult:
assert privileged
self.accounts[args[0]].password = None
return CommandResult("", "", 0)
class _Driver(LocalAccountsMixin):
ACCOUNT_SHELL = "/bin/sh"
@@ -238,6 +249,9 @@ class _Driver(LocalAccountsMixin):
def _account_leave_group(self, name: str, group: str, groups: list[str]) -> list[str]:
return ["leavegroup", name, group]
def _account_clear_password(self, name: str) -> list[list[str]]:
return [["clearpw", name]]
@pytest.fixture
def host() -> _Host:
@@ -362,7 +376,9 @@ def test_a_new_account_gets_the_key_and_the_sudo_rule(host: _Host, driver: _Driv
def test_every_command_root_runs_is_one_fixed_command(host: _Host, driver: _Driver) -> None:
driver.add_local_account("alice", ALICE_KEY, password_hash=HASH, admin=True)
driver.add_local_account("alice", OLD_KEY, password_hash=HASH, admin=True)
driver.add_local_account("alice", ALICE_KEY, replaces=[OLD_KEY], clear_password=True)
driver.remove_local_account_keys("alice", [ALICE_KEY])
driver.lock_local_account("alice")
for command in _root_commands(host):
@@ -690,3 +706,344 @@ def test_a_key_file_the_host_adds_is_read_too(host: _Host) -> None:
[account] = _ExtraFile(host).get_local_accounts(["alice"])
assert account["authorized_keys"] == [OWN_KEY, ALICE_KEY]
# -- replacing a key (NetOrk/netork#2013) ----------------------------------------------
def _alice_keys(host: _Host) -> str:
return host.files.get("/home/alice/.ssh/authorized_keys", "")
def test_a_replaced_key_goes_and_the_persons_own_keys_stay(host: _Host, driver: _Driver) -> None:
_add_alice(host, keys=f"{OLD_KEY}\n{OWN_KEY}\n")
result = driver.add_local_account("alice", ALICE_KEY, replaces=[OLD_KEY])
assert result["success"], result["output"]
assert [s["step"] for s in result["steps"]] == ["account", "sudo", "keys", "old keys"]
assert _alice_keys(host) == f"{OWN_KEY}\n{ALICE_KEY}\n"
account = result["account"]
assert account is not None and account["authorized_keys"] == [OWN_KEY, ALICE_KEY]
def test_a_replaced_key_goes_from_every_key_file_with_any_options_or_comment(
host: _Host, driver: _Driver
) -> None:
# The same key with options in front and another comment is the same key.
blob = OLD_KEY.rsplit(" ", 1)[0]
_add_alice(host, keys=f'from="10.0.0.0/8" {blob} renamed\n# the laptop\n')
host.files["/home/alice/.ssh/authorized_keys2"] = f"{OWN_KEY}\n{blob}\n"
result = driver.add_local_account("alice", ALICE_KEY, replaces=[OLD_KEY])
assert result["success"], result["output"]
assert _alice_keys(host) == f"# the laptop\n{ALICE_KEY}\n"
assert host.files["/home/alice/.ssh/authorized_keys2"] == f"{OWN_KEY}\n"
assert not [p for p in host.files if p.startswith("/tmp/")]
def test_a_replaced_key_that_is_not_there_writes_nothing(host: _Host, driver: _Driver) -> None:
_add_alice(host, keys=f"{ALICE_KEY}\n")
result = driver.add_local_account("alice", ALICE_KEY, replaces=[OLD_KEY])
assert result["success"], result["output"]
assert host.staged == 0
assert "none of the keys" in result["steps"][-1]["output"]
def test_the_new_key_is_never_taken_for_a_replaced_one(host: _Host, driver: _Driver) -> None:
_add_alice(host, keys=f"{OLD_KEY}\n")
result = driver.add_local_account("alice", ALICE_KEY, replaces=[OLD_KEY, ALICE_KEY])
assert result["success"], result["output"]
assert _alice_keys(host) == f"{ALICE_KEY}\n"
def test_without_replaces_no_key_goes(host: _Host, driver: _Driver) -> None:
_add_alice(host, keys=f"{OLD_KEY}\n")
result = driver.add_local_account("alice", ALICE_KEY)
assert [s["step"] for s in result["steps"]] == ["account", "sudo", "keys"]
assert _alice_keys(host) == f"{OLD_KEY}\n{ALICE_KEY}\n"
@pytest.mark.parametrize("replaces", [["not a key"], [OLD_KEY, ""], OLD_KEY])
def test_a_replaced_key_that_is_none_is_refused(
host: _Host, driver: _Driver, replaces: list[str]
) -> None:
with pytest.raises(ValueError, match="key"):
driver.add_local_account("alice", ALICE_KEY, replaces=replaces)
assert _root_commands(host) == []
def test_the_old_key_goes_even_when_the_new_one_could_not_be_added(host: _Host) -> None:
class _AddFails(_Driver):
def _account_add_key(self, name: str, home: str, public_key: str) -> FixStepOutcomeDict:
return {"ok": False, "output": "disk full"}
_add_alice(host, keys=f"{OLD_KEY}\n{OWN_KEY}\n")
result = _AddFails(host).add_local_account("alice", ALICE_KEY, replaces=[OLD_KEY])
assert not result["success"]
assert next(s for s in result["steps"] if s["step"] == "old keys")["ok"]
assert _alice_keys(host) == f"{OWN_KEY}\n"
@pytest.mark.parametrize(
("failing", "ignored", "said"),
[
({"as cat"}, set(), "cannot read"),
({"as install"}, set(), "authorized_keys"),
(set(), {"as install"}, "still has"),
],
)
def test_a_replaced_key_that_did_not_go_says_so(
host: _Host, driver: _Driver, failing: set[str], ignored: set[str], said: str
) -> None:
_add_alice(host, keys=f"{OLD_KEY}\n{ALICE_KEY}\n")
host.failing |= failing
host.ignored |= ignored
result = driver.add_local_account("alice", ALICE_KEY, replaces=[OLD_KEY])
old = next(s for s in result["steps"] if s["step"] == "old keys")
assert not old["ok"] and said in old["output"]
assert not result["success"]
# -- removing the password (NetOrk/netork#2013) ----------------------------------------
def test_clearing_the_password_leaves_a_key_only_account_without_sudo(
host: _Host, driver: _Driver
) -> None:
_add_alice(host, keys=f"{ALICE_KEY}\n")
host.files["/etc/sudoers.d/80-account-alice"] = sudoers_rule("alice")
result = driver.add_local_account("alice", ALICE_KEY, clear_password=True)
assert result["success"], result["output"]
assert [s["step"] for s in result["steps"]] == ["account", "sudo", "password", "keys"]
a = host.accounts["alice"]
assert a.password is None and not a.locked and a.groups == ["alice"]
assert "/etc/sudoers.d/80-account-alice" not in host.files
assert _alice_keys(host) == f"{ALICE_KEY}\n"
def test_a_new_account_without_a_password_gets_it_cleared_too(host: _Host, driver: _Driver) -> None:
result = driver.add_local_account("alice", ALICE_KEY, clear_password=True)
assert result["success"], result["output"]
assert ["clearpw", "alice"] == host.ran_words("clearpw")
def test_without_clear_password_the_password_stays(host: _Host, driver: _Driver) -> None:
_add_alice(host)
result = driver.add_local_account("alice", ALICE_KEY)
assert "password" not in [s["step"] for s in result["steps"]]
assert host.accounts["alice"].password == HASH
@pytest.mark.parametrize(
"kwargs",
[{"password_hash": HASH}, {"admin": True}],
)
def test_clearing_the_password_refuses_a_password_or_sudo(
host: _Host, driver: _Driver, kwargs: dict[str, object]
) -> None:
# The sudo rule asks for the password: an account without one cannot use it.
with pytest.raises(ValueError, match="password"):
driver.add_local_account("alice", ALICE_KEY, clear_password=True, **kwargs) # type: ignore[arg-type]
assert _root_commands(host) == []
def test_a_password_that_cannot_be_cleared_fails_its_step_only(
host: _Host, driver: _Driver
) -> None:
_add_alice(host)
host.failing.add("clearpw")
result = driver.add_local_account("alice", ALICE_KEY, clear_password=True)
assert not result["success"]
password = next(s for s in result["steps"] if s["step"] == "password")
assert not password["ok"] and "clearpw: failed" in password["output"]
# sudo and the key do not wait for it.
assert next(s for s in result["steps"] if s["step"] == "sudo")["ok"]
assert _alice_keys(host) == f"{ALICE_KEY}\n"
def test_a_driver_that_cannot_clear_a_password_says_so(host: _Host) -> None:
class _NoClear(_Driver):
_account_clear_password = LocalAccountsMixin._account_clear_password
_add_alice(host)
result = _NoClear(host).add_local_account("alice", ALICE_KEY, clear_password=True)
password = next(s for s in result["steps"] if s["step"] == "password")
assert not password["ok"] and "cannot remove" in password["output"]
assert host.accounts["alice"].password == HASH
# -- removing keys without a new one (NetOrk/netork#2013) -------------------------------
def test_removing_keys_takes_only_those_and_leaves_the_account_open(
host: _Host, driver: _Driver
) -> None:
_add_alice(host, keys=f"{ALICE_KEY}\n{OWN_KEY}\n")
host.files["/etc/sudoers.d/80-account-alice"] = sudoers_rule("alice")
result = driver.remove_local_account_keys("alice", [ALICE_KEY])
assert result["success"], result["output"]
assert [s["step"] for s in result["steps"]] == ["keys"]
assert _alice_keys(host) == f"{OWN_KEY}\n"
a = host.accounts["alice"]
assert not a.locked and a.password == HASH and a.groups == ["alice", "wheel"]
assert "/etc/sudoers.d/80-account-alice" in host.files
account = result["account"]
assert account is not None and account["authorized_keys"] == [OWN_KEY]
def test_removing_the_last_key_leaves_an_empty_key_file(host: _Host, driver: _Driver) -> None:
_add_alice(host, keys=f"{ALICE_KEY}\n")
result = driver.remove_local_account_keys("alice", [ALICE_KEY])
assert result["success"], result["output"]
assert _alice_keys(host) == ""
assert not host.accounts["alice"].locked
def test_removing_keys_changes_nothing_as_root(host: _Host, driver: _Driver) -> None:
# Root only asks whether a key file is there, and reads the account back
# (its lock, sudo -l); the files are written as the person.
_add_alice(host, keys=f"{ALICE_KEY}\n")
driver.remove_local_account_keys("alice", [ALICE_KEY])
for command in _root_commands(host):
words = shlex.split(command)
assert words[0] in {"test", "islocked", "sudo"}, command
if words[0] == "sudo":
assert words[:3] in (["sudo", "-n", "-u"], ["sudo", "-l", "-U"]), command
def test_removing_keys_of_an_account_that_is_not_there_fails(driver: _Driver) -> None:
result = driver.remove_local_account_keys("bob", [ALICE_KEY])
assert not result["success"]
assert "no account" in result["output"]
assert result["account"] is None
@pytest.mark.parametrize("keys", [[], ["not a key"], ALICE_KEY])
def test_removing_keys_refuses_what_is_no_list_of_keys(
host: _Host, driver: _Driver, keys: list[str]
) -> None:
_add_alice(host, keys=f"{ALICE_KEY}\n")
with pytest.raises(ValueError, match="key"):
driver.remove_local_account_keys("alice", keys)
assert _alice_keys(host) == f"{ALICE_KEY}\n"
@pytest.mark.parametrize(("name", "said"), [("netadmin", "logs in"), ("daemon", "system")])
def test_removing_keys_never_touches_the_login_user_or_a_system_account(
host: _Host, driver: _Driver, name: str, said: str
) -> None:
with pytest.raises(ValueError, match=said):
driver.remove_local_account_keys(name, [ALICE_KEY])
assert _root_commands(host) == []
def test_a_driver_with_its_own_key_store_drops_keys_there(host: _Host) -> None:
class _Store(_OwnKeyStore):
def _account_drop_keys(self, name: str, home: str, keys: list[str]) -> FixStepOutcomeDict:
gone = {k.split()[1] for k in keys}
self.store[name] = [k for k in self.store.get(name, []) if k.split()[2] not in gone]
return {"ok": True, "output": "dropped some"}
driver = _Store(host)
driver.add_local_account("alice", OLD_KEY)
driver.add_local_account("alice", ALICE_KEY, replaces=[OLD_KEY])
assert driver.store == {"alice": [f"/home/alice {ALICE_KEY}"]}
removed = driver.remove_local_account_keys("alice", [ALICE_KEY])
assert removed["success"], removed["output"]
assert driver.store == {"alice": []}
# -- removing keys and the password at once (NetOrk/netork#2033) ------------------------
def test_removing_keys_with_clear_password_takes_the_password_and_sudo_too(
host: _Host, driver: _Driver
) -> None:
"""A person who deleted their key: the password netOrk no longer knows goes too, and
sudo, which asks for it. The account stays open, its other keys stay."""
_add_alice(host, keys=f"{ALICE_KEY}\n{OWN_KEY}\n")
host.files["/etc/sudoers.d/80-account-alice"] = sudoers_rule("alice")
result = driver.remove_local_account_keys("alice", [ALICE_KEY], clear_password=True)
assert result["success"], result["output"]
assert [s["step"] for s in result["steps"]] == ["keys", "password", "sudo"]
assert _alice_keys(host) == f"{OWN_KEY}\n"
a = host.accounts["alice"]
assert a.password is None and not a.locked and a.groups == ["alice"]
assert "/etc/sudoers.d/80-account-alice" not in host.files
account = result["account"]
assert account is not None and not account["admin"] and not account["locked"]
def test_removing_keys_with_clear_password_runs_each_step_when_one_failed(
host: _Host, driver: _Driver
) -> None:
_add_alice(host, keys=f"{ALICE_KEY}\n")
host.files["/etc/sudoers.d/80-account-alice"] = sudoers_rule("alice")
host.failing.add("clearpw")
result = driver.remove_local_account_keys("alice", [ALICE_KEY], clear_password=True)
assert not result["success"]
steps = {s["step"]: s for s in result["steps"]}
assert steps["keys"]["ok"] and not steps["password"]["ok"]
# sudo does not wait for the password: without a known password it has to go anyway.
assert steps["sudo"]["ok"]
assert "/etc/sudoers.d/80-account-alice" not in host.files
assert _alice_keys(host) == ""
def test_removing_keys_with_a_driver_that_cannot_clear_a_password_says_so(host: _Host) -> None:
class _NoClear(_Driver):
_account_clear_password = LocalAccountsMixin._account_clear_password
_add_alice(host, keys=f"{ALICE_KEY}\n")
result = _NoClear(host).remove_local_account_keys("alice", [ALICE_KEY], clear_password=True)
password = next(s for s in result["steps"] if s["step"] == "password")
assert not password["ok"] and "cannot remove" in password["output"]
assert not result["success"]
def test_removing_keys_of_an_account_that_is_not_there_clears_nothing(driver: _Driver) -> None:
result = driver.remove_local_account_keys("bob", [ALICE_KEY], clear_password=True)
assert not result["success"] and "no account" in result["output"]
assert [s["step"] for s in result["steps"]] == ["keys"]
def test_clear_password_is_a_keyword(driver: _Driver) -> None:
with pytest.raises(TypeError):
driver.remove_local_account_keys("alice", [ALICE_KEY], True) # type: ignore[misc]
+158
View File
@@ -0,0 +1,158 @@
"""Where a host's logs go beyond it: rsyslog's forwarding, its file inputs, and
whether it gets the journal (NetOrk/netork#2053)."""
from __future__ import annotations
import pytest
from napalm_device_types.log_forwarding import journal_to_syslog, parse_rsyslog
DEBIAN = """\
module(load="imuxsock")
module(load="imklog")
$FileOwner root
$IncludeConfig /etc/rsyslog.d/*.conf
*.*;auth,authpriv.none\t\t-/var/log/syslog
auth,authpriv.*\t\t\t/var/log/auth.log
*.emerg\t\t\t\t:omusrmsg:*
"""
GRAYLOG = "*.* @graylog.example.net:1514;RSYSLOG_SyslogProtocol23Format\n"
def _rules(text: str) -> list[dict]:
return parse_rsyslog(text)["rules"]
class TestRules:
def test_a_debian_host_forwards_nothing(self):
assert _rules(DEBIAN) == []
def test_netorks_own_graylog_rule_forwards_everything_over_udp(self):
assert _rules(DEBIAN + GRAYLOG) == [
{
"protocol": "udp",
"target": "graylog.example.net",
"port": 1514,
"selector": "*.*",
"complete": True,
"ruleset": None,
},
]
def test_two_ats_are_tcp_and_a_missing_port_stays_unknown(self):
(rule,) = _rules("*.* @@logs.lan\n")
assert (rule["protocol"], rule["target"], rule["port"]) == ("tcp", "logs.lan", None)
def test_an_ipv6_target_keeps_its_brackets_off(self):
(rule,) = _rules("*.* @[2001:db8::5]:514\n")
assert (rule["target"], rule["port"]) == ("2001:db8::5", 514)
def test_a_narrower_selector_is_partial_and_says_which(self):
(rule,) = _rules("authpriv.* @10.0.0.5\n")
assert (rule["selector"], rule["complete"]) == ("authpriv.*", False)
def test_excluding_a_facility_is_partial_too(self):
(rule,) = _rules("*.*;mail.none @10.0.0.5\n")
assert rule["complete"] is False
def test_an_ampersand_line_forwards_what_the_line_before_selected(self):
rules = _rules("kern.* /var/log/kern.log\n& @10.0.0.5:514\n")
assert [(r["selector"], r["target"]) for r in rules] == [("kern.*", "10.0.0.5")]
def test_relp_is_its_own_protocol(self):
(rule,) = _rules("*.* :omrelp:10.0.0.6:2514\n")
assert (rule["protocol"], rule["target"], rule["port"]) == ("relp", "10.0.0.6", 2514)
def test_an_action_over_several_lines_is_read_whole(self):
text = '*.* action(type="omfwd"\n target="10.0.0.7" port="6514"\n protocol="tcp")\n'
(rule,) = _rules(text)
assert (rule["protocol"], rule["target"], rule["port"], rule["complete"]) == (
"tcp",
"10.0.0.7",
6514,
True,
)
def test_an_action_on_its_own_takes_every_message(self):
(rule,) = _rules('action(type="omfwd" Target="10.0.0.7")\n')
assert (rule["selector"], rule["complete"], rule["protocol"]) == ("*.*", True, "udp")
def test_an_action_under_a_condition_is_partial_and_names_it(self):
(rule,) = _rules(
'if $programname == \'nginx\' then action(type="omfwd" target="10.0.0.8")\n'
)
assert rule["complete"] is False
assert rule["selector"] == "$programname == 'nginx'"
def test_a_condition_whose_action_is_on_the_next_line_still_holds(self):
(rule,) = _rules('if $msg contains "x" then\n action(type="omfwd" target="10.0.0.14")\n')
assert (rule["selector"], rule["complete"]) == ('$msg contains "x"', False)
def test_a_block_under_a_condition_makes_each_of_its_actions_partial(self):
text = 'if $syslogseverity <= 4 then {\n action(type="omfwd" target="a")\n action(type="omrelp" target="b")\n}\n*.* @c\n'
rules = _rules(text)
assert [(r["target"], r["complete"]) for r in rules] == [
("a", False),
("b", False),
("c", True),
]
def test_a_property_filter_is_a_partial_selector(self):
(rule,) = _rules(':programname, isequal, "sshd" @10.0.0.9\n')
assert rule["complete"] is False
assert rule["selector"] == ':programname, isequal, "sshd"'
def test_an_action_in_a_ruleset_belongs_to_that_ruleset(self):
text = 'ruleset(name="remote") {\n action(type="omfwd" target="10.0.0.10" port="514")\n}\n'
(rule,) = _rules(text)
assert (rule["ruleset"], rule["complete"]) == ("remote", True)
def test_comments_are_no_rules(self):
assert _rules("# *.* @10.0.0.11\n*.* /var/log/all.log # @10.0.0.12\n") == []
def test_a_hash_inside_quotes_is_no_comment(self):
(rule,) = _rules('action(type="omfwd" target="10.0.0.13" template="a#b")\n')
assert rule["target"] == "10.0.0.13"
def test_a_local_action_is_no_forwarding(self):
assert _rules('action(type="omfile" file="/var/log/x.log")\n*.* /var/log/y.log\n') == []
class TestInputs:
def test_an_imfile_input_names_its_file_and_ruleset(self):
text = 'input(type="imfile" File="/var/log/nginx/*.log" Tag="nginx" Ruleset="remote")\n'
assert parse_rsyslog(text)["file_inputs"] == [
{"path": "/var/log/nginx/*.log", "ruleset": "remote"}
]
def test_a_legacy_imfile_input_goes_to_the_default_ruleset(self):
text = "$ModLoad imfile\n$InputFileName /var/log/app.log\n$InputFileTag app:\n$InputRunFileMonitor\n"
assert parse_rsyslog(text)["file_inputs"] == [{"path": "/var/log/app.log", "ruleset": None}]
class TestModules:
def test_imjournal_reads_the_journal(self):
assert parse_rsyslog('module(load="imjournal")\n')["imjournal"] is True
def test_legacy_modload_counts(self):
modules = parse_rsyslog("$ModLoad imuxsock\n")
assert (modules["imuxsock"], modules["imjournal"]) == (True, False)
class TestJournalToSyslog:
"""``ForwardToSyslog``: the last setting wins; with none, the shipped default
the distribution prints commented out in its ``journald.conf``."""
@pytest.mark.parametrize(
("lines", "expected"),
[
(["#ForwardToSyslog=yes"], True),
(["#ForwardToSyslog=no"], False),
(["#ForwardToSyslog=yes", "ForwardToSyslog=no"], False),
(["ForwardToSyslog=no", "ForwardToSyslog = yes"], True),
([], None),
],
)
def test_the_last_setting_wins_else_the_shipped_default(self, lines, expected):
assert journal_to_syslog(lines) is expected
+394
View File
@@ -0,0 +1,394 @@
"""get_log_sources: which logs a host writes, and who writes each of them.
Before anything can watch a host's logs -- a person in a log window, CrowdSec,
a syslog forwarder -- something has to say which logs there are. Reading that
is the same on every Linux host: journald names the units it holds entries for,
``/proc/<pid>/fd`` shows the log files running processes have open, and
``/proc/<pid>/cgroup`` says which systemd service or container a writer runs in.
So all of it lives here once, and a driver only carries the command across
(NetOrk/netork#2049).
"""
from __future__ import annotations
import shutil
import subprocess
import pytest
from napalm_device_types.log_sources import (
LOG_SOURCES_COMMAND,
LogSourcesMixin,
LogSourcesUnavailable,
parse_log_forwarding,
parse_log_sources,
)
CID = "4f1c" + "0" * 60
OTHER = "9a2b" + "1" * 60
JOURNALD = """nginx.service
ssh.service
postgresql@16-main.service
session-3.scope
init.scope
"""
OPEN = f"""/proc/610/fd/7 /var/log/syslog
/proc/610/fd/8 /var/log/auth.log
/proc/620/fd/9 /var/log/journal/0123abcd/system.journal
/proc/900/fd/5 /var/log/nginx/access.log
/proc/900/fd/6 /var/log/nginx/error.log
/proc/901/fd/5 /var/log/nginx/access.log
/proc/700/fd/4 /var/log/old.log (deleted)
/proc/1100/fd/30 /var/lib/docker/containers/{CID}/{CID}-json.log
/proc/1100/fd/31 /var/lib/docker/containers/{OTHER}/local-logs/container.log
/proc/1400/fd/3 /usr/src/paperless/data/log/paperless.log
/proc/1500/fd/3 /srv/app/log with space.log
/proc/1500/fd/4 /srv/app/Local Storage/leveldb/000004.log
/proc/1500/fd/5 /var/lib/rocksdb/012345.log
/proc/300/fd/11 /var/log/nginx/access.log
/proc/300/fd/12 /mnt/docroot/data/log/paperless.log
/proc/300/fd/13 socket:[48213]
/proc/300/fd/14 pipe:[48214]
/proc/1600/fd/6 /var/log/host/syslog
"""
#: Each descriptor's ``flags`` from ``/proc/<pid>/fdinfo/<fd>``, octal: the access
#: mode (``& 3``) is 1 or 2 for a writer -- 0102001 is O_WRONLY|O_APPEND|O_LARGEFILE
#: -- and 0 for a reader (0100000, O_RDONLY|O_LARGEFILE). CrowdSec (300) reads
#: nginx's log and a log on a mount; a promtail in a container (1600) reads the
#: host's syslog as its own /var/log/host/syslog. 1500 opens its log O_RDWR.
FLAGS = """/proc/610/fdinfo/7:flags:\t0102001
/proc/610/fdinfo/8:flags:\t0102001
/proc/620/fdinfo/9:flags:\t02100002
/proc/900/fdinfo/5:flags:\t0102001
/proc/900/fdinfo/6:flags:\t0102001
/proc/901/fdinfo/5:flags:\t0102001
/proc/700/fdinfo/4:flags:\t0102001
/proc/1100/fdinfo/30:flags:\t0102001
/proc/1100/fdinfo/31:flags:\t0102001
/proc/1400/fdinfo/3:flags:\t0102001
/proc/1500/fdinfo/3:flags:\t0100002
/proc/1500/fdinfo/4:flags:\t0100002
/proc/1500/fdinfo/5:flags:\t0100002
/proc/300/fdinfo/11:flags:\t0100000
/proc/300/fdinfo/12:flags:\t0100000
/proc/300/fdinfo/13:flags:\t02004002
/proc/300/fdinfo/14:flags:\t02004001
/proc/1600/fdinfo/6:flags:\t0100000
"""
STAT = """/proc/610/fd/7 99000 1762250000
/proc/610/fd/8 4100 1762250100
/proc/900/fd/5 2048 1762250200
/proc/900/fd/6 512 1762250300
/proc/901/fd/5 2048 1762250200
/proc/1100/fd/30 7000 1762250400
/proc/1100/fd/31 300 1762250500
/proc/1400/fd/3 0 1762250600
/proc/300/fd/11 2048 1762250200
/proc/300/fd/12 81920 1762250700
"""
CGROUPS = f"""610 0::/system.slice/rsyslog.service
900 0::/system.slice/nginx.service
901 0::/system.slice/nginx.service
1100 0::/system.slice/docker.service
1400 0::/system.slice/docker-{CID}.scope
1500 0::/user.slice/user-1000.slice/session-3.scope
300 0::/system.slice/crowdsec.service
1600 0::/system.slice/docker-{OTHER}.scope
"""
COMM = """610 rsyslogd
900 nginx
901 nginx
1100 dockerd
1400 python3
1500 app
300 crowdsec
1600 promtail
"""
def _wire(
*,
journald: str | None = JOURNALD,
open_: str = OPEN,
stat: str = STAT,
cgroups: str = CGROUPS,
comm: str = COMM,
flags: str | None = FLAGS,
noise: str = "",
forwarding: str = "",
) -> str:
"""The report as the command prints it, framed."""
parts = [noise, "LOGS_BEGIN\n"]
if journald is not None:
parts += ["[journald]\n", journald]
parts += ["[open]\n", open_, "[stat]\n", stat, "[cgroups]\n", cgroups, "[comm]\n", comm]
if flags is not None:
parts += ["[flags]\n", flags]
parts += [forwarding, "LOGS_END\n"]
return "".join(parts)
def _sources(output: str | None = None) -> list:
return parse_log_sources(_wire() if output is None else output)[1]
def _by_key(sources: list) -> dict:
return {(s["kind"], s["container_id"], s["path"] or s["unit"]): s for s in sources}
class TestParse:
def test_journald_names_each_service_unit_once_without_its_suffix(self) -> None:
journald, sources = parse_log_sources(_wire())
units = [s["unit"] for s in sources if s["kind"] == "journald"]
assert units == ["nginx", "postgresql@16-main", "ssh"]
assert journald is True
def test_a_host_without_journald_says_so(self) -> None:
journald, sources = parse_log_sources(_wire(journald=None))
assert journald is False
assert not [s for s in sources if s["kind"] == "journald"]
def test_an_open_file_names_its_writer_size_and_last_write(self) -> None:
sources = _by_key(_sources())
access = sources[("file", None, "/var/log/nginx/access.log")]
assert access == {
"kind": "file",
"path": "/var/log/nginx/access.log",
"unit": "nginx",
"container_id": None,
"process": "nginx",
"size": 2048,
"modified": "2025-11-04T09:56:40+00:00",
}
def test_a_file_two_processes_hold_is_one_source(self) -> None:
paths = [s["path"] for s in _sources() if s["kind"] == "file"]
assert paths.count("/var/log/nginx/access.log") == 1
def test_a_file_without_a_suffix_under_var_log_counts(self) -> None:
sources = _by_key(_sources())
assert sources[("file", None, "/var/log/syslog")]["unit"] == "rsyslog"
def test_a_file_a_container_writes_carries_the_container_and_its_own_path(self) -> None:
sources = _by_key(_sources())
paperless = sources[("file", CID, "/usr/src/paperless/data/log/paperless.log")]
assert paperless["unit"] is None
assert paperless["process"] == "python3"
assert paperless["size"] == 0
def test_the_engines_own_log_file_is_the_containers_stream(self) -> None:
sources = _by_key(_sources())
json_file = sources[("container", CID, f"/var/lib/docker/containers/{CID}/{CID}-json.log")]
assert json_file["process"] == "dockerd"
assert json_file["size"] == 7000
local = sources[
("container", OTHER, f"/var/lib/docker/containers/{OTHER}/local-logs/container.log")
]
assert local["modified"] == "2025-11-04T10:01:40+00:00"
assert not [s for s in _sources() if "docker" in (s["unit"] or "")]
def test_journal_files_and_deleted_files_are_not_sources(self) -> None:
paths = [s["path"] or "" for s in _sources()]
assert not [p for p in paths if "/var/log/journal/" in p or p.startswith("/var/log/old")]
def test_a_databases_write_ahead_log_is_not_a_source(self) -> None:
# LevelDB and RocksDB name theirs 000004.log; it holds records, not lines.
paths = [s["path"] or "" for s in _sources()]
assert not [p for p in paths if p.endswith(("000004.log", "012345.log"))]
def test_a_path_with_a_space_survives(self) -> None:
sources = _by_key(_sources())
spaced = sources[("file", None, "/srv/app/log with space.log")]
# The writer runs in a login session, not in a service; stat did not answer.
assert spaced["unit"] is None
assert spaced["process"] == "app"
assert spaced["size"] is None
assert spaced["modified"] is None
def test_sources_come_sorted_by_kind_then_container_then_name(self) -> None:
sources = _sources()
order = [(s["kind"], s["container_id"] or "", s["path"] or s["unit"]) for s in sources]
assert order == sorted(order)
def test_noise_before_the_frame_is_ignored(self) -> None:
assert _sources(_wire(noise="Last login: today\r\n\x1b[0m"))
def test_a_host_without_gnu_find_raises_rather_than_reading_as_no_files(self) -> None:
# busybox find has neither -lname nor -printf: no files would read as none.
with pytest.raises(LogSourcesUnavailable, match="find"):
parse_log_sources("LOGS_BEGIN\n[journald]\nnginx.service\n[no-find]\nLOGS_END\n")
def test_a_report_cut_short_raises_rather_than_reading_as_fewer_logs(self) -> None:
cut = _wire().replace("LOGS_END\n", "")
with pytest.raises(ValueError, match="log source report"):
parse_log_sources(cut)
class TestWriters:
"""A file is attributed to the process writing it, not to one reading it
(NetOrk/netork#2092): CrowdSec, Promtail or a ``tail -F`` holding a log is
no producer of it."""
def test_a_reader_with_the_lower_pid_does_not_take_the_writers_file(self) -> None:
access = _by_key(_sources())[("file", None, "/var/log/nginx/access.log")]
assert (access["unit"], access["process"]) == ("nginx", "nginx")
def test_a_file_only_readers_hold_is_listed_without_a_producer(self) -> None:
paperless = _by_key(_sources())[("file", None, "/mnt/docroot/data/log/paperless.log")]
assert paperless == {
"kind": "file",
"path": "/mnt/docroot/data/log/paperless.log",
"unit": None,
"container_id": None,
"process": None,
"size": 81920,
"modified": "2025-11-04T10:05:00+00:00",
}
def test_a_file_open_for_reading_and_writing_is_written(self) -> None:
spaced = _by_key(_sources())[("file", None, "/srv/app/log with space.log")]
assert spaced["process"] == "app"
def test_a_file_only_a_reader_in_a_container_holds_is_left_out(self) -> None:
# Its path is the reader's view of a mount: it names nothing on the host,
# and the container does not write it.
assert not [s for s in _sources() if s["path"] == "/var/log/host/syslog"]
def test_sockets_and_pipes_are_no_sources(self) -> None:
assert not [s for s in _sources() if not (s["path"] or "/").startswith("/")]
def test_a_deleted_file_stays_out_though_it_is_written(self) -> None:
assert not [s for s in _sources() if (s["path"] or "").startswith("/var/log/old")]
def test_a_holder_whose_flags_did_not_answer_is_no_writer(self) -> None:
# The process ended between the listing and the read of its fdinfo.
flags = FLAGS.replace("/proc/900/fdinfo/5:flags:\t0102001\n", "")
flags = flags.replace("/proc/901/fdinfo/5:flags:\t0102001\n", "")
access = _by_key(_sources(_wire(flags=flags)))[("file", None, "/var/log/nginx/access.log")]
assert (access["unit"], access["process"]) == (None, None)
def test_a_report_without_flags_names_the_first_holder_as_before(self) -> None:
# An older command printed no [flags]: who writes cannot be told apart.
access = _by_key(_sources(_wire(flags=None)))[("file", None, "/var/log/nginx/access.log")]
assert (access["unit"], access["process"]) == ("crowdsec", "crowdsec")
#: rsyslog running, forwarding to Graylog, its config prefixed so that no line reads as a section.
FORWARDING = """[rsyslog]
running
[rsyslog-conf]
| module(load="imuxsock")
| [comment-looking line]
|
| *.* @graylog.lan:1514;RSYSLOG_SyslogProtocol23Format
| input(type="imfile" File="/var/log/nginx/*.log" Tag="nginx")
[journald-conf]
#ForwardToSyslog=yes
"""
class TestForwarding:
"""Where the logs go beyond the host, read in the same command (NetOrk/netork#2053)."""
def test_rsyslog_forwarding_and_the_journal_reaching_it(self) -> None:
forwarding = parse_log_forwarding(_wire(forwarding=FORWARDING))
assert forwarding is not None
assert forwarding["rsyslog"] is True
assert forwarding["journal"] is True
assert [(r["target"], r["port"], r["complete"]) for r in forwarding["rules"]] == [
("graylog.lan", 1514, True)
]
assert forwarding["file_inputs"] == [{"path": "/var/log/nginx/*.log", "ruleset": None}]
def test_a_config_line_in_brackets_is_no_section(self) -> None:
forwarding = parse_log_forwarding(_wire(forwarding=FORWARDING))
assert forwarding is not None and forwarding["rules"]
def test_an_rsyslog_that_does_not_run_forwards_nothing(self) -> None:
stopped = FORWARDING.replace("running\n", "")
forwarding = parse_log_forwarding(_wire(forwarding=stopped))
assert forwarding == {"rsyslog": False, "journal": False, "rules": [], "file_inputs": []}
def test_a_report_without_the_sections_says_nothing(self) -> None:
assert parse_log_forwarding(_wire()) is None
def test_the_reading_carries_it(self) -> None:
reading = _Host({True: _wire(forwarding=FORWARDING)}).get_log_sources()
assert reading["forwarding"] is not None
assert reading["forwarding"]["rules"][0]["target"] == "graylog.lan"
class _Host(LogSourcesMixin):
"""Carries the command the way a driver does; *answers* per privilege."""
def __init__(self, answers: dict[bool, str]) -> None:
self.answers = answers
self.calls: list[tuple[str, bool]] = []
def _run_log_sources_command(self, command: str, *, privileged: bool) -> str:
self.calls.append((command, privileged))
return self.answers[privileged]
class TestMixin:
def test_it_reads_as_root_in_one_sh_call(self) -> None:
host = _Host({True: _wire()})
reading = host.get_log_sources()
assert reading["attributed"] is True
command, privileged = host.calls[0]
assert privileged is True
assert command.startswith("sh -c ")
assert len(host.calls) == 1
def test_without_root_it_reads_again_and_says_so(self) -> None:
host = _Host({True: "sudo: a password is required\n", False: _wire()})
reading = host.get_log_sources()
assert reading["attributed"] is False
assert [privileged for _, privileged in host.calls] == [True, False]
assert reading["sources"]
def test_a_host_without_gnu_find_is_not_read_again_unprivileged(self) -> None:
host = _Host({True: "LOGS_BEGIN\n[no-find]\nLOGS_END\n"})
with pytest.raises(LogSourcesUnavailable):
host.get_log_sources()
assert len(host.calls) == 1
def test_a_host_that_answers_neither_way_raises(self) -> None:
host = _Host({True: "", False: "garbage"})
with pytest.raises(ValueError):
host.get_log_sources()
class TestCommand:
def test_it_is_valid_posix_sh(self) -> None:
assert subprocess.run(["sh", "-n", "-c", LOG_SOURCES_COMMAND]).returncode == 0
@pytest.mark.skipif(shutil.which("find") is None, reason="needs find on this host")
def test_here_a_file_held_for_writing_has_a_producer_and_one_held_for_reading_none(
self, tmp_path
) -> None:
written, read = tmp_path / "written.log", tmp_path / "read.log"
read.write_text("one line\n")
with written.open("a") as w, read.open() as r:
out = subprocess.run(
["sh", "-c", LOG_SOURCES_COMMAND], capture_output=True, text=True, timeout=60
).stdout
assert w.writable() and r.readable()
sources = {s["path"]: s for s in _sources(out)}
assert sources[str(written)]["process"] is not None
assert sources[str(read)]["process"] is None
@pytest.mark.skipif(shutil.which("find") is None, reason="needs find on this host")
def test_it_runs_here_and_parses(self, tmp_path) -> None:
log = tmp_path / "held.log"
log.write_text("one line\n")
with log.open() as held:
out = subprocess.run(
["sh", "-c", LOG_SOURCES_COMMAND], capture_output=True, text=True, timeout=60
).stdout
assert held.readable()
# This test's own process holds the file, so it must be among the sources.
mine = [s for s in _sources(out) if s["path"] == str(log)]
assert mine and mine[0]["size"] == len("one line\n")
# Whether rsyslog runs here or not, the report says where logs go.
assert parse_log_forwarding(out) is not None
+51
View File
@@ -0,0 +1,51 @@
"""get_running_backups: the backups running on a device right now.
An application that restarts a device asks first, right before the restart: a
restart aborts a backup that runs on the device -- a hypervisor's vzdump, a
backup server's backup, sync, verify or garbage collection, a host's own restic
or borg run -- and a guest's restart breaks the backup its hypervisor is taking
of it (NetOrk/netork#2318). A driver mixes it in when it can tell; no role
demands it, and a driver without it cannot tell.
"""
from __future__ import annotations
import inspect
from typing import get_type_hints
from napalm_device_types import (
RUNNING_BACKUP_KINDS,
BackupServerDriver,
DeviceTypeDriver,
HypervisorDriver,
OSDriver,
RunningBackupDict,
RunningBackupsMixin,
)
def test_no_role_demands_it():
for role in (BackupServerDriver, HypervisorDriver, OSDriver, DeviceTypeDriver):
assert not issubclass(role, RunningBackupsMixin)
def test_declared_not_implemented():
"""hasattr is the caller's probe: only a driver that can tell answers True."""
assert not hasattr(RunningBackupsMixin, "get_running_backups")
def test_declaration_documents_the_contract():
source = inspect.getsource(RunningBackupsMixin)
assert "def get_running_backups(self) -> list[RunningBackupDict]" in source
assert "empty list for a failed read" in source
def test_the_kinds_cover_a_backup_servers_jobs_and_the_backup_itself():
assert set(RUNNING_BACKUP_KINDS) == {"backup", "sync", "verify", "gc", "prune", "tape"}
def test_the_shape():
hints = get_type_hints(RunningBackupDict)
assert set(hints) == {"kind", "job", "task", "guests", "since"}
assert hints["guests"] == list[str]
assert hints["since"] == int | None
+108
View File
@@ -0,0 +1,108 @@
"""The scheduled-jobs contract: its digest, the names and specs it takes (NetOrk/netork#1933)."""
from __future__ import annotations
import pytest
from napalm_device_types import ScheduledJobsMixin
from napalm_device_types.scheduled_jobs import (
ACTIONS,
ScheduledJobChanged,
ScheduledJobRefused,
check_job_name,
check_spec,
job_digest,
)
JOB = {
"kind": "cron",
"source": "/etc/crontab",
"run_as": "root",
"schedule": "*/5 * * * *",
"command": "/usr/local/bin/backup",
"enabled": True,
}
class TestDigest:
def test_sixteen_hex_characters(self):
digest = job_digest(JOB)
assert len(digest) == 16
int(digest, 16)
def test_the_same_definition_has_the_same_digest(self):
assert job_digest(JOB) == job_digest(dict(JOB))
@pytest.mark.parametrize(
"field, value",
[
("schedule", "0 * * * *"),
("command", "/usr/local/bin/other"),
("run_as", "nobody"),
("enabled", False),
("source", "/etc/cron.d/backup"),
("kind", "anacron"),
],
)
def test_every_part_of_the_definition_changes_it(self, field, value):
assert job_digest({**JOB, field: value}) != job_digest(JOB)
def test_content_the_definition_points_at_changes_it(self):
assert job_digest(JOB, content="12345 678") != job_digest(JOB)
def test_what_the_host_says_of_its_runs_does_not(self):
assert job_digest({**JOB, "last_run": "2026-11-03T06:00:00Z"}) == job_digest(JOB)
class TestNames:
@pytest.mark.parametrize("name", ["backup", "netork-backup-db", "a", "x1"])
def test_a_good_name(self, name):
check_job_name(name)
@pytest.mark.parametrize("name", ["", "-x", "Backup", "a b", "a/b", "a.timer", "a" * 64])
def test_a_name_a_file_or_unit_could_not_carry_safely(self, name):
with pytest.raises(ScheduledJobRefused):
check_job_name(name)
class TestSpec:
SPEC = {
"name": "backup",
"schedule": "0 3 * * *",
"run_as": "root",
"command": "/usr/local/bin/backup",
"enabled": True,
}
def test_a_good_spec(self):
check_spec(self.SPEC)
check_spec({**self.SPEC, "schedule": "@reboot"})
@pytest.mark.parametrize(
"field, value",
[
("schedule", "every day"),
("run_as", "root; rm"),
("run_as", ""),
("command", ""),
("command", " "),
("description", "two\nlines"),
],
)
def test_a_spec_that_cannot_be_written(self, field, value):
with pytest.raises(ScheduledJobRefused):
check_spec({**self.SPEC, field: value})
class TestContract:
def test_the_mixin_declares_nothing_at_runtime(self):
driver = ScheduledJobsMixin()
for method in ("get_scheduled_jobs", "manage_scheduled_job", "put_scheduled_job"):
assert not hasattr(driver, method)
def test_the_actions(self):
assert ACTIONS == ("enable", "disable", "run")
def test_a_refusal_is_a_value_error_and_a_change_is_not(self):
assert issubclass(ScheduledJobRefused, ValueError)
assert not issubclass(ScheduledJobChanged, ValueError)
+540
View File
@@ -0,0 +1,540 @@
"""Scheduled jobs on a POSIX host: the report as root, switching, running, an application's jobs.
The reading command is run by a real ``sh`` over a directory tree that stands in
for the host, with ``systemctl``, ``crontab``, ``dpkg``, ``install`` and
``journalctl`` replaced by scripts that keep a log -- so the shell itself is what
is tested, not a hand-written copy of its output (NetOrk/netork#1933).
"""
from __future__ import annotations
import os
import subprocess
from pathlib import Path
import pytest
from napalm_device_types import PosixScheduledJobsMixin
from napalm_device_types._scheduled_reading import parse_reading
from napalm_device_types.channel import CommandResult
from napalm_device_types.scheduled_jobs import ScheduledJobChanged, ScheduledJobRefused
TIMERS = """\
Id=apt-daily.timer
Unit=apt-daily.service
Description=Daily apt download activities
TimersCalendar={ OnCalendar=*-*-* 06,18:00:00 ; next_elapse=@1762192800 }
NextElapseUSecRealtime=@1762192800
LastTriggerUSec=@1762149600
UnitFileState=enabled
ActiveState=active
FragmentPath=/usr/lib/systemd/system/apt-daily.timer
Id=fstrim.timer
Unit=fstrim.service
Description=Discard unused blocks once a week
TimersMonotonic={ OnUnitActiveUSec=1w ; next_elapse=n/a }
NextElapseUSecRealtime=n/a
LastTriggerUSec=n/a
UnitFileState=static
ActiveState=inactive
FragmentPath=/etc/systemd/system/fstrim.timer
Id=backup.timer
Unit=backup.service
Description=netork backup
TimersCalendar={ OnCalendar=*-*-* 03:00:00 ; next_elapse=n/a }
NextElapseUSecRealtime=Tue 2026-11-04 03:00:00 UTC
LastTriggerUSec=n/a
UnitFileState=enabled
ActiveState=active
FragmentPath=/etc/systemd/system/backup.timer
"""
UNITS = """\
Id=apt-daily.service
ExecStart={ path=/usr/lib/apt/apt.systemd.daily ; argv[]=/usr/lib/apt/apt.systemd.daily update ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }
User=
Result=success
FragmentPath=/usr/lib/systemd/system/apt-daily.service
Id=fstrim.service
ExecStart={ path=/sbin/fstrim ; argv[]=/sbin/fstrim --listed-in /etc/fstab ; ignore_errors=no }
Result=success
FragmentPath=/usr/lib/systemd/system/fstrim.service
Id=backup.service
ExecStart={ path=SCRIPTS/backup ; argv[]=SCRIPTS/backup ; ignore_errors=no }
User=backup
Result=exit-code
FragmentPath=/etc/systemd/system/backup.service
"""
SYSTEMCTL = """#!/bin/sh
echo "systemctl $*" >> "$LOG"
case "$1" in
list-unit-files) printf 'apt-daily.timer enabled enabled\\nfstrim.timer static -\\nbackup.timer enabled enabled\\ngetty@.timer static -\\n' ;;
show) case "$*" in *Unit,Description*) sed "s#SCRIPTS#$SCRIPTS#g" "$FIX/timers" ;; *) sed "s#SCRIPTS#$SCRIPTS#g" "$FIX/units" ;; esac ;;
esac
"""
CRONTAB = """#!/bin/sh
echo "crontab $*" >> "$LOG"
[ "$1" = -u ] || exit 2
if [ "$3" = -l ]; then
[ -f "$SPOOL/$2" ] || { echo "no crontab for $2" >&2; exit 1; }
cat "$SPOOL/$2"
else
cp "$3" "$SPOOL/$2"
fi
"""
DPKG = """#!/bin/sh
for p in "$@"; do case " $OWNED " in *" $p "*) echo "cron: $p" ;; esac; done
"""
INSTALL = """#!/bin/sh
echo "install $*" >> "$LOG"
args=""
while [ $# -gt 0 ]; do case "$1" in -o|-g) shift 2 ;; *) args="$args '$1'"; shift ;; esac; done
eval "/usr/bin/install $args"
"""
ECHO_LOG = """#!/bin/sh
echo "$(basename "$0") $*" >> "$LOG"
[ "$(basename "$0")" = journalctl ] && echo "backup ran fine"
exit 0
"""
class Host(PosixScheduledJobsMixin):
"""A host in a directory; every command is run by sh, root or not."""
def __init__(self, root: Path, *, systemd: bool = True, own_cron_d: bool = True) -> None:
self.root = root
etc = root / "etc"
self.CRON_SYSTEM_FILES = (str(etc / "crontab"),) # type: ignore[misc]
self.CRON_D_DIRS = (str(etc / "cron.d"),) # type: ignore[misc]
self.CRON_SPOOL_DIRS = (str(root / "spool"),) # type: ignore[misc]
self.RUN_PARTS_DIRS = (("daily", str(etc / "cron.daily")),) # type: ignore[misc]
self.ANACRONTAB = str(etc / "anacrontab") # type: ignore[misc]
self.JOB_SCRIPT_DIR = str(root / "scripts") # type: ignore[misc]
self.UNIT_DIR = str(root / "units") # type: ignore[misc]
self.OWN_CRON_D_DIR = str(etc / "cron.d") if own_cron_d else None # type: ignore[misc]
self.ROOT_GROUP = subprocess.run(
["id", "-gn"], capture_output=True, text=True
).stdout.strip() # type: ignore[misc]
self.systemd_dir = root / ("run-systemd" if systemd else "no-systemd")
self.commands: list[str] = []
def _sj_paths(self): # type: ignore[no-untyped-def]
paths = super()._sj_paths()
from dataclasses import replace
return replace(paths, systemd_marker=str(self.systemd_dir))
def run_command(self, command, *, privileged=False, timeout=60, stdin=None): # type: ignore[no-untyped-def]
self.commands.append(command)
env = {
**os.environ,
"PATH": f"{self.root / 'bin'}:{os.environ['PATH']}",
"LOG": str(self.root / "log"),
"SPOOL": str(self.root / "spool"),
"FIX": str(self.root / "fix"),
"SCRIPTS": str(self.root / "scripts"),
"OWNED": str(self.root / "etc" / "crontab"),
}
done = subprocess.run(
["sh", "-c", command], input=stdin, capture_output=True, env=env, timeout=30
)
return CommandResult(done.stdout.decode(), done.stderr.decode(), done.returncode)
def log(self) -> str:
log = self.root / "log"
return log.read_text() if log.exists() else ""
def _tool(root: Path, name: str, body: str) -> None:
path = root / "bin" / name
path.write_text(body)
path.chmod(0o755)
@pytest.fixture
def root(tmp_path: Path) -> Path:
for d in (
"etc/cron.d",
"etc/cron.daily",
"spool",
"scripts",
"units",
"bin",
"fix",
"run-systemd",
):
(tmp_path / d).mkdir(parents=True)
(tmp_path / "etc/crontab").write_text(
"SHELL=/bin/sh\nOUT="
+ str(tmp_path / "out")
+ "\n17 * * * * root cd / && run-parts /etc/cron.hourly\n"
'*/5 * * * * root echo "$OUT" > ' + str(tmp_path / "ran") + "\n"
)
(tmp_path / "etc/cron.d/certbot").write_text("0 */12 * * * root certbot -q renew\n")
(tmp_path / "etc/anacrontab").write_text("1 5 cron.daily run-parts /etc/cron.daily\n")
(tmp_path / "spool/www-data").write_text("*/5 * * * * php /var/www/cron.php\n")
for name, mode in (("logrotate", 0o755), ("off", 0o644), ("x.dpkg-old", 0o755)):
script = tmp_path / "etc/cron.daily" / name
script.write_text("#!/bin/sh\necho rotated\n")
script.chmod(mode)
(tmp_path / "scripts/backup").write_text(
"#!/bin/sh\n# napalm-scheduled-job: backup nightly backup\n# schedule: 0 3 * * *\n/usr/local/bin/backup --all\n"
)
(tmp_path / "fix/timers").write_text(TIMERS)
(tmp_path / "fix/units").write_text(UNITS)
for name, body in (
("systemctl", SYSTEMCTL),
("crontab", CRONTAB),
("dpkg", DPKG),
("install", INSTALL),
("journalctl", ECHO_LOG),
("systemd-analyze", ECHO_LOG),
):
_tool(tmp_path, name, body)
return tmp_path
def _jobs(host: Host) -> dict[str, dict]:
return {j["key"]: j for j in host.get_scheduled_jobs()["jobs"]}
class TestReading:
def test_every_source_is_read(self, root: Path):
jobs = _jobs(Host(root))
kinds = sorted({j["kind"] for j in jobs.values()})
assert kinds == ["anacron", "cron", "run_parts", "systemd_timer"]
users = {j["run_as"] for j in jobs.values() if j["kind"] == "cron"}
assert users == {"root", "www-data"}
def test_a_package_owns_what_it_installed(self, root: Path):
jobs = _jobs(Host(root))
crontab = [j for j in jobs.values() if j["source"] == str(root / "etc/crontab")]
assert {j["managed_by"] for j in crontab} == {"package"}
certbot = next(j for j in jobs.values() if j["source"].endswith("certbot"))
assert certbot["managed_by"] == "admin"
def test_run_parts_by_execute_bit_and_the_names_run_parts_runs(self, root: Path):
jobs = _jobs(Host(root))
daily = root / "etc/cron.daily"
assert jobs[f"run-parts:{daily}/logrotate"]["enabled"]
assert not jobs[f"run-parts:{daily}/off"]["enabled"]
assert f"run-parts:{daily}/x.dpkg-old" not in jobs
def test_a_changed_script_changes_its_digest(self, root: Path):
host = Host(root)
key = f"run-parts:{root}/etc/cron.daily/logrotate"
before = _jobs(host)[key]["digest"]
(root / "etc/cron.daily/logrotate").write_text("#!/bin/sh\necho something else\n")
assert _jobs(host)[key]["digest"] != before
def test_timers_with_their_unit_and_runs(self, root: Path):
apt = _jobs(Host(root))["timer:apt-daily.timer"]
assert apt["schedule"] == "OnCalendar=*-*-* 06,18:00:00"
assert apt["schedule_kind"] == "calendar"
assert apt["command"] == "/usr/lib/apt/apt.systemd.daily update"
assert (apt["run_as"], apt["managed_by"], apt["enabled"]) == ("root", "package", True)
assert apt["last_run"] == "2025-11-03T06:00:00+00:00"
assert apt["next_run"] == "2025-11-03T18:00:00+00:00"
assert apt["last_result"] == "success"
assert apt["actions"] == ["enable", "disable", "run"]
def test_a_static_timer_only_runs(self, root: Path):
fstrim = _jobs(Host(root))["timer:fstrim.timer"]
assert fstrim["schedule"] == "OnUnitActiveSec=1w"
assert fstrim["schedule_kind"] == "monotonic"
assert fstrim["actions"] == ["run"]
assert fstrim.get("last_run") is None
assert fstrim["managed_by"] == "admin"
def test_an_applications_timer_by_its_script(self, root: Path):
backup = _jobs(Host(root))["job:backup"]
assert backup["managed_by"] == "application"
assert (backup["name"], backup["description"]) == ("backup", "nightly backup")
assert (backup["schedule"], backup["schedule_kind"]) == ("0 3 * * *", "cron")
assert backup["command"] == "/usr/local/bin/backup --all"
assert backup["run_as"] == "backup"
assert "edit" in backup["actions"] and "delete" in backup["actions"]
assert backup["next_run"] == "2026-11-04T03:00:00+00:00"
assert "last_result" not in backup
def test_a_file_without_a_last_newline_does_not_swallow_the_next(self, root: Path):
(root / "etc/cron.d/certbot").write_text("0 */12 * * * root certbot -q renew")
jobs = _jobs(Host(root))
assert any(j["source"].endswith("certbot") for j in jobs.values())
assert any(j["source"].endswith("spool/www-data") for j in jobs.values())
def test_without_systemd_no_timers(self, root: Path):
jobs = _jobs(Host(root, systemd=False))
assert not [j for j in jobs.values() if j["kind"] == "systemd_timer"]
def test_a_report_cut_short_raises(self, root: Path):
host = Host(root)
full = host.run_command
host.run_command = lambda c, **kw: full(c, **kw)._replace(
stdout=full(c, **kw).stdout.split("SJ_END")[0][:-3]
) # type: ignore[method-assign]
with pytest.raises(ValueError, match="cut short"):
host.get_scheduled_jobs()
class TestDriverSettings:
def test_what_the_platform_generates(self, root: Path):
host = Host(root)
host.PLATFORM_SOURCES = ("*/cron.d/cert*",) # type: ignore[misc]
certbot = next(j for j in _jobs(host).values() if j["source"].endswith("certbot"))
assert certbot["managed_by"] == "platform"
def test_the_time_zone_and_owners_commands_are_the_drivers(self, root: Path):
host = Host(root)
host.TIME_ZONE_SH = "echo America/New_York" # type: ignore[misc]
host.OWNERS_SH = "cat >/dev/null" # type: ignore[misc]
reading = host.get_scheduled_jobs()
assert reading["time_zone"] == "America/New_York"
assert {j["managed_by"] for j in reading["jobs"] if j["kind"] == "cron"} == {"admin"}
class TestParseReading:
def test_the_time_zone_from_either_file(self):
out = "SJ_BEGIN\n[tz]\n/usr/share/zoneinfo/Europe/Berlin\nSJ_END\n"
assert parse_reading(out).time_zone == "Europe/Berlin"
assert parse_reading("SJ_BEGIN\n[tz]\nEtc/UTC\nSJ_END").time_zone == "Etc/UTC"
assert parse_reading("SJ_BEGIN\n[tz]\nSJ_END").time_zone is None
def test_owners_skip_diversions(self):
out = "SJ_BEGIN\n[owners]\ndiversion by x from: /etc/crontab\ncron, cron-daemon: /etc/cron.d/x\nSJ_END"
reading = parse_reading(out + "")
assert reading.jobs == []
def test_a_time_not_in_utc_is_left_unknown(self):
from napalm_device_types._scheduled_reading import _timestamp
assert _timestamp("Tue 2026-11-04 03:00:00 CET") is None
assert _timestamp("@0") is None
class TestSwitching:
def _cron(self, host: Host, path: str) -> dict:
return next(j for j in _jobs(host).values() if j["source"] == path and j["kind"] == "cron")
def test_a_system_line_switched_off_and_on(self, root: Path):
host = Host(root)
path = str(root / "etc/cron.d/certbot")
job = self._cron(host, path)
mode = os.stat(path).st_mode
assert host.manage_scheduled_job(job["key"], "disable", digest=job["digest"])["success"]
assert (
root / "etc/cron.d/certbot"
).read_text() == "#napalm-disabled: 0 */12 * * * root certbot -q renew\n"
off = _jobs(host)[job["key"]]
assert not off["enabled"]
host.manage_scheduled_job(job["key"], "enable", digest=off["digest"])
assert (root / "etc/cron.d/certbot").read_text() == "0 */12 * * * root certbot -q renew\n"
assert f"install -D -m {mode & 0o777:o}" in host.log()
assert os.stat(path).st_mode == mode
def test_a_linked_file_is_written_through_its_link(self, root: Path):
real = root / "etc/vzdump.cron"
real.write_text("0 1 * * * root vzdump --all\n")
real.chmod(0o640)
link = root / "etc/cron.d/vzdump"
link.symlink_to(real)
host = Host(root)
job = self._cron(host, str(link))
host.manage_scheduled_job(job["key"], "disable", digest=job["digest"])
assert link.is_symlink()
assert real.read_text() == "#napalm-disabled: 0 1 * * * root vzdump --all\n"
assert real.stat().st_mode & 0o777 == 0o640
def test_a_users_line_goes_through_crontab(self, root: Path):
host = Host(root)
job = self._cron(host, str(root / "spool/www-data"))
host.manage_scheduled_job(job["key"], "disable", digest=job["digest"])
assert (root / "spool/www-data").read_text().startswith("#napalm-disabled: */5")
assert "crontab -u www-data /" in host.log()
def test_a_change_it_did_not_see_is_refused(self, root: Path):
host = Host(root)
job = self._cron(host, str(root / "etc/cron.d/certbot"))
with pytest.raises(ScheduledJobChanged):
host.manage_scheduled_job(job["key"], "disable", digest="0" * 16)
def test_an_unknown_job_and_an_unknown_action(self, root: Path):
host = Host(root)
with pytest.raises(LookupError):
host.manage_scheduled_job("cron:/nowhere:1", "disable", digest="x")
with pytest.raises(ScheduledJobRefused):
host.manage_scheduled_job("cron:/nowhere:1", "explode", digest="x")
def test_a_static_timer_is_not_disabled(self, root: Path):
host = Host(root)
fstrim = _jobs(host)["timer:fstrim.timer"]
with pytest.raises(ScheduledJobRefused):
host.manage_scheduled_job("timer:fstrim.timer", "disable", digest=fstrim["digest"])
def test_a_timer_and_a_run_parts_script(self, root: Path):
host = Host(root)
jobs = _jobs(host)
apt = jobs["timer:apt-daily.timer"]
host.manage_scheduled_job(apt["key"], "disable", digest=apt["digest"])
assert "systemctl disable --now -- apt-daily.timer" in host.log()
key = f"run-parts:{root}/etc/cron.daily/off"
host.manage_scheduled_job(key, "enable", digest=jobs[key]["digest"])
assert os.access(root / "etc/cron.daily/off", os.X_OK)
def test_an_applications_timer_is_switched_by_its_name(self, root: Path):
host = Host(root)
backup = _jobs(host)["job:backup"]
host.manage_scheduled_job("job:backup", "disable", digest=backup["digest"])
assert "systemctl disable --now -- backup.timer" in host.log()
class TestRunning:
def test_a_cron_line_runs_as_cron_would(self, root: Path):
host = Host(root)
job = next(j for j in _jobs(host).values() if "ran" in j["command"])
result = host.manage_scheduled_job(job["key"], "run", digest=job["digest"])
assert result["success"]
assert (root / "ran").read_text() == f"{root / 'out'}\n"
assert "timeout 300 sh -c" in host.commands[-1]
def test_a_timer_runs_its_unit_and_says_what_its_journal_said(self, root: Path):
host = Host(root)
backup = _jobs(host)["job:backup"]
result = host.manage_scheduled_job("job:backup", "run", digest=backup["digest"])
assert result == {"success": True, "output": "backup ran fine"}
assert "systemctl start -- backup.service" in host.log()
def test_a_run_parts_script_runs(self, root: Path):
host = Host(root)
key = f"run-parts:{root}/etc/cron.daily/logrotate"
result = host.manage_scheduled_job(key, "run", digest=_jobs(host)[key]["digest"])
assert result == {"success": True, "output": "rotated"}
def test_another_user_runs_through_su(self):
assert Host(Path("/x"))._sj_as("www-data", "true") == [
"su",
"-s",
"/bin/sh",
"www-data",
"-c",
"true",
]
SPEC = {
"name": "netork-tidy",
"schedule": "*/30 1-2 * * *",
"run_as": "root",
"command": "find /tmp -mtime +7 -delete",
"enabled": True,
"description": "tidy %tmp",
}
class TestApplicationJobs:
def test_a_timer_on_a_host_with_systemd(self, root: Path):
host = Host(root)
assert host.put_scheduled_job(SPEC)["output"] == "job:netork-tidy created"
timer = (root / "units/netork-tidy.timer").read_text()
assert "OnCalendar=*-*-* 01,02:00,30:00" in timer
assert "Description=tidy %%tmp" in timer
service = (root / "units/netork-tidy.service").read_text()
assert f"ExecStart={root}/scripts/netork-tidy" in service and "User=root" in service
script = (root / "scripts/netork-tidy").read_text()
assert script.splitlines()[1:] == [
"# napalm-scheduled-job: netork-tidy tidy %tmp",
"# schedule: */30 1-2 * * *",
"find /tmp -mtime +7 -delete",
]
assert "systemctl daemon-reload" in host.log()
assert "systemctl enable --now -- netork-tidy.timer" in host.log()
def test_reboot_is_a_boot_timer(self, root: Path):
Host(root).put_scheduled_job({**SPEC, "schedule": "@reboot", "enabled": False})
assert "OnBootSec=1min" in (root / "units/netork-tidy.timer").read_text()
def test_creating_one_that_is_there_and_changing_one_that_is_not(self, root: Path):
host = Host(root)
with pytest.raises(ScheduledJobChanged):
host.put_scheduled_job({**SPEC, "name": "backup"})
with pytest.raises(LookupError):
host.put_scheduled_job(SPEC, digest="x")
with pytest.raises(ScheduledJobChanged):
host.put_scheduled_job({**SPEC, "name": "backup"}, digest="x")
def test_an_unknown_user_is_refused(self, root: Path):
with pytest.raises(ScheduledJobRefused, match="no user"):
Host(root).put_scheduled_job({**SPEC, "run_as": "no-such-user-here"})
def test_a_timer_deleted(self, root: Path):
host = Host(root)
backup = _jobs(host)["job:backup"]
host.delete_scheduled_job("job:backup", digest=backup["digest"])
assert not (root / "scripts/backup").exists()
assert "systemctl disable --now -- backup.timer" in host.log()
def test_only_an_applications_job_is_deleted(self, root: Path):
host = Host(root)
apt = _jobs(host)["timer:apt-daily.timer"]
with pytest.raises(ScheduledJobRefused):
host.delete_scheduled_job(apt["key"], digest=apt["digest"])
def test_a_cron_d_file_without_systemd(self, root: Path):
host = Host(root, systemd=False)
host.put_scheduled_job(SPEC)
assert (root / "etc/cron.d/netork-tidy").read_text() == (
"# napalm-scheduled-job: netork-tidy tidy %tmp\n*/30 1-2 * * * root find /tmp -mtime +7 -delete\n"
)
job = _jobs(host)["job:netork-tidy"]
assert job["managed_by"] == "application" and job["kind"] == "cron"
host.delete_scheduled_job("job:netork-tidy", digest=job["digest"])
assert not (root / "etc/cron.d/netork-tidy").exists()
def test_marked_lines_in_the_users_crontab_on_a_busybox_host(self, root: Path):
host = Host(root, systemd=False, own_cron_d=False)
host.put_scheduled_job({**SPEC, "run_as": "www-data"})
text = (root / "spool/www-data").read_text()
assert text.endswith(
"# napalm-scheduled-job: netork-tidy tidy %tmp\n*/30 1-2 * * * find /tmp -mtime +7 -delete\n"
)
job = _jobs(host)["job:netork-tidy"]
host.put_scheduled_job(
{**SPEC, "run_as": "www-data", "enabled": False}, digest=job["digest"]
)
assert "#napalm-disabled: */30" in (root / "spool/www-data").read_text()
job = _jobs(host)["job:netork-tidy"]
host.put_scheduled_job({**SPEC, "run_as": "nobody"}, digest=job["digest"])
assert "netork-tidy" not in (root / "spool/www-data").read_text()
assert "netork-tidy" in (root / "spool/nobody").read_text()
job = _jobs(host)["job:netork-tidy"]
host.delete_scheduled_job("job:netork-tidy", digest=job["digest"])
assert (root / "spool/nobody").read_text() == ""
def test_a_cron_d_that_is_not_there_is_the_crontab(self, root: Path):
host = Host(root, systemd=False)
host.OWN_CRON_D_DIR = str(root / "etc/no-cron.d") # type: ignore[misc]
host.put_scheduled_job(SPEC)
assert "netork-tidy" in (root / "spool/root").read_text()
def test_a_cron_line_holds_one_line(self, root: Path):
with pytest.raises(ScheduledJobRefused, match="one line"):
Host(root, systemd=False).put_scheduled_job({**SPEC, "command": "a\nb"})
def test_a_calendar_systemd_refuses(self, root: Path):
_tool(root, "systemd-analyze", "#!/bin/sh\necho 'Failed to parse' >&2\nexit 1\n")
with pytest.raises(ScheduledJobRefused, match="systemd does not take"):
Host(root).put_scheduled_job(SPEC)
def test_a_failed_install_says_so(self, root: Path):
_tool(root, "install", "#!/bin/sh\necho 'read-only file system' >&2\nexit 1\n")
with pytest.raises(RuntimeError, match="read-only"):
Host(root).put_scheduled_job(SPEC)
+27
View File
@@ -14,6 +14,7 @@ import pytest
from napalm_device_types.channel import CommandResult
from napalm_device_types.ssh_server import (
DROPIN_DIR,
DROPIN_PATH,
MAIN_CONFIG,
OpenSshServerMixin,
@@ -401,6 +402,32 @@ def test_mixin_falls_back_to_sshd_config_without_an_include():
assert host._effective()["mac"] == ["hmac-sha2-256"]
class GeneratedConfigHost(FakeHost):
"""A platform that writes sshd_config from its own database (OpenMediaVault)."""
SSHD_CONFIG_GENERATED = True
def test_generated_sshd_config_still_gets_the_dropin():
host = GeneratedConfigHost()
result = host.harden_ssh_server({"mac": ["hmac-sha1"]})
assert result.applied
assert result.path == DROPIN_PATH
assert host.files[MAIN_CONFIG] == "Include /etc/ssh/sshd_config.d/*.conf\nPort 22\n"
def test_generated_sshd_config_without_an_include_is_left_alone():
"""The platform would write its own file over the change on its next apply."""
host = GeneratedConfigHost(main="Port 22\nMACs hmac-sha1,hmac-sha2-256\n")
result = host.harden_ssh_server({"mac": ["hmac-sha1"]})
assert not result.applied and not result.rolled_back
assert result.path is None
assert host.files[MAIN_CONFIG] == "Port 22\nMACs hmac-sha1,hmac-sha2-256\n"
assert list(host.files) == [MAIN_CONFIG]
assert host.reloads == 0
assert DROPIN_DIR in result.message
def test_mixin_restores_when_sshd_rejects_the_config():
host = FakeHost()
host.syntax_ok = False
+353
View File
@@ -0,0 +1,353 @@
"""How an OpenSSH server lets people in: the effective settings and sshd's PAM auth stack.
The parsers are pure. The getter runs against a fake host that answers ``sshd -T``
and the dump of ``/etc/pam.d`` from a dict, so the root read, the PAM includes
and a failed read are covered without a machine.
The ``sshd -T`` texts are abridged from what OpenSSH 9.2p1 on Debian 12 prints
(the keywords and their order as sshd writes them; the lines that say nothing
about authentication are left out). OpenSSH before 8.7 also prints
``challengeresponseauthentication``; OpenBSD's sshd has no ``usepam``.
"""
from __future__ import annotations
import pytest
from napalm_device_types.channel import CommandResult
from napalm_device_types.ssh_server import OpenSshServerMixin
from napalm_device_types.ssh_server_auth import (
PAM_FILES_COMMAND,
SSHD_EFFECTIVE_COMMAND,
SshServerAuthMixin,
pam_auth_stack,
parse_pam_auth,
parse_pam_files,
parse_sshd_auth,
)
#: Debian 12 as installed: password login on, PAM on, no AuthenticationMethods.
SSHD_T_DEBIAN = """\
port 22
addressfamily any
listenaddress [::]:22
listenaddress 0.0.0.0:22
usepam yes
logingracetime 120
x11displayoffset 10
maxauthtries 6
maxsessions 10
clientaliveinterval 0
clientalivecountmax 3
permitrootlogin without-password
ignorerhosts yes
ignoreuserknownhosts no
hostbasedauthentication no
hostbasedusesnamefrompacketonly no
pubkeyauthentication yes
kerberosauthentication no
kerberosorlocalpasswd yes
kerberosticketcleanup yes
gssapiauthentication no
gssapicleanupcredentials yes
gssapikeyexchange no
gssapistrictacceptorcheck yes
gssapistorecredentialsonrotate no
passwordauthentication yes
kbdinteractiveauthentication no
printmotd no
x11forwarding yes
authenticationmethods any
kexalgorithms sntrup761x25519-sha512@openssh.com,curve25519-sha256
"""
#: Key only: password and keyboard-interactive off.
SSHD_T_KEY_ONLY = SSHD_T_DEBIAN.replace(
"passwordauthentication yes", "passwordauthentication no"
).replace("permitrootlogin without-password", "permitrootlogin no")
#: A key and then an OTP through PAM, for everyone.
SSHD_T_KEY_AND_OTP = SSHD_T_KEY_ONLY.replace(
"kbdinteractiveauthentication no", "kbdinteractiveauthentication yes"
).replace("authenticationmethods any", "authenticationmethods publickey,keyboard-interactive")
#: OpenSSH 8.4p1 (Debian 11): the old keyword beside the new one.
SSHD_T_OLD = SSHD_T_KEY_ONLY.replace(
"kbdinteractiveauthentication no",
"kbdinteractiveauthentication yes\nchallengeresponseauthentication no",
)
#: OpenBSD 7.6: no usepam line.
SSHD_T_OPENBSD = SSHD_T_KEY_ONLY.replace("usepam yes\n", "")
#: Debian 12's /etc/pam.d/sshd, abridged: the auth stack comes from common-auth.
PAM_SSHD_DEBIAN = """\
# PAM configuration for the Secure Shell service
# Standard Un*x authentication.
@include common-auth
# Disallow non-root logins when /etc/nologin exists.
account required pam_nologin.so
# Standard Un*x authorization.
@include common-account
session [success=ok ignore=ignore module_unknown=ignore default=bad] pam_selinux.so close
@include common-session
@include common-password
"""
PAM_COMMON_AUTH = """\
auth\t[success=1 default=ignore]\tpam_unix.so nullok
auth\trequisite\t\t\tpam_deny.so
auth\trequired\t\t\tpam_permit.so
"""
PAM_COMMON_ACCOUNT = "account [success=1 new_authtok_reqd=done default=ignore] pam_unix.so\n"
#: The usual way to add a TOTP: a line of its own in sshd's stack.
PAM_SSHD_OTP = PAM_SSHD_DEBIAN.replace(
"@include common-auth",
"#@include common-auth\nauth required pam_google_authenticator.so nullok",
)
#: Red Hat's form: ``auth substack password-auth``, a module by its full path.
PAM_SSHD_RHEL = """\
#%PAM-1.0
auth substack password-auth
auth include postlogin
-auth optional /usr/lib64/security/pam_reauthorize.so prepare
account required pam_sepermit.so
"""
PAM_PASSWORD_AUTH = "auth required pam_env.so\nauth sufficient pam_unix.so try_first_pass nullok\n"
PAM_POSTLOGIN = "session optional pam_umask.so silent\n"
# ── parse_sshd_auth ──────────────────────────────────────────────────────
def test_parse_reads_the_settings_that_decide_how_people_log_in():
auth = parse_sshd_auth(SSHD_T_DEBIAN)
assert auth == {
"password_authentication": True,
"kbd_interactive_authentication": False,
"pubkey_authentication": True,
"hostbased_authentication": False,
"gssapi_authentication": False,
"authentication_methods": [],
"permit_root_login": "without-password",
"use_pam": True,
"pam_auth_stack": None,
}
def test_parse_splits_authentication_methods_into_lists_of_methods():
auth = parse_sshd_auth(
SSHD_T_KEY_ONLY.replace(
"authenticationmethods any",
"authenticationmethods publickey,keyboard-interactive:pam publickey,password",
)
)
assert auth["authentication_methods"] == [
["publickey", "keyboard-interactive:pam"],
["publickey", "password"],
]
def test_parse_counts_the_old_challenge_response_keyword_as_keyboard_interactive():
"""Before 8.7 either keyword could switch PAM's prompts on: either one is enough."""
assert parse_sshd_auth(SSHD_T_OLD)["kbd_interactive_authentication"] is True
old_only = SSHD_T_KEY_ONLY.replace(
"kbdinteractiveauthentication no",
"kbdinteractiveauthentication no\nchallengeresponseauthentication yes",
)
assert parse_sshd_auth(old_only)["kbd_interactive_authentication"] is True
def test_parse_says_none_for_usepam_where_sshd_has_no_pam():
assert parse_sshd_auth(SSHD_T_OPENBSD)["use_pam"] is None
def test_parse_takes_a_method_sshd_was_built_without_as_off():
"""sshd built without GSSAPI prints no gssapiauthentication line: that method cannot be used."""
text = SSHD_T_DEBIAN.replace("gssapiauthentication no\n", "")
assert parse_sshd_auth(text)["gssapi_authentication"] is False
def test_parse_refuses_output_that_is_not_sshd_t():
with pytest.raises(ValueError, match="passwordauthentication"):
parse_sshd_auth("sshd: no hostkeys available -- exiting.\n")
# ── PAM ──────────────────────────────────────────────────────────────────
def _pairs(entries) -> list[tuple[str, str]]:
return [(e["control"], e["module"]) for e in entries]
def _modules(stack) -> list[str]:
return [e["module"] for e in stack]
def test_pam_auth_lists_modules_and_includes_in_stack_order():
# Every @include: which of them hold auth lines is for the included file to say.
assert _pairs(parse_pam_auth(PAM_SSHD_DEBIAN)) == [
("include", "common-auth"),
("include", "common-account"),
("include", "common-session"),
("include", "common-password"),
]
def test_pam_auth_reads_bracketed_controls_paths_and_optional_lines():
assert _pairs(parse_pam_auth(PAM_SSHD_RHEL)) == [
("substack", "password-auth"),
("include", "postlogin"),
("optional", "pam_reauthorize.so"),
]
assert parse_pam_auth(PAM_COMMON_AUTH) == [
{"module": "pam_unix.so", "control": "[success=1 default=ignore]", "args": ["nullok"]},
{"module": "pam_deny.so", "control": "requisite", "args": []},
{"module": "pam_permit.so", "control": "required", "args": []},
]
def test_pam_auth_keeps_a_modules_arguments():
"""``nullok`` lets a user without an OTP secret through: the consumer has to see it."""
assert parse_pam_auth(PAM_SSHD_OTP)[0] == {
"module": "pam_google_authenticator.so",
"control": "required",
"args": ["nullok"],
}
def test_pam_auth_skips_comments_and_other_types():
assert _pairs(parse_pam_auth(PAM_SSHD_OTP)) == [
("required", "pam_google_authenticator.so"),
("include", "common-account"),
("include", "common-session"),
("include", "common-password"),
]
def test_pam_auth_ignores_an_include_that_is_not_a_service_name():
assert parse_pam_auth("auth include ../../etc/shadow\n@include\nauth required\n") == []
def _dump(files: dict[str, str]) -> str:
"""What ``tail -n +1`` prints for several files: a header before each."""
return "\n".join(f"==> /etc/pam.d/{name} <==\n{text}" for name, text in files.items())
def test_pam_files_splits_the_dump_by_file():
files = {"sshd": PAM_SSHD_DEBIAN, "common-auth": PAM_COMMON_AUTH}
assert parse_pam_files(_dump(files)) == files
def test_auth_stack_follows_includes_and_substacks_in_stack_order():
files = {
"sshd": PAM_SSHD_RHEL + "auth include password-auth\n",
"password-auth": PAM_PASSWORD_AUTH,
"postlogin": PAM_POSTLOGIN,
}
assert _modules(pam_auth_stack(files)) == ["pam_env.so", "pam_unix.so", "pam_reauthorize.so"]
def test_auth_stack_stops_at_an_include_loop_and_skips_a_missing_file():
files = {
"sshd": "@include a\n@include gone\n",
"a": "@include sshd\nauth required pam_unix.so\n",
}
assert _modules(pam_auth_stack(files)) == ["pam_unix.so"]
def test_auth_stack_is_none_without_a_file_for_the_service():
assert pam_auth_stack({"login": PAM_COMMON_AUTH}) is None
# ── the getter ───────────────────────────────────────────────────────────
class _Host(SshServerAuthMixin):
"""Answers ``sshd -T`` (root only) and the dump of /etc/pam.d."""
def __init__(self, sshd_t: str | None, pam: dict[str, str] | None = None) -> None:
self.sshd_t = sshd_t
self.pam = pam
self.calls: list[tuple[str, bool]] = []
def run_command(
self,
command: str,
*,
privileged: bool = False,
timeout: float = 60,
stdin: bytes | None = None,
) -> CommandResult:
self.calls.append((command, privileged))
if command == SSHD_EFFECTIVE_COMMAND:
if self.sshd_t is None or not privileged:
return CommandResult("", "sshd: no hostkeys available -- exiting.", 1)
return CommandResult(self.sshd_t, "", 0)
assert command == PAM_FILES_COMMAND, command
if self.pam is None:
return CommandResult("", "tail: cannot open '/etc/pam.d/*' for reading", 1)
return CommandResult(_dump(self.pam), "", 0)
def test_getter_reads_sshd_t_as_root_and_pam_without():
host = _Host(
SSHD_T_DEBIAN,
{
"sshd": PAM_SSHD_DEBIAN,
"common-auth": PAM_COMMON_AUTH,
"common-account": PAM_COMMON_ACCOUNT,
},
)
auth = host.get_ssh_server_auth()
assert auth["password_authentication"] is True
assert _modules(auth["pam_auth_stack"]) == ["pam_unix.so", "pam_deny.so", "pam_permit.so"]
# sshd -T once, as root; PAM's files once, as the login user.
assert host.calls == [(SSHD_EFFECTIVE_COMMAND, True), (PAM_FILES_COMMAND, False)]
def test_getter_names_an_otp_module_in_sshds_own_stack():
host = _Host(SSHD_T_KEY_AND_OTP, {"sshd": PAM_SSHD_OTP})
stack = host.get_ssh_server_auth()["pam_auth_stack"]
assert _modules(stack) == ["pam_google_authenticator.so"]
def test_getter_keeps_what_it_read_when_one_pam_file_is_unreadable():
"""tail goes on past a file it cannot open and exits 1: the rest is still read."""
class _Partial(_Host):
def run_command(self, command: str, **kwargs) -> CommandResult: # type: ignore[override]
result = super().run_command(command, **kwargs)
if command == PAM_FILES_COMMAND:
return CommandResult(result.stdout, "tail: cannot open 'x'", 1)
return result
host = _Partial(SSHD_T_DEBIAN, {"sshd": "auth required pam_unix.so\n"})
assert _modules(host.get_ssh_server_auth()["pam_auth_stack"]) == ["pam_unix.so"]
def test_getter_leaves_pam_unread_without_usepam():
host = _Host(SSHD_T_OPENBSD)
auth = host.get_ssh_server_auth()
assert auth["pam_auth_stack"] is None
assert host.calls == [(SSHD_EFFECTIVE_COMMAND, True)]
def test_getter_says_none_when_pam_cannot_be_read():
assert _Host(SSHD_T_DEBIAN).get_ssh_server_auth()["pam_auth_stack"] is None
def test_getter_raises_when_sshd_t_fails():
with pytest.raises(RuntimeError, match="no hostkeys"):
_Host(None).get_ssh_server_auth()
def test_the_hardening_mixin_carries_the_reader():
"""Every driver that can harden sshd can also say how it lets people in."""
assert issubclass(OpenSshServerMixin, SshServerAuthMixin)