Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
aaa9f0504a | ||
|
|
bc47220318 | ||
|
|
08a62838f7 | ||
|
|
ac38c68feb | ||
|
|
8516f2a1ee | ||
|
|
c74173523a | ||
|
|
6c7f164004 | ||
|
|
073087da69 | ||
|
|
278a707157 | ||
|
|
56e5e82a59 | ||
|
|
e6f145d4a3 | ||
|
|
66037e878e | ||
|
|
afffd0915c | ||
|
|
92891473be | ||
|
|
658f1c8ab2 | ||
|
|
aacde6be3f | ||
|
|
58c7616401 | ||
|
|
b7e31205d5 | ||
|
|
1d71ff8f91 | ||
|
|
58ed0f0a94 | ||
|
|
c1a05d6dba | ||
|
|
fccf902c07 | ||
|
|
f5a2811333 | ||
|
|
98d1dbef4c | ||
|
|
252ea3b81c | ||
|
|
b685b200cc | ||
|
|
8557710724 | ||
|
|
1b600416f0 | ||
|
|
e622b26d42 | ||
|
|
5daba5d37c | ||
|
|
dacd10fcce | ||
|
|
8a54674e40 | ||
|
|
d45455adf1 | ||
|
|
ac49e2b5d4 | ||
|
|
c967635cba | ||
|
|
c470686305 | ||
|
|
1edf458feb | ||
|
|
00452d0d37 | ||
|
|
4e40f95bc1 | ||
|
|
70e1c9b218 | ||
|
|
a70501ff60 | ||
|
|
c6d119eb83 | ||
|
|
8e9d21c368 | ||
|
|
25f28297e4 | ||
|
|
847c902193 | ||
|
|
ffbb194300 | ||
|
|
edef6cf07c | ||
|
|
df476a12f2 | ||
|
|
834cb2e0ea | ||
|
|
f573fa53d2 | ||
|
|
fdf2f89115 | ||
|
|
d63abff467 | ||
|
|
9d6a1a65ed | ||
|
|
95e3aa1e26 |
No files matched your search
+202
@@ -7,6 +7,208 @@ recorded here; their history is the git log.
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [4.36.0] - 2026-11-11
|
||||
|
||||
### Added
|
||||
- `TimeSyncDict.dhcp_ntp_servers` (not required): the NTP servers a device got by DHCP
|
||||
(option 42) and uses, in its order; `[]` for none, absent or None where the driver cannot
|
||||
tell. `ntp_servers` stays the servers configured on the device and no longer holds
|
||||
DHCP's. A device that gets the site's time giver by DHCP is aligned, so an application
|
||||
has to tell the two apart (NetOrk/netork#2380).
|
||||
- `PosixTimeSyncMixin` reads them: chrony's sources from the DHCP client's `sourcedir`
|
||||
(`CHRONY_DHCP_DIRS`, `/run/chrony-dhcp` by default), systemd-timesyncd's
|
||||
`LinkNTPServers`. Where ntpd or OpenNTPD runs they are unknown (None).
|
||||
`timesyncd_link_servers` is public.
|
||||
|
||||
### Changed
|
||||
- `configure_time_sync` leaves DHCP-provided NTP servers alone. chrony: only `server`,
|
||||
`pool` and `peer` lines are commented out; no `sourcedir` line is any more, and the DHCP
|
||||
client's files are left alone. systemd-timesyncd: a link's servers stay and no longer
|
||||
fail the `verify` step.
|
||||
- A write gives back a DHCP client's `sourcedir` that 4.35.0 commented out
|
||||
(`#napalm-time-sync: sourcedir /run/chrony-dhcp`), so a host aligned by 4.35.0 hears DHCP
|
||||
again; chronyd restarts for it. Another `sourcedir` 4.35.0 took out stays out.
|
||||
- `timesyncd_servers` returns the configured servers (runtime, then system) without a
|
||||
link's; the fallback only while there is no server at all.
|
||||
|
||||
## [4.35.1] - 2026-11-11
|
||||
|
||||
### Fixed
|
||||
- `apply_dhcp_reservationset()` no longer stops at the first reservation the device refuses.
|
||||
One refused write (on OPNsense, an address in a subnet Kea does not manage) ended the run
|
||||
before the commit: every later reservation went unwritten, and the earlier ones stayed
|
||||
saved but inactive, where the next diff found them configured and never committed them. A
|
||||
refused change now gets a `[failed] <mac> -> <ip>: <reason>` line, the others are written,
|
||||
any non-empty diff commits, and `DhcpApplyError` (a `RuntimeError`, exported) is raised
|
||||
after the commit with how many of how many were refused (NetOrk/netork#2381).
|
||||
|
||||
## [4.35.0] - 2026-11-11
|
||||
|
||||
### Added
|
||||
- `TimeSyncMixin`: a device's time sources and time zone. `get_time_sync()` (declared only)
|
||||
returns `TimeSyncDict` -- the configured NTP servers in the device's order, the zone as an
|
||||
IANA name (None where the device keeps no name), whether the clock is synchronised now,
|
||||
and the client (`chrony`, `systemd-timesyncd`, `ntpd`, `openntpd`; None for the device's
|
||||
own). `configure_time_sync(ntp_servers=None, timezone=None)` is a fix template: None
|
||||
leaves a value alone, the device ends up with exactly the servers given, in order, and a
|
||||
`verify` step reads it back. An unknown zone, a malformed server, an empty list or a value
|
||||
outside the driver's `time_sync_writes` is refused before anything is written.
|
||||
`TIME_SYNC_VALUES` and `check_ntp_servers` are public (NetOrk/netork#2361).
|
||||
- `PosixTimeSyncMixin` implements it over `run_command`: read in one round trip as the login
|
||||
user (`timedatectl`, chrony's configuration and `chronyc -n sources`, `timedatectl
|
||||
show-timesync`, ntpd's and OpenNTPD's configuration); the zone by `timedatectl
|
||||
set-timezone`; chrony's servers in a file of their own (`sources.d`, else a `confdir` or
|
||||
an included file) with every other source commented out, then `chronyc reload sources` or
|
||||
a restart; systemd-timesyncd's in a drop-in that resets `NTP=` and empties `FallbackNTP=`.
|
||||
ntpd and OpenNTPD are read, not written. `parse_time_sync` is public.
|
||||
- `napalm_device_types.posix_tz` (from netOrk): `posix_tz`, `is_iana_zone`, `is_posix_tz` and
|
||||
`UnknownTimezone`, and `parse_posix_tz`, which takes a POSIX rule apart into the UTC
|
||||
offset and the summer-time transitions a switch wants (`PosixTzDict`,
|
||||
`DstTransitionDict`).
|
||||
|
||||
### Changed
|
||||
- Needs `tzdata` (2024.1 or later): the zones' rules come from its TZif files.
|
||||
|
||||
## [4.34.0] - 2026-11-10
|
||||
|
||||
### Added
|
||||
- `RunningBackupsMixin.get_running_backups()`: the backups running on a device right now --
|
||||
a hypervisor's backup of its guests (with the guests it is taking now), a backup server's
|
||||
backup, sync, verify, garbage collection, prune or tape run, a host's own file-level backup.
|
||||
Declared only; a driver mixes it in when it can tell, and a driver without it cannot tell.
|
||||
`RunningBackupDict`, `RunningBackupKind` and `RUNNING_BACKUP_KINDS` are public. An
|
||||
application asks right before it restarts a device (NetOrk/netork#2318).
|
||||
|
||||
## [4.33.0] - 2026-11-07
|
||||
|
||||
### Added
|
||||
- `get_host_status()` tells an interrupted package manager on hosts without dpkg too, and
|
||||
says which one it read in the new `package_manager` field (`dpkg`, `dnf`, `yum`, `apk`,
|
||||
`opkg`, or None). dnf and yum: a package installed in two versions (not the kernels and
|
||||
other packages dnf keeps side by side) as state `duplicate`, or a transaction that never
|
||||
ended (dnf 4's history, yum's `transaction-all` journal). apk: a package apk marked with
|
||||
broken files or a broken script (`broken-files`, `broken-script`), the ones `apk fix` would
|
||||
take. opkg: a package left `unpacked`, `half-configured` or `half-installed`, or flagged for
|
||||
reinstallation. All read without root; while one of them runs the answer is unknown
|
||||
(NetOrk/netork#851).
|
||||
|
||||
## [4.32.0] - 2026-11-07
|
||||
|
||||
### Added
|
||||
- `SshServerAuthMixin.get_ssh_server_auth()`: how an OpenSSH server lets people log in --
|
||||
password, keyboard-interactive, public key, host-based and GSSAPI authentication,
|
||||
`AuthenticationMethods` as lists of methods, `PermitRootLogin` and `UsePAM`, from `sshd -T`
|
||||
as root, and sshd's PAM `auth` stack (each module with its control and arguments, includes
|
||||
followed), from `/etc/pam.d` as the login user. `OpenSshServerMixin` carries it, so every
|
||||
driver that can harden sshd has it. `parse_sshd_auth`, `parse_pam_auth`, `parse_pam_files`,
|
||||
`pam_auth_stack` and `SSHD_EFFECTIVE_COMMAND` are public (NetOrk/netork#662).
|
||||
|
||||
## [4.31.1] - 2026-11-06
|
||||
|
||||
### Fixed
|
||||
- `get_log_sources()` names as a file's writer only a process that holds it open for writing.
|
||||
A log reader -- CrowdSec, Promtail, rsyslog's `imfile`, a `tail -F` -- was given the file
|
||||
it reads as if it wrote it. A file only readers hold is now listed without a producer
|
||||
(`unit` and `process` None); one only a reader inside a container holds is left out, as its
|
||||
path is that container's view of a mount. The access mode comes from
|
||||
`/proc/<pid>/fdinfo/<fd>`, read in the same command (NetOrk/netork#2092).
|
||||
|
||||
## [4.31.0] - 2026-11-06
|
||||
|
||||
### Added
|
||||
- `get_log_sources()` also says where the logs go beyond the host: `forwarding` holds whether
|
||||
rsyslog runs, its forwarding rules (legacy `@`/`@@`/`:omrelp:` and RainerScript
|
||||
`omfwd`/`omrelp`, with selector, ruleset and whether they take every message), its `imfile`
|
||||
inputs, and whether the journal reaches it (`imjournal`, or `imuxsock` with journald's
|
||||
`ForwardToSyslog`). Read in the same command; `parse_log_forwarding` and `parse_rsyslog` are
|
||||
public (NetOrk/netork#2053).
|
||||
|
||||
## [4.30.0] - 2026-11-04
|
||||
|
||||
### Added
|
||||
- `LogSourcesMixin.get_log_sources()`: every log a Linux host writes. Sources are the units
|
||||
journald holds entries for, the log files running processes hold open (with size, last write
|
||||
and the service or container writing them), and the engine's file of each container's output.
|
||||
A file a container writes is named by the container's own path. `LOG_SOURCES_COMMAND` and
|
||||
`parse_log_sources` are public, and a host whose `find` cannot list open files (busybox)
|
||||
raises `LogSourcesUnavailable`. A driver supplies `_run_log_sources_command(command,
|
||||
privileged=)`, as for listening sockets (NetOrk/netork#2049).
|
||||
|
||||
## [4.29.0] - 2026-11-04
|
||||
|
||||
### Added
|
||||
- `remove_local_account_keys(name, keys, *, clear_password=False)`: with `clear_password`
|
||||
the account's password goes along with the keys, and so do its sudo rule and admin
|
||||
groups, as sudo asks for that password. For a person who deleted their key and whose
|
||||
password went with it. The account stays open; the steps are `keys`, `password` and
|
||||
`sudo`, each run when another failed (NetOrk/netork#2033).
|
||||
|
||||
## [4.28.0] - 2026-11-04
|
||||
|
||||
### Added
|
||||
- `LocalAccountsMixin.add_local_account(..., replaces=[...])`: the keys the new one takes
|
||||
the place of go from the account, matched by their base64 whatever a line's options or
|
||||
comment. Keys the person added on the host stay. The step is `old keys`; it runs even
|
||||
when the new key could not be added (NetOrk/netork#2013).
|
||||
- `add_local_account(..., clear_password=True)`: the account's password goes, and it logs
|
||||
in with a key only. Step `password`. It cannot be combined with a `password_hash` or
|
||||
with `admin=True`, as the sudo rule asks for that password. A driver supplies
|
||||
`_account_clear_password`; one that does not fails the step and says so.
|
||||
- `LocalAccountsMixin.remove_local_account_keys(name, keys)`: takes the named keys off an
|
||||
account and leaves it open, its password and sudo as they are -- for a person who
|
||||
deleted a key and has no new one. Step `keys`.
|
||||
- A driver whose host keeps keys elsewhere overrides `_account_drop_keys` next to
|
||||
`_account_add_key` and `_account_remove_keys`.
|
||||
|
||||
## [4.27.0] - 2026-11-03
|
||||
|
||||
### Added
|
||||
- `ScheduledJobsMixin`: what a host runs on its own schedule as one shape,
|
||||
`ScheduledJobDict` — cron lines, run-parts scripts, anacron, systemd timers, `periodic`,
|
||||
a platform's own jobs, Windows tasks — with who owns each definition (`managed_by`) and
|
||||
what the driver can do to it (`actions`). `get_scheduled_jobs` reads them with the host's
|
||||
time zone; `manage_scheduled_job` enables, disables or runs one now; `put_scheduled_job`
|
||||
and `delete_scheduled_job` create, change and delete a job an application writes. Every
|
||||
write names the `digest` the caller saw (`job_digest`) and raises `ScheduledJobChanged`
|
||||
when the host's definition differs; `ScheduledJobRefused` for a job that cannot take a
|
||||
verb (NetOrk/netork#1933, #86).
|
||||
- `PosixScheduledJobsMixin`: the implementation over the command channel. One framed report
|
||||
as root reads `/etc/crontab`, `/etc/cron.d`, the user crontabs, the run-parts directories,
|
||||
`/etc/anacrontab` and every systemd timer with the unit it starts, and which package owns
|
||||
each definition. A cron line is switched off behind `#napalm-disabled: `; an
|
||||
application's job is a timer where systemd runs, else a file in `/etc/cron.d` or marked
|
||||
lines in the user's crontab.
|
||||
- `napalm_device_types.cron`: cron expressions as Vixie cron reads them, their next runs in
|
||||
a time zone, and as systemd `OnCalendar` lines that fire on exactly the same minutes.
|
||||
|
||||
### Removed
|
||||
- `OSDriver.get_cron_jobs` and `CronJobDict`. No application called them; the readers in
|
||||
napalm-linux and napalm-bsd go with them. `get_scheduled_jobs` replaces them.
|
||||
|
||||
## [4.26.0] - 2026-11-03
|
||||
|
||||
### Added
|
||||
- `OpenSshServerMixin.SSHD_CONFIG_GENERATED`: a driver whose platform writes sshd_config
|
||||
from its own database, but includes `/etc/ssh/sshd_config.d` (OpenMediaVault), sets it
|
||||
and keeps `harden_ssh_server`. The hardening then goes only into the drop-in; a
|
||||
sshd_config without that include is left alone and the result says why, since the
|
||||
platform would write over a change to it on its next apply (#88, NetOrk/netork#1950).
|
||||
|
||||
## [4.25.0] - 2026-11-02
|
||||
|
||||
### Added
|
||||
- `BackupServerDriver` (`ROLE = "backup_server"`), a role base for backup servers such as
|
||||
Proxmox Backup Server (#82). It declares `get_backup_datastores`, `get_backup_groups`
|
||||
(each group with its newest snapshot), `get_backup_snapshots`, `get_backup_jobs`
|
||||
(garbage collection, verify, prune, sync and tape in one list) and `get_backup_remotes`,
|
||||
and the actions `start_garbage_collection`, `run_backup_job`, `verify_backup_snapshot`,
|
||||
`set_backup_snapshot_protected` and `delete_backup_snapshot`, which return a task to
|
||||
follow with `get_backup_task`. The shapes live in `napalm_device_types.backups`. A
|
||||
server that manages its own disks lists `StorageDriver` as its second role.
|
||||
- `ClusterStorageDict.backup_server` (optional): which backup server, datastore and
|
||||
namespace a hypervisor's backup storage writes to, with the certificate fingerprint it
|
||||
pins, so an application can tie a server's backup groups to a cluster's guests.
|
||||
|
||||
## [4.24.0] - 2026-11-02
|
||||
|
||||
### Added
|
||||
|
||||
@@ -47,6 +47,18 @@ UUID and NIC MACs -- what ties the BMC to the host it manages. Its sensors come
|
||||
NAPALM's `get_environment`, with `cpu` and `memory` empty, since a BMC does not know the
|
||||
host OS's load; it has no `HealthMetricsMixin` (SNMP) and no command channel.
|
||||
|
||||
A backup server (Proxmox Backup Server) is a role of its own too, `BackupServerDriver`
|
||||
(`backup_server`). Its datastores, the backup groups in them (each with its newest
|
||||
snapshot), its jobs (garbage collection, verify, prune, sync, tape) and its remotes have
|
||||
no counterpart in `StorageDriver`, so they are not bent into pools, volumes and
|
||||
replication jobs. A server that also manages its own disks and pools lists
|
||||
`StorageDriver` second: `class ProxmoxBackupDriver(BackupServerDriver, StorageDriver, ...)`.
|
||||
Its actions (garbage collection, running a job, verifying, protecting or deleting a
|
||||
snapshot) start a task and return its id at once; `get_backup_task` follows it. On the
|
||||
hypervisor side, a cluster storage that writes to a backup server says which one
|
||||
(`ClusterStorageDict.backup_server`: server, port, datastore, namespace, fingerprint), so
|
||||
an application can tie the server's groups to the cluster's guests.
|
||||
|
||||
### Role bases declare; they never implement
|
||||
|
||||
**A role base must not contain a single runtime method — not even a
|
||||
@@ -112,7 +124,8 @@ application wants made on a device runs as fixed steps the driver implements, wi
|
||||
the caller supplies: `SnmpAgentMixin.configure_snmp_agent(settings)` (install, configure,
|
||||
firewall, start), `LldpAgentMixin.configure_lldp_agent(interfaces)` (install, write the
|
||||
interfaces, enable and restart; the driver writes only the interfaces),
|
||||
`EnsurePackageMixin.ensure_package(name)`, `PackageProxyMixin.configure_package_proxy(url)` and
|
||||
`EnsurePackageMixin.ensure_package(name)`, `PackageProxyMixin.configure_package_proxy(url)`,
|
||||
`TimeSyncMixin.configure_time_sync(ntp_servers, timezone)` and
|
||||
`ContainerEngineMixin.grant_container_engine_access(user)`. A driver whose device takes the
|
||||
change in one call overrides the template method. A driver mixes in the fixes its device
|
||||
can take; none is on a role base. They replace `run_device_action`, which is deprecated.
|
||||
@@ -162,9 +175,12 @@ A driver supplies `_run_apt_command`, `_run_apt_root_command` and `_write_apt_fi
|
||||
`HostStatusMixin` (`get_host_status`) is mixed in the same way: whether a Linux host needs
|
||||
a reboot to finish an update (`/var/run/reboot-required`, `needs-restarting -r`, or a newer
|
||||
kernel of the running flavour installed), whether it patches itself (unattended-upgrades,
|
||||
dnf-automatic) and when that last succeeded (apt's `upgrade-stamp`), and whether dpkg was
|
||||
left halfway (a journal in `/var/lib/dpkg/updates` or a half-configured package; unknown
|
||||
while a package manager is running), and how full `/`, `/boot` and `/var` are (`df -Pk`,
|
||||
dnf-automatic) and when that last succeeded (apt's `upgrade-stamp`), and whether its package
|
||||
manager was left halfway (unknown while one is running): dpkg (a journal in
|
||||
`/var/lib/dpkg/updates` or a half-configured package), dnf or yum (two versions of a package
|
||||
in the rpm database, a transaction that never ended), apk (a package marked with broken files
|
||||
or a broken script) or opkg (a package left `unpacked`), with `package_manager` naming which,
|
||||
and how full `/`, `/boot` and `/var` are (`df -Pk`,
|
||||
each filesystem once, in bytes; how full is too full is the caller's decision). `package_updates` holds the shared apt and dnf parsers: apt's suites become
|
||||
an update's `origin`, a `-security` suite makes it a security update, and dnf's security
|
||||
advisories do the same.
|
||||
@@ -179,6 +195,23 @@ without `ss` is read with `netstat -lntup` (OpenWrt's busybox, old net-tools); o
|
||||
cgroup names the procd service (`/services/<name>/<instance>`). A host with neither raises
|
||||
`ListeningSocketsUnavailable`.
|
||||
|
||||
`LogSourcesMixin` (`get_log_sources`) is mixed in the same way: every log the host writes, in
|
||||
one round trip. The units journald holds entries for (`journalctl -F _SYSTEMD_UNIT`, services
|
||||
only), the log files running processes hold open (`/proc/<pid>/fd`: `*.log`, `*.log.N` and
|
||||
anything under `/var/log`) with size and last write from `stat -L` on the descriptor, and the
|
||||
engine's own file of a container's output (`/var/lib/docker/containers/<id>/…`) as that
|
||||
container's stream. Each file names the service or container of its first writer from
|
||||
`/proc/<pid>/cgroup`; a file a container writes carries the container's own path. journald's
|
||||
store, deleted files and a database's write-ahead log (LevelDB's `000004.log`) are left out; a
|
||||
host whose `find` cannot list open files (busybox) raises `LogSourcesUnavailable`. A
|
||||
driver supplies `_run_log_sources_command(command, privileged=)`; without root only the login
|
||||
user's processes show their files, and the reading says so (`attributed: false`).
|
||||
The same round trip reads where the logs go beyond the host (`forwarding`,
|
||||
`napalm_device_types.log_forwarding`): whether rsyslog runs, its forwarding rules (`@`, `@@`,
|
||||
`:omrelp:`, `action(type="omfwd"|"omrelp")`, each *complete* only for `*.*` with no condition
|
||||
around it), the files it reads in (`imfile`), and whether the journal reaches it (`imjournal`,
|
||||
or `imuxsock` with journald's `ForwardToSyslog`, the distribution's commented default counting).
|
||||
|
||||
LLDP neighbours are rows: `LldpNeighborRow` is one neighbour as its local port heard it
|
||||
(chassis and port id with their type, name, description, management addresses,
|
||||
capabilities in NAPALM's names), and `lldp_neighbors(rows)` / `lldp_neighbors_detail(rows,
|
||||
@@ -266,6 +299,16 @@ A driver mixes it in and supplies `run_command`; a platform that regenerates ssh
|
||||
itself (an appliance with its own database) must not, or opts out with
|
||||
`harden_ssh_server = None`.
|
||||
|
||||
`SshServerAuthMixin` (`get_ssh_server_auth`) says how an OpenSSH server lets people log in:
|
||||
password, keyboard-interactive, public key, host-based and GSSAPI authentication on or off,
|
||||
`AuthenticationMethods` as lists of methods (empty for `any`), `PermitRootLogin` and
|
||||
`UsePAM` (None where sshd has no PAM), from `sshd -T` as root -- the same fixed command the
|
||||
hardening reads. With PAM on, it also names the modules of sshd's PAM `auth` stack
|
||||
with each control and its arguments, read from `/etc/pam.d` in one listing as the
|
||||
login user and followed through `@include`, `include` and `substack`. Which of them is a
|
||||
second factor is the consumer's call. `Match` blocks are not read. `OpenSshServerMixin`
|
||||
carries it; a driver can also mix it in alone and supply `run_command`.
|
||||
|
||||
`SudoHistoryMixin` (`get_history_sudo_lines`, `remove_history_sudo_lines`) deals with what
|
||||
a driver left behind when it typed `echo '<password>' | sudo -S -p "" …` into an interactive
|
||||
shell: that line, password and all, in the login user's shell history, and files its partial
|
||||
@@ -289,6 +332,48 @@ trip (`list-unit-files` plus one `systemctl show` over every loaded unit) instea
|
||||
`is-enabled` and a `show` per unit. A host without systemd raises `SystemdUnavailable`,
|
||||
a `NotImplementedError`, so a driver can fall back to another init system.
|
||||
|
||||
`ScheduledJobsMixin` (`get_scheduled_jobs`, `manage_scheduled_job`, `put_scheduled_job`,
|
||||
`delete_scheduled_job`) is what a host runs on its own schedule — cron lines, run-parts
|
||||
scripts, anacron, systemd timers, `periodic`, a platform's own jobs, Windows tasks — as one
|
||||
shape, `ScheduledJobDict`. Every job says who owns its definition (`managed_by`: package,
|
||||
admin, platform, application) and what the driver can do to it (`actions`): every job a
|
||||
driver can switch takes `enable`, `disable` and `run`; only a job an application wrote
|
||||
through `put_scheduled_job` takes `edit` and `delete`. Every write names the `digest` the
|
||||
caller saw and raises `ScheduledJobChanged` when the host's definition is no longer that.
|
||||
`PosixScheduledJobsMixin` implements it over `run_command`: one framed report as root,
|
||||
a cron line switched off behind `#napalm-disabled: `, a timer by `systemctl`, an
|
||||
application's job as a timer where systemd runs, else in `/etc/cron.d` or the user's
|
||||
crontab. A driver sets the paths its host keeps jobs in. `napalm_device_types.cron` parses
|
||||
cron expressions, gives their next runs and writes them as `OnCalendar` lines.
|
||||
|
||||
`TimeSyncMixin` (`get_time_sync`, `configure_time_sync`) is how a device keeps its clock:
|
||||
the NTP servers it is configured with, in its order, its time zone as an IANA name, whether
|
||||
it is synchronised now and which client does it (`TimeSyncDict`). The servers it got by
|
||||
DHCP (option 42) and uses are apart, in `dhcp_ntp_servers` (`[]` for none, left out where
|
||||
the driver cannot tell): DHCP counts, a device that gets the site's time giver that way is
|
||||
aligned. `get_time_sync` is declared only; `configure_time_sync(ntp_servers=None,
|
||||
timezone=None)` is a fix template -- `None` leaves a value alone, the device ends up
|
||||
configured with exactly the servers given, in that order, its DHCP-provided servers left
|
||||
alone, and a `verify` step reads it back. A zone that is not in the IANA database, a
|
||||
malformed server, an empty list or a value outside the driver's `time_sync_writes` is
|
||||
refused before anything is written. A driver states in `time_sync_writes` which of
|
||||
`ntp_servers` and `timezone` it can write (a BMC that runs on UTC only: `ntp_servers`).
|
||||
`napalm_device_types.posix_tz` turns an IANA name into the POSIX rule libc wants in `TZ`
|
||||
(OpenWrt) and that rule into a UTC offset and summer-time transitions (a switch), from the
|
||||
`tzdata` package. `PosixTimeSyncMixin` implements it over `run_command`: one reading as the
|
||||
login user (`timedatectl show`, the active client: chrony's configuration and `chronyc -n
|
||||
sources`, `timedatectl show-timesync`, `ntp.conf`, OpenNTPD's `ntpd.conf`); the zone by
|
||||
`timedatectl set-timezone`; chrony's servers in a file of their own --
|
||||
`/etc/chrony/sources.d/netork.sources` where chrony reads that directory, else a `confdir`
|
||||
or an included `/etc/chrony.d/netork.conf` -- with every other `server`, `pool` and `peer`
|
||||
line commented out behind `#napalm-time-sync: `, then `chronyc reload sources` or a
|
||||
restart; systemd-timesyncd's in `/etc/systemd/timesyncd.conf.d/netork.conf` (`NTP=` reset,
|
||||
`FallbackNTP=` emptied), then a restart. What DHCP brings is read apart and kept: chrony's
|
||||
DHCP `sourcedir` (`CHRONY_DHCP_DIRS`, `/run/chrony-dhcp`) is never commented out -- one
|
||||
that 4.35.0 commented out comes back on the next write -- and systemd-timesyncd's
|
||||
`LinkNTPServers` stay. ntpd and OpenNTPD are read, not written; what DHCP gives them is
|
||||
unknown.
|
||||
|
||||
### Access channels: why there is no container model here
|
||||
|
||||
`CommandChannelMixin` declares a driver's public channel to its host:
|
||||
@@ -461,6 +546,7 @@ Requires Python ≥ 3.12 (tested up to 3.13) and NAPALM ≥ 4.0.
|
||||
| `PhoneDriver` | IP telephony endpoints | Yealink |
|
||||
| `MediaDriver` | Speakers, streamers, media renderers | Sonos, Chromecast |
|
||||
| `ManagementControllerDriver` | Server baseboard management controllers | Supermicro BMC, Dell iDRAC, HPE iLO, Lenovo XCC |
|
||||
| `BackupServerDriver` | Backup servers: datastores, backup groups, GC/verify/prune/sync/tape jobs | Proxmox Backup Server |
|
||||
|
||||
Mixins that come with the classes above: `PingSweepMixin` (subnet sweeps, in
|
||||
every driver through `DeviceTypeDriver`) and `DhcpServerMixin` (static DHCP
|
||||
@@ -608,10 +694,6 @@ class LinuxDriver(OSDriver):
|
||||
def get_processes(self):
|
||||
# return List[ProcessDict]
|
||||
...
|
||||
|
||||
def get_cron_jobs(self):
|
||||
# return List[CronJobDict]
|
||||
...
|
||||
```
|
||||
|
||||
### Storage / NAS
|
||||
|
||||
+2
-1
@@ -35,7 +35,7 @@ application SNMP polling, ping sweeps, discovery scoring, guest
|
||||
|
||||
| Kind | Rule | Modules |
|
||||
|---|---|---|
|
||||
| **Role** | Declares only, under `TYPE_CHECKING`. No runtime method. | `access_point`, `switch`, `firewall`, `hypervisor`, `os`, `storage`, `phone`, `media`, `residential_gateway`, `management_controller` |
|
||||
| **Role** | Declares only, under `TYPE_CHECKING`. No runtime method. | `access_point`, `switch`, `firewall`, `hypervisor`, `os`, `storage`, `phone`, `media`, `residential_gateway`, `management_controller`, `backup_server` |
|
||||
| **Capability** | Declares only. | `packages`, `services`, `updates`, `nat_vpn`, `mac_acl`, `config_lifecycle`, `interface_filter`, `host_reboot` |
|
||||
| **Template** | A concrete method that runs a fixed sequence of abstract steps. A driver implements the steps, or overrides the whole method when its device does it in one go. All values come from the caller. | the fix routines below |
|
||||
| **Contract types** | Interfaces and data shapes other packages implement or fill. | `CommandChannelMixin`, `CommandResult`, `ByteStream`; the declaration of `ContainerEngineMixin`; the reader mixins' public method declarations; the SNMP profile type; `models` |
|
||||
@@ -151,6 +151,7 @@ the driver.
|
||||
| Module | Knows |
|
||||
|---|---|
|
||||
| `host_status` | apt, dnf, dpkg, systemd timers, `df` |
|
||||
| `package_managers` | rpm, dnf and yum history, apk's and opkg's databases, `/proc/<pid>/comm` |
|
||||
| `kernel` | `/proc`, `/lib/modules`, gzip, base64 |
|
||||
| `listening` | `ss`, `netstat`, `/proc/<pid>/cgroup` |
|
||||
| `systemd` | `systemctl` |
|
||||
|
||||
@@ -22,6 +22,7 @@ driver do X".
|
||||
Role bases -- what a device *is*:
|
||||
|
||||
* :class:`~napalm_device_types.access_point.AccessPointDriver`
|
||||
* :class:`~napalm_device_types.backup_server.BackupServerDriver`
|
||||
* :class:`~napalm_device_types.switch.SwitchDriver`
|
||||
* :class:`~napalm_device_types.firewall.FirewallDriver`
|
||||
* :class:`~napalm_device_types.hypervisor.HypervisorDriver`
|
||||
@@ -49,16 +50,23 @@ instead of being restated on every role that happens to need it:
|
||||
* :class:`~napalm_device_types.kernel.KernelFactsMixin`
|
||||
* :class:`~napalm_device_types.listening.ListeningSocketsMixin`
|
||||
* :class:`~napalm_device_types.local_accounts.LocalAccountsMixin`
|
||||
* :class:`~napalm_device_types.log_sources.LogSourcesMixin`
|
||||
* :class:`~napalm_device_types.host_firewall.HostFirewallMixin`
|
||||
* :class:`~napalm_device_types.mac_acl.MacAclMixin`
|
||||
* :class:`~napalm_device_types.nat_vpn.NatVpnMixin`
|
||||
* :class:`~napalm_device_types.packages.PackageManagementMixin`
|
||||
* :class:`~napalm_device_types.ping_sweep.PingSweepMixin`
|
||||
* :class:`~napalm_device_types.running_backups.RunningBackupsMixin`
|
||||
* :class:`~napalm_device_types.service_clusters.ServiceClustersMixin`
|
||||
* :class:`~napalm_device_types.service_packages.ServicePackagesMixin`
|
||||
* :class:`~napalm_device_types.ssh_server_auth.SshServerAuthMixin`
|
||||
* :class:`~napalm_device_types.sudo_history.SudoHistoryMixin`
|
||||
* :class:`~napalm_device_types.scheduled_jobs.ScheduledJobsMixin`
|
||||
* :class:`~napalm_device_types.scheduled_jobs_posix.PosixScheduledJobsMixin`
|
||||
* :class:`~napalm_device_types.services.ServiceControlMixin`
|
||||
* :class:`~napalm_device_types.systemd.SystemdServicesMixin`
|
||||
* :class:`~napalm_device_types.time_sync.TimeSyncMixin`
|
||||
* :class:`~napalm_device_types.time_sync_posix.PosixTimeSyncMixin`
|
||||
* :class:`~napalm_device_types.updates.UpdateMixin`
|
||||
* :class:`~napalm_device_types.virtual_ips.VirtualIpsMixin`
|
||||
* :class:`~napalm_device_types.vm_stop_request.VmStopRequestMixin`
|
||||
@@ -78,6 +86,22 @@ from napalm_device_types.apt_proxy import (
|
||||
same_proxy_url,
|
||||
)
|
||||
from napalm_device_types.arp import ARP_AGE_UNKNOWN
|
||||
from napalm_device_types.backup_server import BackupServerDriver
|
||||
from napalm_device_types.backups import (
|
||||
BACKUP_JOB_KINDS,
|
||||
BackupDatastoreDict,
|
||||
BackupGroupDict,
|
||||
BackupJobKind,
|
||||
BackupRemoteDict,
|
||||
BackupRunState,
|
||||
BackupServerJobDict,
|
||||
BackupServerRefDict,
|
||||
BackupSnapshotDict,
|
||||
BackupSnapshotRefDict,
|
||||
BackupTaskDict,
|
||||
BackupTaskStatusDict,
|
||||
BackupVerifyState,
|
||||
)
|
||||
from napalm_device_types.base import DeviceTypeDriver, FingerprintRule, PortSpec, rules_max_weight
|
||||
from napalm_device_types.channel import (
|
||||
ByteStream,
|
||||
@@ -103,7 +127,7 @@ from napalm_device_types.container_engine import (
|
||||
ContainerEngineMixin,
|
||||
ContainerEngineUnavailable,
|
||||
)
|
||||
from napalm_device_types.dhcp import DhcpServerMixin, normalize_cidr, normalize_mac
|
||||
from napalm_device_types.dhcp import DhcpApplyError, DhcpServerMixin, normalize_cidr, normalize_mac
|
||||
from napalm_device_types.dns_forward import DnsForwardMixin
|
||||
from napalm_device_types.firewall import FirewallDriver
|
||||
from napalm_device_types.firewall_rules import FirewallRuleMixin
|
||||
@@ -147,6 +171,21 @@ from napalm_device_types.local_accounts import (
|
||||
LocalAccountDict,
|
||||
LocalAccountsMixin,
|
||||
)
|
||||
from napalm_device_types.log_forwarding import (
|
||||
FileInputDict,
|
||||
ForwardRuleDict,
|
||||
LogForwardingDict,
|
||||
parse_rsyslog,
|
||||
)
|
||||
from napalm_device_types.log_sources import (
|
||||
LOG_SOURCES_COMMAND,
|
||||
LogSourceDict,
|
||||
LogSourcesDict,
|
||||
LogSourcesMixin,
|
||||
LogSourcesUnavailable,
|
||||
parse_log_forwarding,
|
||||
parse_log_sources,
|
||||
)
|
||||
from napalm_device_types.mac_acl import MacAclMixin
|
||||
from napalm_device_types.management_controller import ManagementControllerDriver
|
||||
from napalm_device_types.media import MediaDriver
|
||||
@@ -185,9 +224,35 @@ from napalm_device_types.package_updates import (
|
||||
from napalm_device_types.packages import PackageManagementMixin
|
||||
from napalm_device_types.phone import PhoneDriver
|
||||
from napalm_device_types.ping_sweep import PingSweepMixin, driver_supports_ping
|
||||
from napalm_device_types.posix_tz import (
|
||||
DstTransitionDict,
|
||||
PosixTzDict,
|
||||
UnknownTimezone,
|
||||
is_iana_zone,
|
||||
is_posix_tz,
|
||||
parse_posix_tz,
|
||||
posix_tz,
|
||||
)
|
||||
from napalm_device_types.provisioning import VMProvisionIncomplete
|
||||
from napalm_device_types.residential_gateway import ResidentialGatewayDriver
|
||||
from napalm_device_types.roles import primary_role_of, role_keys_of, roles_of
|
||||
from napalm_device_types.running_backups import (
|
||||
RUNNING_BACKUP_KINDS,
|
||||
RunningBackupDict,
|
||||
RunningBackupKind,
|
||||
RunningBackupsMixin,
|
||||
)
|
||||
from napalm_device_types.scheduled_jobs import (
|
||||
ScheduledJobChanged,
|
||||
ScheduledJobDict,
|
||||
ScheduledJobRefused,
|
||||
ScheduledJobResultDict,
|
||||
ScheduledJobsMixin,
|
||||
ScheduledJobSpecDict,
|
||||
ScheduledJobsReadingDict,
|
||||
job_digest,
|
||||
)
|
||||
from napalm_device_types.scheduled_jobs_posix import PosixScheduledJobsMixin
|
||||
from napalm_device_types.service_clusters import (
|
||||
SERVICE_CLUSTERS_COMMAND,
|
||||
ServiceClustersMixin,
|
||||
@@ -225,6 +290,14 @@ from napalm_device_types.ssh_server import (
|
||||
SshHardeningResult,
|
||||
plan_ssh_hardening,
|
||||
)
|
||||
from napalm_device_types.ssh_server_auth import (
|
||||
SSHD_EFFECTIVE_COMMAND,
|
||||
PamAuthEntryDict,
|
||||
SshServerAuthDict,
|
||||
SshServerAuthMixin,
|
||||
pam_auth_stack,
|
||||
parse_sshd_auth,
|
||||
)
|
||||
from napalm_device_types.storage import StorageDriver
|
||||
from napalm_device_types.sudo_history import (
|
||||
HistoryCleanupDict,
|
||||
@@ -246,6 +319,13 @@ from napalm_device_types.systemd import (
|
||||
parse_systemd_services,
|
||||
)
|
||||
from napalm_device_types.terminal import strip_terminal_codes
|
||||
from napalm_device_types.time_sync import (
|
||||
TIME_SYNC_VALUES,
|
||||
TimeSyncDict,
|
||||
TimeSyncMixin,
|
||||
check_ntp_servers,
|
||||
)
|
||||
from napalm_device_types.time_sync_posix import PosixTimeSyncMixin, parse_time_sync
|
||||
from napalm_device_types.updates import UpdateMixin
|
||||
from napalm_device_types.virtual_ips import (
|
||||
VIRTUAL_IPS_COMMAND,
|
||||
@@ -256,6 +336,19 @@ from napalm_device_types.vm_migration import MigrationPreflightDict
|
||||
from napalm_device_types.vm_stop_request import VmStopRequestMixin
|
||||
|
||||
__all__ = [
|
||||
"BACKUP_JOB_KINDS",
|
||||
"BackupDatastoreDict",
|
||||
"BackupGroupDict",
|
||||
"BackupJobKind",
|
||||
"BackupRemoteDict",
|
||||
"BackupRunState",
|
||||
"BackupServerJobDict",
|
||||
"BackupServerRefDict",
|
||||
"BackupSnapshotDict",
|
||||
"BackupSnapshotRefDict",
|
||||
"BackupTaskDict",
|
||||
"BackupTaskStatusDict",
|
||||
"BackupVerifyState",
|
||||
"BackupDict",
|
||||
"BackupJobDict",
|
||||
"ClusterDict",
|
||||
@@ -295,6 +388,7 @@ __all__ = [
|
||||
"SnmpMemorySpec",
|
||||
"SnmpValueSpec",
|
||||
"AccessPointDriver",
|
||||
"BackupServerDriver",
|
||||
"ByteStream",
|
||||
"CommandChannelMixin",
|
||||
"CommandResult",
|
||||
@@ -310,6 +404,7 @@ __all__ = [
|
||||
"DhcpHaDict",
|
||||
"DhcpHaPeerDict",
|
||||
"DeviceTypeDriver",
|
||||
"DhcpApplyError",
|
||||
"DhcpServerMixin",
|
||||
"FingerprintRule",
|
||||
"rules_max_weight",
|
||||
@@ -354,6 +449,17 @@ __all__ = [
|
||||
"parse_host_firewall",
|
||||
"ListeningSocketsMixin",
|
||||
"ListeningSocketsUnavailable",
|
||||
"LOG_SOURCES_COMMAND",
|
||||
"FileInputDict",
|
||||
"ForwardRuleDict",
|
||||
"LogForwardingDict",
|
||||
"parse_log_forwarding",
|
||||
"parse_rsyslog",
|
||||
"LogSourceDict",
|
||||
"LogSourcesDict",
|
||||
"LogSourcesMixin",
|
||||
"LogSourcesUnavailable",
|
||||
"parse_log_sources",
|
||||
"parse_listening_sockets",
|
||||
"LLDPCTL_KEYVALUE_COMMAND",
|
||||
"LldpNeighborDict",
|
||||
@@ -369,6 +475,15 @@ __all__ = [
|
||||
"PortSpec",
|
||||
"ResidentialGatewayDriver",
|
||||
"ServiceControlMixin",
|
||||
"PosixScheduledJobsMixin",
|
||||
"ScheduledJobChanged",
|
||||
"ScheduledJobDict",
|
||||
"ScheduledJobRefused",
|
||||
"ScheduledJobResultDict",
|
||||
"ScheduledJobSpecDict",
|
||||
"ScheduledJobsMixin",
|
||||
"ScheduledJobsReadingDict",
|
||||
"job_digest",
|
||||
"StorageDriver",
|
||||
"SwitchDriver",
|
||||
"SYSTEMD_SERVICES_COMMAND",
|
||||
@@ -379,6 +494,23 @@ __all__ = [
|
||||
"VIRTUAL_IPS_COMMAND",
|
||||
"VirtualIpsMixin",
|
||||
"VmStopRequestMixin",
|
||||
"RUNNING_BACKUP_KINDS",
|
||||
"RunningBackupDict",
|
||||
"RunningBackupKind",
|
||||
"RunningBackupsMixin",
|
||||
"TIME_SYNC_VALUES",
|
||||
"TimeSyncDict",
|
||||
"TimeSyncMixin",
|
||||
"check_ntp_servers",
|
||||
"PosixTimeSyncMixin",
|
||||
"parse_time_sync",
|
||||
"DstTransitionDict",
|
||||
"PosixTzDict",
|
||||
"UnknownTimezone",
|
||||
"is_iana_zone",
|
||||
"is_posix_tz",
|
||||
"parse_posix_tz",
|
||||
"posix_tz",
|
||||
"VMProvisionIncomplete",
|
||||
"SERVICE_CLUSTERS_COMMAND",
|
||||
"ServiceClustersMixin",
|
||||
@@ -389,6 +521,12 @@ __all__ = [
|
||||
"is_placeholder_identity",
|
||||
"parse_system_identity",
|
||||
"OpenSshServerMixin",
|
||||
"PamAuthEntryDict",
|
||||
"SshServerAuthDict",
|
||||
"SshServerAuthMixin",
|
||||
"SSHD_EFFECTIVE_COMMAND",
|
||||
"pam_auth_stack",
|
||||
"parse_sshd_auth",
|
||||
"SSH_ALGORITHM_CLASSES",
|
||||
"SshHardeningPlan",
|
||||
"SshHardeningResult",
|
||||
|
||||
@@ -0,0 +1,189 @@
|
||||
"""How chrony is given exactly the time sources a caller names.
|
||||
|
||||
The sources go to one file of their own (:func:`plan_chrony` picks it):
|
||||
|
||||
1. the sources file in a ``sourcedir`` the main configuration reads (Debian's
|
||||
``/etc/chrony/sources.d``): ``chronyc reload sources`` takes it without a restart;
|
||||
2. else ``netork.conf`` -- the include file's name -- in the first ``confdir``;
|
||||
3. else the include file, where an ``include`` pattern already takes it (SUSE's
|
||||
``/etc/chrony.d/*.conf``);
|
||||
4. else the include file, with an ``include`` line appended to the main configuration
|
||||
(RHEL's ``/etc/chrony.conf``).
|
||||
|
||||
Every other configured source stops counting: a ``server``, ``pool`` or ``peer``
|
||||
line in any other file chrony reads is commented out behind :data:`COMMENT_MARK`.
|
||||
``sourcedir`` lines stay. The DHCP client's directory (``/run/chrony-dhcp``) and its
|
||||
files are left alone: DHCP option 42 counts (NetOrk/netork#2380), and the client
|
||||
rewrites those files on every lease anyway.
|
||||
|
||||
4.35.0 commented out every ``sourcedir`` other than the sources file's, the DHCP
|
||||
client's among them, which left a host deaf to DHCP. A write gives a DHCP client's
|
||||
``sourcedir`` back where it finds one behind :data:`COMMENT_MARK`; any other one it
|
||||
took out stays out, its sources would be commented out now anyway.
|
||||
|
||||
chronyd reads its main configuration only when it starts: a change there means a
|
||||
restart.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import posixpath
|
||||
import re
|
||||
import shlex
|
||||
from dataclasses import dataclass, field
|
||||
from fnmatch import fnmatchcase
|
||||
|
||||
from napalm_device_types._time_sync_reading import (
|
||||
CHRONY_DHCP_DIRS,
|
||||
SOURCE_LINE,
|
||||
ConfigFile,
|
||||
in_dhcp_dir,
|
||||
)
|
||||
|
||||
#: What a source line taken out of chrony's configuration starts with.
|
||||
COMMENT_MARK = "#napalm-time-sync: "
|
||||
#: The comment above an ``include`` line appended to the main configuration.
|
||||
INCLUDE_COMMENT = "# napalm-time-sync: the time sources"
|
||||
|
||||
_DIRECTORY_LINE = re.compile(r"\s*(sourcedir|confdir|include)\s+(.+?)\s*$")
|
||||
#: A ``sourcedir`` line 4.35.0 commented out.
|
||||
_MARKED_SOURCEDIR = re.compile(rf"{re.escape(COMMENT_MARK)}sourcedir\s+(.+?)\s*$")
|
||||
|
||||
|
||||
@dataclass
|
||||
class ChronyPlan:
|
||||
"""What :func:`plan_chrony` decided."""
|
||||
|
||||
target: str # the file the sources go to
|
||||
content: str
|
||||
comment_out: dict[str, list[int]] = field(default_factory=dict) # path: line numbers
|
||||
restore: dict[str, list[int]] = field(default_factory=dict) # path: marked sourcedir lines
|
||||
include_in: str | None = None # the main configuration an include of target is appended to
|
||||
reload: bool = False # True: ``chronyc reload sources`` takes the change
|
||||
|
||||
|
||||
def source_line(server: str) -> str:
|
||||
"""The line a server is written as: a pool's name as a ``pool``, anything else a ``server``."""
|
||||
directive = "pool" if ".pool." in f".{server.lower()}." else "server"
|
||||
return f"{directive} {server} iburst"
|
||||
|
||||
|
||||
def _directives(main: ConfigFile) -> list[tuple[int, str, list[str]]]:
|
||||
found = []
|
||||
for number, line in enumerate(main.lines, start=1):
|
||||
match = _DIRECTORY_LINE.match(line)
|
||||
if match:
|
||||
found.append((number, match[1], match[2].split()))
|
||||
return found
|
||||
|
||||
|
||||
def _arguments(directives: list[tuple[int, str, list[str]]], kind: str) -> list[str]:
|
||||
return [arg for _, k, args in directives if k == kind for arg in args]
|
||||
|
||||
|
||||
def _target(main: ConfigFile, sources_file: str, include_file: str) -> tuple[str, str | None, bool]:
|
||||
"""The file the sources go to, the main configuration to include it from, and
|
||||
whether it is a sources file."""
|
||||
directives = _directives(main)
|
||||
if posixpath.dirname(sources_file) in _arguments(directives, "sourcedir"):
|
||||
return sources_file, None, True
|
||||
confdirs = _arguments(directives, "confdir")
|
||||
if confdirs:
|
||||
return posixpath.join(confdirs[0], posixpath.basename(include_file)), None, False
|
||||
if any(fnmatchcase(include_file, p) for p in _arguments(directives, "include")):
|
||||
return include_file, None, False
|
||||
return include_file, main.path, False
|
||||
|
||||
|
||||
def _dhcp_sourcedirs_to_restore(main: ConfigFile, dhcp_dirs: tuple[str, ...]) -> list[int]:
|
||||
"""The lines of *main* where 4.35.0 commented out a DHCP client's ``sourcedir``."""
|
||||
dhcp = {posixpath.normpath(d) for d in dhcp_dirs}
|
||||
return [
|
||||
number
|
||||
for number, line in enumerate(main.lines, start=1)
|
||||
if (marked := _MARKED_SOURCEDIR.match(line))
|
||||
and all(posixpath.normpath(arg) in dhcp for arg in marked[1].split())
|
||||
]
|
||||
|
||||
|
||||
def plan_chrony(
|
||||
files: list[ConfigFile],
|
||||
servers: list[str],
|
||||
*,
|
||||
sources_file: str,
|
||||
include_file: str,
|
||||
dhcp_dirs: tuple[str, ...] = CHRONY_DHCP_DIRS,
|
||||
) -> ChronyPlan:
|
||||
"""Where *servers* go, what else is commented out, and which DHCP ``sourcedir``
|
||||
(one of *dhcp_dirs*) comes back.
|
||||
|
||||
:raises RuntimeError: when the reading found no chrony configuration.
|
||||
"""
|
||||
main = next((f for f in files if f.kind == "main"), None)
|
||||
if main is None:
|
||||
raise RuntimeError("chrony runs, but no chrony configuration was found to write to")
|
||||
target, include_in, is_sources = _target(main, sources_file, include_file)
|
||||
comment_out = _comment_out(files, target, dhcp_dirs)
|
||||
restore = _dhcp_sourcedirs_to_restore(main, dhcp_dirs)
|
||||
return ChronyPlan(
|
||||
target=target,
|
||||
content="".join(f"{source_line(s)}\n" for s in servers),
|
||||
comment_out=comment_out,
|
||||
restore={main.path: restore} if restore else {},
|
||||
include_in=include_in,
|
||||
reload=(
|
||||
is_sources
|
||||
and include_in is None
|
||||
and not restore
|
||||
and _only_sources_files(files, comment_out)
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def _only_sources_files(files: list[ConfigFile], changed: dict[str, list[int]]) -> bool:
|
||||
"""Whether every file that changes is one ``chronyc reload sources`` reads again."""
|
||||
return all(f.kind == "sourcedir" for f in files if f.path in changed)
|
||||
|
||||
|
||||
def _comment_out(
|
||||
files: list[ConfigFile], target: str, dhcp_dirs: tuple[str, ...]
|
||||
) -> dict[str, list[int]]:
|
||||
"""Per file, the lines that take a source; the DHCP client's files are left alone."""
|
||||
found: dict[str, list[int]] = {}
|
||||
for f in files:
|
||||
if f.path == target or in_dhcp_dir(f, dhcp_dirs):
|
||||
continue
|
||||
numbers = [n for n, line in enumerate(f.lines, start=1) if SOURCE_LINE.match(line)]
|
||||
if numbers:
|
||||
found[f.path] = numbers
|
||||
return found
|
||||
|
||||
|
||||
def comment_out_command(path: str, numbers: list[int]) -> str:
|
||||
"""``sed`` that comments out those lines of *path* that still hold a source."""
|
||||
edits = " ".join(
|
||||
"-e "
|
||||
+ shlex.quote(f"{n}{{/^[[:space:]]*(server|pool|peer)[[:space:]]/s/^/{COMMENT_MARK}/;}}")
|
||||
for n in numbers
|
||||
)
|
||||
return f"sed -i -E {edits} {shlex.quote(path)}"
|
||||
|
||||
|
||||
def restore_command(path: str, numbers: list[int]) -> str:
|
||||
"""``sed`` that takes :data:`COMMENT_MARK` off those lines of *path* that are still a
|
||||
``sourcedir`` behind it. The mark holds nothing a regular expression or ``sed``'s
|
||||
``s///`` would read as special."""
|
||||
edits = " ".join(
|
||||
"-e " + shlex.quote(f"{n}{{s/^{COMMENT_MARK}(sourcedir[[:space:]])/\\1/;}}")
|
||||
for n in numbers
|
||||
)
|
||||
return f"sed -i -E {edits} {shlex.quote(path)}"
|
||||
|
||||
|
||||
def include_command(main: str, target: str) -> str:
|
||||
"""Append an ``include`` of *target* to *main*, on a line of its own."""
|
||||
lines = " ".join(shlex.quote(line) for line in (INCLUDE_COMMENT, f"include {target}"))
|
||||
return (
|
||||
f'{{ [ -z "$(tail -c 1 -- {shlex.quote(main)})" ] || echo; '
|
||||
f"printf '%s\\n' {lines}; }} >> {shlex.quote(main)}"
|
||||
)
|
||||
@@ -0,0 +1,222 @@
|
||||
"""Crontabs and anacrontabs as text: the jobs in them, and changing one line of them.
|
||||
|
||||
Pure: no I/O. :mod:`napalm_device_types.scheduled_jobs_posix` reads the files and
|
||||
writes them back; this module says what is in them and what the new text is.
|
||||
|
||||
**A line's key survives switching it off.** A job switched off is the same line
|
||||
behind the marker :data:`DISABLED_MARKER`; the key is built from the line without
|
||||
it, so the job keeps its key and the marker is all that changes. Two equal lines
|
||||
in one file get ``:2``, ``:3`` after the first.
|
||||
|
||||
**A job an application wrote** is the line after ``# napalm-scheduled-job: <name>``
|
||||
and is keyed ``job:<name>``, whatever its schedule.
|
||||
|
||||
**Everything else is left as it was**: comments, blank lines, variables, lines
|
||||
cron would not take.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import re
|
||||
from dataclasses import dataclass, field
|
||||
from shlex import quote
|
||||
|
||||
DISABLED_MARKER = "#napalm-disabled: "
|
||||
JOB_MARKER = "# napalm-scheduled-job: "
|
||||
|
||||
_VARIABLE = re.compile(r"^([A-Za-z_][A-Za-z0-9_]*)\s*=\s*(.*)$")
|
||||
_ANACRON_PERIOD = re.compile(r"^(\d+|@(daily|weekly|monthly|yearly|annually))$")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CronLine:
|
||||
"""One job in a crontab: where it is, what it says, and what was set before it."""
|
||||
|
||||
index: int # the line's number in the text, from 0
|
||||
key: str
|
||||
schedule: str
|
||||
run_as: str
|
||||
command: str
|
||||
enabled: bool
|
||||
env: dict[str, str] = field(default_factory=dict)
|
||||
name: str = "" # the name an application gave it
|
||||
description: str = ""
|
||||
|
||||
|
||||
def _unquote(value: str) -> str:
|
||||
value = value.strip()
|
||||
if len(value) >= 2 and value[0] == value[-1] and value[0] in "'\"":
|
||||
return value[1:-1]
|
||||
return value
|
||||
|
||||
|
||||
def _digest(text: str) -> str:
|
||||
return hashlib.sha256(" ".join(text.split()).encode()).hexdigest()[:8]
|
||||
|
||||
|
||||
def _split(content: str, *, system: bool, user: str) -> tuple[str, str, str] | None:
|
||||
"""(schedule, run_as, command) of a cron line; None for one cron would not take."""
|
||||
fields = 1 if content.startswith("@") else 5
|
||||
owner = 1 if system else 0
|
||||
parts = content.split(None, fields + owner)
|
||||
if len(parts) < fields + owner + 1:
|
||||
return None
|
||||
schedule = " ".join(parts[:fields])
|
||||
run_as = parts[fields] if system else user
|
||||
return schedule, run_as, parts[-1]
|
||||
|
||||
|
||||
def _anacron_split(content: str) -> tuple[str, str, str] | None:
|
||||
"""(period, job identifier, command) of an anacrontab line."""
|
||||
parts = content.split(None, 3)
|
||||
if len(parts) < 4 or not _ANACRON_PERIOD.match(parts[0]) or not parts[1].isdigit():
|
||||
return None
|
||||
return parts[0], parts[2], parts[3]
|
||||
|
||||
|
||||
@dataclass
|
||||
class _Scan:
|
||||
"""What is set while a text is read line by line."""
|
||||
|
||||
env: dict[str, str] = field(default_factory=dict)
|
||||
seen: dict[str, int] = field(default_factory=dict)
|
||||
name: str = ""
|
||||
description: str = ""
|
||||
|
||||
def key(self, base: str) -> str:
|
||||
n = self.seen.get(base, 0) + 1
|
||||
self.seen[base] = n
|
||||
return base if n == 1 else f"{base}:{n}"
|
||||
|
||||
def take_name(self) -> tuple[str, str]:
|
||||
named = self.name, self.description
|
||||
self.name, self.description = "", ""
|
||||
return named
|
||||
|
||||
|
||||
def _content(raw: str) -> tuple[str, bool] | None:
|
||||
"""The job text of *raw* and whether it is on; None for a comment or a blank."""
|
||||
line = raw.strip()
|
||||
if line.startswith(DISABLED_MARKER.strip()):
|
||||
return line[len(DISABLED_MARKER.strip()) :].strip(), False
|
||||
if not line or line.startswith("#"):
|
||||
return None
|
||||
return line, True
|
||||
|
||||
|
||||
def _remember(scan: _Scan, raw: str) -> bool:
|
||||
"""Take a variable or an application's marker from *raw*; whether it was one."""
|
||||
line = raw.strip()
|
||||
if line.startswith(JOB_MARKER.strip()):
|
||||
name, _, description = line[len(JOB_MARKER.strip()) :].strip().partition(" ")
|
||||
scan.name, scan.description = name, description.strip()
|
||||
return True
|
||||
variable = _VARIABLE.match(line)
|
||||
if variable:
|
||||
scan.env[variable[1]] = _unquote(variable[2])
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def parse_crontab(path: str, text: str, *, system: bool, user: str = "") -> list[CronLine]:
|
||||
"""The jobs of the crontab *text* at *path*.
|
||||
|
||||
*system*: a sixth field names the user (``/etc/crontab``, ``/etc/cron.d``);
|
||||
otherwise every job runs as *user*.
|
||||
"""
|
||||
scan, jobs = _Scan(), []
|
||||
for index, raw in enumerate(text.splitlines()):
|
||||
if _remember(scan, raw):
|
||||
continue
|
||||
found = _content(raw)
|
||||
split = _split(found[0], system=system, user=user) if found else None
|
||||
if found is None or split is None:
|
||||
continue
|
||||
content, enabled = found
|
||||
name, description = scan.take_name()
|
||||
key = f"job:{name}" if name else scan.key(f"cron:{path}:{_digest(content)}")
|
||||
schedule, run_as, command = split
|
||||
jobs.append(
|
||||
CronLine(
|
||||
index, key, schedule, run_as, command, enabled, dict(scan.env), name, description
|
||||
)
|
||||
)
|
||||
return jobs
|
||||
|
||||
|
||||
def parse_anacrontab(text: str) -> list[CronLine]:
|
||||
"""The jobs of an anacrontab: period, delay, identifier, command; all run as root."""
|
||||
scan, jobs = _Scan(), []
|
||||
for index, raw in enumerate(text.splitlines()):
|
||||
if _remember(scan, raw):
|
||||
continue
|
||||
found = _content(raw)
|
||||
split = _anacron_split(found[0]) if found else None
|
||||
if found is None or split is None:
|
||||
continue
|
||||
period, identifier, command = split
|
||||
key = scan.key(f"anacron:{identifier}")
|
||||
jobs.append(CronLine(index, key, period, "root", command, found[1], dict(scan.env)))
|
||||
return jobs
|
||||
|
||||
|
||||
def _replace(text: str, index: int, lines: list[str]) -> str:
|
||||
rows = text.splitlines()
|
||||
rows[index : index + 1] = lines
|
||||
return "\n".join(rows) + "\n"
|
||||
|
||||
|
||||
def switched(text: str, line: CronLine, *, enabled: bool) -> str:
|
||||
"""*text* with *line* switched on or off: the marker added or taken away, nothing else."""
|
||||
raw = text.splitlines()[line.index]
|
||||
found = _content(raw)
|
||||
if found is None: # pragma: no cover - a CronLine always points at a job
|
||||
raise ValueError(f"line {line.index} is no job")
|
||||
content = found[0]
|
||||
return _replace(text, line.index, [content if enabled else DISABLED_MARKER + content])
|
||||
|
||||
|
||||
def marked_lines(name: str, description: str, line: str, *, enabled: bool) -> list[str]:
|
||||
"""An application's job as crontab lines: its marker, then the job, maybe switched off."""
|
||||
marker = f"{JOB_MARKER}{name} {description}".rstrip()
|
||||
return [marker, line if enabled else DISABLED_MARKER + line]
|
||||
|
||||
|
||||
def with_job(text: str, jobs: list[CronLine], name: str, lines: list[str]) -> str:
|
||||
"""*text* with the application's job *name* as *lines*: replaced, or added at the end."""
|
||||
current = next((j for j in jobs if j.name == name), None)
|
||||
if current is None:
|
||||
body = text if not text or text.endswith("\n") else text + "\n"
|
||||
return body + "\n".join(lines) + "\n"
|
||||
rows = text.splitlines()
|
||||
rows[current.index - 1 : current.index + 1] = lines
|
||||
return "\n".join(rows) + "\n"
|
||||
|
||||
|
||||
def without_job(text: str, jobs: list[CronLine], name: str) -> str:
|
||||
"""*text* without the application's job *name* and its marker."""
|
||||
current = next(j for j in jobs if j.name == name)
|
||||
rows = text.splitlines()
|
||||
del rows[current.index - 1 : current.index + 1]
|
||||
return "\n".join(rows) + ("\n" if rows else "")
|
||||
|
||||
|
||||
def _unescape(part: str) -> str:
|
||||
return part.replace("\\%", "%")
|
||||
|
||||
|
||||
def run_script(command: str, env: dict[str, str]) -> str:
|
||||
"""*command* as ``sh -c`` takes it, the way cron would run it.
|
||||
|
||||
The crontab's variables are exported first. The first ``%`` that is not
|
||||
escaped ends the command; what follows is its stdin, with every further
|
||||
``%`` a newline, and ``\\%`` is a ``%``.
|
||||
"""
|
||||
parts = re.split(r"(?<!\\)%", command)
|
||||
exports = "".join(f"export {k}={quote(v)}; " for k, v in env.items() if k != "MAILTO")
|
||||
head = _unescape(parts[0])
|
||||
if len(parts) == 1:
|
||||
return exports + head
|
||||
stdin = "\n".join(_unescape(p) for p in parts[1:])
|
||||
return f"{exports}printf '%s\\n' {quote(stdin)} | {{ {head}\n}}"
|
||||
@@ -0,0 +1,417 @@
|
||||
"""The scheduled-jobs report of a POSIX host: the command that prints it, and its parse.
|
||||
|
||||
:mod:`napalm_device_types.scheduled_jobs_posix` sends :func:`reading_command` and
|
||||
hands what came back to :func:`parse_reading`. Pure: no I/O.
|
||||
|
||||
**One framed report, as root.** Every crontab, cron.d file, anacrontab and job
|
||||
script is printed behind a header line and with ``|`` before each of its lines,
|
||||
so no line of a file can pass for a header; run-parts scripts by their execute
|
||||
bit and checksum; systemd's timers and the units they start by ``systemctl show``;
|
||||
then the time zone and which package owns each definition file. A report whose
|
||||
end marker is missing raises: a list cut short must never read as jobs that went
|
||||
away.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import UTC, datetime
|
||||
from shlex import quote
|
||||
|
||||
from napalm_device_types._crontab import JOB_MARKER, CronLine, parse_anacrontab, parse_crontab
|
||||
from napalm_device_types.scheduled_jobs import DELETE, EDIT, job_digest
|
||||
from napalm_device_types.terminal import strip_terminal_codes
|
||||
|
||||
_BEGIN = "SJ_BEGIN"
|
||||
_END = "SJ_END"
|
||||
SCHEDULE_HEADER = "# schedule: "
|
||||
|
||||
TIMER_PROPERTIES = (
|
||||
"Id,Unit,Description,TimersCalendar,TimersMonotonic,NextElapseUSecRealtime,"
|
||||
"LastTriggerUSec,UnitFileState,ActiveState,FragmentPath"
|
||||
)
|
||||
UNIT_PROPERTIES = "Id,ExecStart,User,Result,FragmentPath"
|
||||
|
||||
#: The names run-parts runs (Debian's ``--lsbsysinit``-less default).
|
||||
_RUN_PARTS_NAME = re.compile(r"^[A-Za-z0-9_-]+$")
|
||||
_SECTION = re.compile(r"^\[([a-z-]+)(?: (.*))?\]$")
|
||||
|
||||
#: Reads definition paths on stdin, prints ``<package>: <path>`` for each a package owns.
|
||||
OWNERS_SH = (
|
||||
"if command -v dpkg >/dev/null 2>&1; then xargs dpkg -S 2>/dev/null; "
|
||||
"elif command -v rpm >/dev/null 2>&1; then while read -r p; do "
|
||||
'o=$(rpm -qf --qf \'%{NAME}\' -- "$p" 2>/dev/null) && printf \'%s: %s\\n\' "$o" "$p"; done; '
|
||||
"elif command -v apk >/dev/null 2>&1; then while read -r p; do "
|
||||
'o=$(apk info -qW -- "$p" 2>/dev/null) && [ -n "$o" ] && printf \'%s: %s\\n\' "$o" "$p"; done; '
|
||||
"else cat >/dev/null; fi"
|
||||
)
|
||||
|
||||
|
||||
#: Prints the host's time zone: an IANA name, or a path ending in ``zoneinfo/<name>``.
|
||||
TIME_ZONE_SH = "cat /etc/timezone 2>/dev/null || readlink /etc/localtime 2>/dev/null"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Paths:
|
||||
"""Where a host keeps its scheduled jobs."""
|
||||
|
||||
system_files: tuple[str, ...]
|
||||
cron_d_dirs: tuple[str, ...]
|
||||
spool_dirs: tuple[str, ...]
|
||||
run_parts: tuple[tuple[str, str], ...] # (period, directory)
|
||||
anacrontab: str
|
||||
script_dir: str
|
||||
systemd_marker: str = "/run/systemd/system"
|
||||
owners_sh: str = OWNERS_SH
|
||||
time_zone_sh: str = TIME_ZONE_SH
|
||||
|
||||
|
||||
@dataclass
|
||||
class Reading:
|
||||
"""What one report said, beyond the jobs: what writes need to know."""
|
||||
|
||||
jobs: list[dict[str, object]] = field(default_factory=list)
|
||||
time_zone: str | None = None
|
||||
systemd: bool = False
|
||||
#: per key: the unit a timer starts, the crontab line a cron job is
|
||||
units: dict[str, str] = field(default_factory=dict)
|
||||
lines: dict[str, CronLine] = field(default_factory=dict)
|
||||
|
||||
|
||||
def _q(path: str) -> str:
|
||||
return quote(path)
|
||||
|
||||
|
||||
def _files(section: str, pattern: str) -> str:
|
||||
return (
|
||||
f'for f in {pattern}; do [ -f "$f" ] || continue; printf \'[%s %s]\\n\' {section} "$f"; '
|
||||
# GNU sed keeps a missing last newline; without one of its own the next
|
||||
# header would run on into the file's last line.
|
||||
'sed \'s/^/|/\' "$f"; [ -n "$(tail -c 1 "$f")" ] && echo; P="$P\n$f"; done; '
|
||||
)
|
||||
|
||||
|
||||
def _run_parts(period: str, directory: str) -> str:
|
||||
return (
|
||||
f'for f in {_q(directory)}/*; do [ -f "$f" ] || continue; '
|
||||
'[ -x "$f" ] && x=1 || x=0; c=$(cksum < "$f" | cut -d" " -f1); '
|
||||
f'printf \'[run-parts %s %s %s %s]\\n\' {period} "$x" "$c" "$f"; P="$P\n$f"; done; '
|
||||
)
|
||||
|
||||
|
||||
def _timers(paths: Paths) -> str:
|
||||
show = f"systemctl show --no-pager -p {TIMER_PROPERTIES}"
|
||||
units = f"systemctl show --no-pager -p {UNIT_PROPERTIES}"
|
||||
return (
|
||||
f"if [ -d {_q(paths.systemd_marker)} ]; then echo '[systemd]'; "
|
||||
"t=$(systemctl list-unit-files --type=timer --no-legend --no-pager 2>/dev/null "
|
||||
"| awk '$1 !~ /@\\./ {print $1}'); "
|
||||
'if [ -n "$t" ]; then '
|
||||
f"s=$({show} --timestamp=unix $t 2>/dev/null || {show} $t 2>/dev/null); "
|
||||
"echo '[timers]'; printf '%s\\n' \"$s\" | sed 's/^/|/'; "
|
||||
"u=$(printf '%s\\n' \"$s\" | sed -n 's/^Unit=//p'); "
|
||||
f"echo '[units]'; {units} $u 2>/dev/null | sed 's/^/|/'; "
|
||||
"P=\"$P\n$(printf '%s\\n' \"$s\" | sed -n 's/^FragmentPath=//p')\"; fi; fi; "
|
||||
)
|
||||
|
||||
|
||||
def reading_command(paths: Paths) -> str:
|
||||
"""One POSIX ``sh`` script that prints every scheduled job of the host; run it as root."""
|
||||
parts = ["printf '%s%s\\n' SJ_ BEGIN; P=''; "]
|
||||
parts += [_files("system", _q(f)) for f in paths.system_files]
|
||||
parts += [_files("system", f"{_q(d)}/*") for d in paths.cron_d_dirs]
|
||||
parts += [_files("user", f"{_q(d)}/*") for d in paths.spool_dirs]
|
||||
parts.append(_files("anacron", _q(paths.anacrontab)))
|
||||
parts += [_run_parts(period, d) for period, d in paths.run_parts]
|
||||
parts.append(_files("script", f"{_q(paths.script_dir)}/*"))
|
||||
parts.append(_timers(paths))
|
||||
parts.append(
|
||||
f"echo '[tz]'; {{ {paths.time_zone_sh}; }}; "
|
||||
f"echo '[owners]'; printf '%s\\n' \"$P\" | sed '/^$/d' | {{ {paths.owners_sh}; }}; "
|
||||
"printf '%s%s\\n' SJ_ END"
|
||||
)
|
||||
return "".join(parts)
|
||||
|
||||
|
||||
# -- parse ------------------------------------------------------------------------------------
|
||||
|
||||
|
||||
@dataclass
|
||||
class _Section:
|
||||
name: str
|
||||
args: str
|
||||
lines: list[str] = field(default_factory=list)
|
||||
|
||||
|
||||
def _sections(output: str) -> list[_Section]:
|
||||
text = strip_terminal_codes(output)
|
||||
begin, end = text.rfind(_BEGIN), text.rfind(_END)
|
||||
if begin < 0 or end < begin:
|
||||
raise ValueError("the scheduled-jobs report is cut short (no end marker)")
|
||||
sections: list[_Section] = []
|
||||
for line in text[begin + len(_BEGIN) : end].splitlines():
|
||||
header = _SECTION.match(line)
|
||||
if header:
|
||||
sections.append(_Section(header[1], header[2] or ""))
|
||||
elif sections:
|
||||
sections[-1].lines.append(line[1:] if line.startswith("|") else line)
|
||||
return sections
|
||||
|
||||
|
||||
def _owned(lines: list[str]) -> set[str]:
|
||||
owned: set[str] = set()
|
||||
for line in lines:
|
||||
package, sep, path = line.partition(": ")
|
||||
if sep and path.startswith("/") and not package.startswith("diversion"):
|
||||
owned.add(path.strip())
|
||||
return owned
|
||||
|
||||
|
||||
def _time_zone(lines: list[str]) -> str | None:
|
||||
value = next((ln.strip() for ln in lines if ln.strip()), "")
|
||||
if "zoneinfo/" in value:
|
||||
value = value.split("zoneinfo/", 1)[1]
|
||||
return value or None
|
||||
|
||||
|
||||
def _blocks(lines: list[str]) -> list[dict[str, list[str]]]:
|
||||
blocks: list[dict[str, list[str]]] = [{}]
|
||||
for line in lines:
|
||||
if not line.strip():
|
||||
blocks.append({})
|
||||
continue
|
||||
key, sep, value = line.partition("=")
|
||||
if sep:
|
||||
blocks[-1].setdefault(key, []).append(value)
|
||||
return [b for b in blocks if b.get("Id")]
|
||||
|
||||
|
||||
def _one(block: dict[str, list[str]], key: str) -> str:
|
||||
return (block.get(key) or [""])[0].strip()
|
||||
|
||||
|
||||
def _timestamp(value: str) -> str | None:
|
||||
"""systemd's ``@<seconds>`` (or a UTC time) as ISO 8601; None when it never was."""
|
||||
value = value.strip()
|
||||
if value.startswith("@") and value[1:].isdigit() and int(value[1:]) > 0:
|
||||
return datetime.fromtimestamp(int(value[1:]), UTC).isoformat()
|
||||
match = re.search(r"(\d{4}-\d\d-\d\d \d\d:\d\d:\d\d) UTC$", value)
|
||||
if match:
|
||||
return datetime.strptime(match[1], "%Y-%m-%d %H:%M:%S").replace(tzinfo=UTC).isoformat()
|
||||
return None
|
||||
|
||||
|
||||
def _timer_entries(block: dict[str, list[str]], key: str) -> list[str]:
|
||||
"""``{ OnCalendar=daily ; next_elapse=… }`` → ``OnCalendar=daily``, one per entry.
|
||||
|
||||
``systemctl show`` names a monotonic setting by its property (``OnUnitActiveUSec``);
|
||||
it is given as the unit file says it (``OnUnitActiveSec``).
|
||||
"""
|
||||
entries = (v.strip(" {}").split(" ; ")[0].strip() for v in block.get(key, []) if v.strip())
|
||||
return [re.sub(r"^(On\w+)USec=", r"\1Sec=", e) for e in entries]
|
||||
|
||||
|
||||
def _argv(exec_start: list[str]) -> str:
|
||||
commands = []
|
||||
for value in exec_start:
|
||||
match = re.search(r"argv\[\]=(.*?) ; ", value)
|
||||
if match:
|
||||
commands.append(match[1].strip())
|
||||
return "; ".join(commands)
|
||||
|
||||
|
||||
@dataclass
|
||||
class _Script:
|
||||
name: str
|
||||
description: str
|
||||
schedule: str
|
||||
command: str
|
||||
|
||||
|
||||
def _script(path: str, lines: list[str]) -> _Script | None:
|
||||
"""An application's job script: its marker, schedule and command; None for any other file."""
|
||||
marker = next((ln for ln in lines if ln.startswith(JOB_MARKER)), None)
|
||||
schedule = next((ln for ln in lines if ln.startswith(SCHEDULE_HEADER)), None)
|
||||
if marker is None or schedule is None:
|
||||
return None
|
||||
name, _, description = marker[len(JOB_MARKER) :].partition(" ")
|
||||
body = [ln for ln in lines if ln not in (marker, schedule) and not ln.startswith("#!")]
|
||||
return _Script(
|
||||
name, description.strip(), schedule[len(SCHEDULE_HEADER) :], "\n".join(body).strip()
|
||||
)
|
||||
|
||||
|
||||
def _job(
|
||||
*,
|
||||
key: str,
|
||||
kind: str,
|
||||
source: str,
|
||||
run_as: str,
|
||||
schedule: str,
|
||||
schedule_kind: str,
|
||||
command: str,
|
||||
enabled: bool,
|
||||
managed_by: str,
|
||||
actions: list[str],
|
||||
content: str = "",
|
||||
**extra: object,
|
||||
) -> dict[str, object]:
|
||||
job: dict[str, object] = {
|
||||
"key": key,
|
||||
"kind": kind,
|
||||
"source": source,
|
||||
"run_as": run_as,
|
||||
"schedule": schedule,
|
||||
"schedule_kind": schedule_kind,
|
||||
"command": command,
|
||||
"enabled": enabled,
|
||||
"managed_by": managed_by,
|
||||
"actions": actions,
|
||||
}
|
||||
job["digest"] = job_digest(job, content=content)
|
||||
job.update({k: v for k, v in extra.items() if v not in (None, "")})
|
||||
return job
|
||||
|
||||
|
||||
_SWITCH = ["enable", "disable", "run"]
|
||||
_OWN = [*_SWITCH, EDIT, DELETE]
|
||||
|
||||
|
||||
def _cron_jobs(section: _Section, owned: set[str], reading: Reading) -> None:
|
||||
path = section.args
|
||||
text = "\n".join(section.lines)
|
||||
if section.name == "anacron":
|
||||
found = parse_anacrontab(text)
|
||||
else:
|
||||
user = path.rsplit("/", 1)[-1]
|
||||
found = parse_crontab(path, text, system=section.name == "system", user=user)
|
||||
kind = "anacron" if section.name == "anacron" else "cron"
|
||||
for line in found:
|
||||
own = bool(line.name)
|
||||
reading.lines[line.key] = line
|
||||
reading.jobs.append(
|
||||
_job(
|
||||
key=line.key,
|
||||
kind=kind,
|
||||
source=path,
|
||||
run_as=line.run_as,
|
||||
schedule=line.schedule,
|
||||
schedule_kind="period" if kind == "anacron" else "cron",
|
||||
command=line.command,
|
||||
enabled=line.enabled,
|
||||
managed_by="application" if own else ("package" if path in owned else "admin"),
|
||||
actions=_OWN if own else _SWITCH,
|
||||
name=line.name,
|
||||
description=line.description,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _run_parts_job(section: _Section, owned: set[str], reading: Reading) -> None:
|
||||
period, executable, checksum, path = section.args.split(" ", 3)
|
||||
if not _RUN_PARTS_NAME.match(path.rsplit("/", 1)[-1]):
|
||||
return
|
||||
reading.jobs.append(
|
||||
_job(
|
||||
key=f"run-parts:{path}",
|
||||
kind="run_parts",
|
||||
source=path,
|
||||
run_as="root",
|
||||
schedule=period,
|
||||
schedule_kind="period",
|
||||
command=path,
|
||||
enabled=executable == "1",
|
||||
managed_by="package" if path in owned else "admin",
|
||||
actions=_SWITCH,
|
||||
content=checksum,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _timer_job(
|
||||
timer: dict[str, list[str]],
|
||||
unit: dict[str, list[str]],
|
||||
scripts: dict[str, _Script],
|
||||
owned: set[str],
|
||||
) -> dict[str, object]:
|
||||
fragment = _one(timer, "FragmentPath")
|
||||
calendar = _timer_entries(timer, "TimersCalendar")
|
||||
schedule = "; ".join(calendar + _timer_entries(timer, "TimersMonotonic"))
|
||||
command = _argv(unit.get("ExecStart", []))
|
||||
script = scripts.get(command.split(" ", 1)[0])
|
||||
last_run = _timestamp(_one(timer, "LastTriggerUSec"))
|
||||
switchable = _one(timer, "UnitFileState") not in ("static", "generated", "transient")
|
||||
common = {
|
||||
"kind": "systemd_timer",
|
||||
"source": fragment,
|
||||
"run_as": _one(unit, "User") or "root",
|
||||
"enabled": _one(timer, "ActiveState") == "active",
|
||||
"last_run": last_run,
|
||||
"last_result": _one(unit, "Result") or None if last_run else None,
|
||||
"next_run": _timestamp(_one(timer, "NextElapseUSecRealtime")),
|
||||
}
|
||||
if script is not None:
|
||||
return _job(
|
||||
key=f"job:{script.name}",
|
||||
schedule=script.schedule,
|
||||
schedule_kind="cron",
|
||||
command=script.command,
|
||||
managed_by="application",
|
||||
actions=_OWN,
|
||||
name=script.name,
|
||||
description=script.description,
|
||||
**common, # type: ignore[arg-type]
|
||||
)
|
||||
packaged = fragment in owned or fragment.startswith(("/usr/lib/", "/lib/"))
|
||||
return _job(
|
||||
key=f"timer:{_one(timer, 'Id')}",
|
||||
schedule=schedule,
|
||||
schedule_kind="calendar" if calendar else "monotonic",
|
||||
command=command,
|
||||
managed_by="package" if packaged else "admin",
|
||||
actions=_SWITCH if switchable else ["run"],
|
||||
description=_one(timer, "Description"),
|
||||
**common, # type: ignore[arg-type]
|
||||
)
|
||||
|
||||
|
||||
def _timer_jobs(
|
||||
sections: dict[str, _Section], scripts: dict[str, _Script], owned: set[str], reading: Reading
|
||||
) -> None:
|
||||
timers = _blocks(sections["timers"].lines) if "timers" in sections else []
|
||||
units = (
|
||||
{_one(b, "Id"): b for b in _blocks(sections["units"].lines)} if "units" in sections else {}
|
||||
)
|
||||
for timer in timers:
|
||||
unit = units.get(_one(timer, "Unit"), {})
|
||||
job = _timer_job(timer, unit, scripts, owned)
|
||||
reading.units[str(job["key"])] = _one(timer, "Unit")
|
||||
reading.jobs.append(job)
|
||||
|
||||
|
||||
def parse_reading(output: str) -> Reading:
|
||||
"""The jobs, time zone and write context of a report from :func:`reading_command`.
|
||||
|
||||
:raises ValueError: when the report is cut short.
|
||||
"""
|
||||
sections = _sections(output)
|
||||
named = {s.name: s for s in sections if s.name in ("timers", "units", "tz", "owners")}
|
||||
owned = _owned(named["owners"].lines) if "owners" in named else set()
|
||||
reading = Reading(
|
||||
time_zone=_time_zone(named["tz"].lines) if "tz" in named else None,
|
||||
systemd=any(s.name == "systemd" for s in sections),
|
||||
)
|
||||
for section in sections:
|
||||
if section.name in ("system", "user", "anacron"):
|
||||
_cron_jobs(section, owned, reading)
|
||||
elif section.name == "run-parts":
|
||||
_run_parts_job(section, owned, reading)
|
||||
_timer_jobs(named, _scripts(sections), owned, reading)
|
||||
return reading
|
||||
|
||||
|
||||
def _scripts(sections: list[_Section]) -> dict[str, _Script]:
|
||||
"""An application's job scripts, by path."""
|
||||
scripts = ((s.args, _script(s.args, s.lines)) for s in sections if s.name == "script")
|
||||
return {path: script for path, script in scripts if script is not None}
|
||||
@@ -0,0 +1,369 @@
|
||||
"""The reading behind :class:`~napalm_device_types.time_sync_posix.PosixTimeSyncMixin`.
|
||||
|
||||
One command, as the login user, prints framed sections (``@@ts <name> ...``):
|
||||
|
||||
* ``timedatectl`` -- ``timedatectl show``: the zone, and whether the kernel clock is
|
||||
synchronised;
|
||||
* ``zone`` -- ``/etc/timezone`` or the target of ``/etc/localtime``, for a host
|
||||
without timedatectl;
|
||||
* ``active`` -- one ``<unit> <daemon> <state>`` line per time sync service;
|
||||
* ``timesync`` -- ``timedatectl show-timesync``: systemd-timesyncd's servers;
|
||||
* ``chronyc`` -- ``chronyc -n sources``: whether chronyd has selected a source;
|
||||
* ``chrony <line> <kind> <path>`` -- chrony's main configuration (line 0, kind
|
||||
``main``), then each file an ``include``, ``confdir`` or ``sourcedir`` on that
|
||||
line of it reads;
|
||||
* ``ntpd <path>``, ``openntpd <path>`` -- their configuration;
|
||||
* ``end`` -- the reading is complete.
|
||||
|
||||
What DHCP brought is read apart from the configured servers: chrony's files in a
|
||||
DHCP client's ``sourcedir`` (:data:`CHRONY_DHCP_DIRS`), systemd-timesyncd's
|
||||
``LinkNTPServers``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import posixpath
|
||||
import re
|
||||
import shlex
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
from napalm_device_types.time_sync import TimeSyncDict
|
||||
|
||||
_MARK = "@@ts "
|
||||
|
||||
#: ``<unit> <daemon>``: the services that keep a host's clock; the first active one counts.
|
||||
TIME_SYNC_UNITS: tuple[tuple[str, str], ...] = (
|
||||
("chrony", "chrony"),
|
||||
("chronyd", "chrony"),
|
||||
("systemd-timesyncd", "systemd-timesyncd"),
|
||||
("ntpsec", "ntpd"),
|
||||
("ntp", "ntpd"),
|
||||
("ntpd", "ntpd"),
|
||||
("openntpd", "openntpd"),
|
||||
)
|
||||
|
||||
#: The ``sourcedir`` a DHCP client writes chrony's sources to, one file per interface
|
||||
#: and lease: the NetworkManager dispatcher and dhclient hook of Debian, Ubuntu and
|
||||
#: Red Hat. What chrony reads from there is DHCP's (option 42), not configuration.
|
||||
CHRONY_DHCP_DIRS: tuple[str, ...] = ("/run/chrony-dhcp", "/var/run/chrony-dhcp")
|
||||
|
||||
#: A source line of chrony's or ntpd's configuration: the directive and what follows.
|
||||
SOURCE_LINE = re.compile(r"\s*(server|pool|peer)\s+(.*)")
|
||||
_OPENNTPD_SOURCE = re.compile(r"\s*(servers?)\s+(\S+)")
|
||||
_ZONE_NAME = re.compile(r"[A-Za-z0-9_+-]+(?:/[A-Za-z0-9_+-]+)*")
|
||||
#: ntpd's reference clocks are addresses in 127.127.0.0/16, not servers.
|
||||
_REFCLOCK = "127.127."
|
||||
|
||||
# The files a chrony configuration reads, each after a marker with the line that names it.
|
||||
_CHRONY_FOLLOW = """\
|
||||
awk '$1=="include"||$1=="confdir"||$1=="sourcedir"{for(i=2;i<=NF;i++)print NR, $1, $i}' "$c" |
|
||||
while read -r n k p; do
|
||||
case $k in include) set -- $p ;; confdir) set -- "$p"/*.conf ;; *) set -- "$p"/*.sources ;; esac
|
||||
for f in "$@"; do
|
||||
[ -f "$f" ] && { printf '@@ts chrony %s %s %s\\n' "$n" "$k" "$f"; cat -- "$f"; echo; }
|
||||
done
|
||||
done"""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ReadPaths:
|
||||
"""Where a host keeps what the reading reads."""
|
||||
|
||||
units: tuple[tuple[str, str], ...]
|
||||
chrony_configs: tuple[str, ...]
|
||||
ntpd_configs: tuple[str, ...]
|
||||
openntpd_configs: tuple[str, ...]
|
||||
time_zone_sh: str
|
||||
|
||||
|
||||
@dataclass
|
||||
class ConfigFile:
|
||||
"""One configuration file as the reading printed it."""
|
||||
|
||||
path: str
|
||||
kind: str # "main", "include", "confdir", "sourcedir"; "ntpd", "openntpd"
|
||||
at: int # the line of the main configuration that names it; 0 for that file itself
|
||||
lines: list[str] = field(default_factory=list)
|
||||
|
||||
|
||||
@dataclass
|
||||
class Reading:
|
||||
"""Everything one reading said."""
|
||||
|
||||
timedatectl: dict[str, str] = field(default_factory=dict)
|
||||
zone_files: list[str] = field(default_factory=list)
|
||||
active: list[tuple[str, str]] = field(default_factory=list) # (unit, daemon), active ones
|
||||
timesync: dict[str, str] = field(default_factory=dict)
|
||||
chronyc: str = ""
|
||||
chrony: list[ConfigFile] = field(default_factory=list)
|
||||
ntpd: ConfigFile | None = None
|
||||
openntpd: ConfigFile | None = None
|
||||
|
||||
@property
|
||||
def daemon(self) -> str | None:
|
||||
return self.active[0][1] if self.active else None
|
||||
|
||||
@property
|
||||
def unit(self) -> str | None:
|
||||
return self.active[0][0] if self.active else None
|
||||
|
||||
|
||||
def _first_file(kind: str, paths: tuple[str, ...]) -> str:
|
||||
candidates = " ".join(shlex.quote(p) for p in paths)
|
||||
return (
|
||||
f'for c in {candidates}; do [ -f "$c" ] && '
|
||||
f'{{ printf \'@@ts {kind} %s\\n\' "$c"; cat -- "$c"; echo; break; }}; done'
|
||||
)
|
||||
|
||||
|
||||
def reading_command(paths: ReadPaths) -> str:
|
||||
"""The command that prints the reading; ``sh`` runs it, whatever the login shell."""
|
||||
active = "\n".join(
|
||||
f"printf '%s %s %s\\n' {shlex.quote(unit)} {shlex.quote(daemon)} "
|
||||
f'"$(systemctl is-active {shlex.quote(unit + ".service")} 2>/dev/null)"'
|
||||
for unit, daemon in paths.units
|
||||
)
|
||||
chrony = " ".join(shlex.quote(p) for p in paths.chrony_configs)
|
||||
script = "\n".join(
|
||||
[
|
||||
"echo '@@ts timedatectl'",
|
||||
"timedatectl show 2>/dev/null",
|
||||
"echo '@@ts zone'",
|
||||
f"{{ {paths.time_zone_sh}; }}",
|
||||
"echo '@@ts active'",
|
||||
active,
|
||||
"echo '@@ts timesync'",
|
||||
"timedatectl show-timesync 2>/dev/null",
|
||||
"echo '@@ts chronyc'",
|
||||
"chronyc -n sources 2>/dev/null",
|
||||
f'for c in {chrony}; do [ -f "$c" ] || continue',
|
||||
'printf \'@@ts chrony 0 main %s\\n\' "$c"; cat -- "$c"; echo',
|
||||
_CHRONY_FOLLOW,
|
||||
"break; done",
|
||||
_first_file("ntpd", paths.ntpd_configs),
|
||||
_first_file("openntpd", paths.openntpd_configs),
|
||||
"echo '@@ts end'",
|
||||
]
|
||||
)
|
||||
return f"sh -c {shlex.quote(script)}"
|
||||
|
||||
|
||||
def parse_key_values(text: str) -> dict[str, str]:
|
||||
"""``Key=value`` lines (``timedatectl show``, ``show-timesync``)."""
|
||||
values: dict[str, str] = {}
|
||||
for line in text.splitlines():
|
||||
key, sep, value = line.partition("=")
|
||||
if sep and key.strip():
|
||||
values[key.strip()] = value.strip()
|
||||
return values
|
||||
|
||||
|
||||
def _sections(text: str) -> list[tuple[list[str], list[str]]]:
|
||||
sections: list[tuple[list[str], list[str]]] = []
|
||||
for line in text.splitlines():
|
||||
if line.startswith(_MARK):
|
||||
sections.append((line[len(_MARK) :].split(maxsplit=3), []))
|
||||
elif sections:
|
||||
sections[-1][1].append(line)
|
||||
return sections
|
||||
|
||||
|
||||
def _active(lines: list[str]) -> list[tuple[str, str]]:
|
||||
rows = [line.split() for line in lines]
|
||||
return [(r[0], r[1]) for r in rows if len(r) == 3 and r[2] == "active"]
|
||||
|
||||
|
||||
def _take(reading: Reading, header: list[str], lines: list[str]) -> None:
|
||||
name, text = header[0], "\n".join(lines)
|
||||
if name == "chrony" and len(header) == 4:
|
||||
reading.chrony.append(ConfigFile(header[3], header[2], int(header[1]), lines))
|
||||
elif name in ("ntpd", "openntpd") and len(header) == 2:
|
||||
setattr(reading, name, ConfigFile(header[1], name, 0, lines))
|
||||
else:
|
||||
simple = {
|
||||
"timedatectl": ("timedatectl", parse_key_values(text)),
|
||||
"zone": ("zone_files", lines),
|
||||
"active": ("active", _active(lines)),
|
||||
"timesync": ("timesync", parse_key_values(text)),
|
||||
"chronyc": ("chronyc", text),
|
||||
}
|
||||
if name in simple:
|
||||
setattr(reading, *simple[name])
|
||||
|
||||
|
||||
def parse_reading(text: str) -> Reading:
|
||||
"""The reading's sections. :raises ValueError: for a reading cut short."""
|
||||
sections = _sections(text)
|
||||
if not sections or sections[-1][0] != ["end"]:
|
||||
raise ValueError("the time sync reading was cut short")
|
||||
reading = Reading()
|
||||
for header, lines in sections:
|
||||
_take(reading, header, lines)
|
||||
return reading
|
||||
|
||||
|
||||
def _source_name(rest: str) -> str | None:
|
||||
"""The server in what follows ``server``/``pool``/``peer``: ntpd puts ``-4`` first."""
|
||||
return next((word for word in rest.split() if not word.startswith("-")), None)
|
||||
|
||||
|
||||
def in_dhcp_dir(f: ConfigFile, dhcp_dirs: tuple[str, ...]) -> bool:
|
||||
"""Whether *f* is a sources file a DHCP client wrote to one of *dhcp_dirs*."""
|
||||
if f.kind != "sourcedir":
|
||||
return False
|
||||
directory = posixpath.normpath(posixpath.dirname(f.path))
|
||||
return directory in {posixpath.normpath(d) for d in dhcp_dirs}
|
||||
|
||||
|
||||
def _named_on(files: list[ConfigFile], number: int) -> list[ConfigFile]:
|
||||
"""The files the main configuration's line *number* reads."""
|
||||
return [f for f in files if f.at == number and f.kind != "main"]
|
||||
|
||||
|
||||
def _read_order(files: list[ConfigFile], dhcp_dirs: tuple[str, ...]) -> list[tuple[bool, str]]:
|
||||
"""Every line chrony reads, in its order -- each file in place of the line naming
|
||||
it -- and whether a DHCP client wrote it."""
|
||||
main = next((f for f in files if f.kind == "main"), None)
|
||||
if main is None:
|
||||
return []
|
||||
found: list[tuple[bool, str]] = []
|
||||
for number, line in enumerate(main.lines, start=1):
|
||||
found.append((False, line))
|
||||
for included in _named_on(files, number):
|
||||
dhcp = in_dhcp_dir(included, dhcp_dirs)
|
||||
found.extend((dhcp, text) for text in included.lines)
|
||||
return found
|
||||
|
||||
|
||||
def chrony_servers(files: list[ConfigFile], dhcp_dirs: tuple[str, ...] = ()) -> list[str]:
|
||||
"""chrony's configured sources in the order it reads them; not those in *dhcp_dirs*."""
|
||||
return _sources([line for dhcp, line in _read_order(files, dhcp_dirs) if not dhcp])
|
||||
|
||||
|
||||
def chrony_dhcp_servers(files: list[ConfigFile], dhcp_dirs: tuple[str, ...]) -> list[str]:
|
||||
"""The sources chrony reads from the DHCP client's files in *dhcp_dirs*, in its order.
|
||||
|
||||
A ``sourcedir`` chrony does not read (commented out) prints no files: nothing then.
|
||||
"""
|
||||
return _sources([line for dhcp, line in _read_order(files, dhcp_dirs) if dhcp])
|
||||
|
||||
|
||||
def _sources(lines: list[str]) -> list[str]:
|
||||
names = []
|
||||
for line in lines:
|
||||
match = SOURCE_LINE.match(line)
|
||||
if match and (name := _source_name(match[2])):
|
||||
names.append(name)
|
||||
return names
|
||||
|
||||
|
||||
def ntpd_servers(lines: list[str]) -> list[str]:
|
||||
"""ntpd's servers, pools and peers; reference clocks are not servers."""
|
||||
return [name for name in _sources(lines) if not name.startswith(_REFCLOCK)]
|
||||
|
||||
|
||||
def openntpd_servers(lines: list[str]) -> list[str]:
|
||||
"""OpenNTPD's ``server`` and ``servers``; ``constraint`` and ``sensor`` are no sources."""
|
||||
return [m[2] for line in lines if (m := _OPENNTPD_SOURCE.match(line))]
|
||||
|
||||
|
||||
def _listed(values: dict[str, str], *keys: str) -> list[str]:
|
||||
found: list[str] = []
|
||||
for key in keys:
|
||||
for server in values.get(key, "").split():
|
||||
if server not in found:
|
||||
found.append(server)
|
||||
return found
|
||||
|
||||
|
||||
def timesyncd_servers(values: dict[str, str]) -> list[str]:
|
||||
"""systemd-timesyncd's configured servers, runtime then system; the fallback only
|
||||
while it has no other server, a link's included. A link's servers are not among
|
||||
them: :func:`timesyncd_link_servers`."""
|
||||
configured = _listed(values, "RuntimeNTPServers", "SystemNTPServers")
|
||||
if configured or timesyncd_link_servers(values):
|
||||
return configured
|
||||
return _listed(values, "FallbackNTPServers")
|
||||
|
||||
|
||||
def timesyncd_link_servers(values: dict[str, str]) -> list[str]:
|
||||
"""The servers network links gave systemd-timesyncd: systemd-networkd's, from a DHCP
|
||||
lease (option 42) or a ``.network`` file's ``NTP=`` -- the reading cannot tell those
|
||||
apart and counts them as DHCP's."""
|
||||
return _listed(values, "LinkNTPServers")
|
||||
|
||||
|
||||
def chrony_synchronized(sources: str) -> bool | None:
|
||||
"""Whether chronyd has selected a source (``^*``); None for no answer from chronyc."""
|
||||
rows = [line for line in sources.splitlines() if line[:1] in ("^", "=", "#")]
|
||||
if not rows:
|
||||
return None
|
||||
return any(line[1:2] == "*" for line in rows)
|
||||
|
||||
|
||||
def _zone_name(value: str) -> str | None:
|
||||
value = value.strip()
|
||||
if "zoneinfo/" in value:
|
||||
value = value.split("zoneinfo/", 1)[1]
|
||||
for prefix in ("posix/", "right/"):
|
||||
value = value.removeprefix(prefix)
|
||||
return value if _ZONE_NAME.fullmatch(value) else None
|
||||
|
||||
|
||||
def _time_zone(reading: Reading) -> str | None:
|
||||
named = reading.timedatectl.get("Timezone", "")
|
||||
if named and named != "n/a":
|
||||
return _zone_name(named)
|
||||
return next((z for line in reading.zone_files if (z := _zone_name(line))), None)
|
||||
|
||||
|
||||
def _servers(reading: Reading, dhcp_dirs: tuple[str, ...]) -> list[str]:
|
||||
if reading.daemon == "chrony":
|
||||
return chrony_servers(reading.chrony, dhcp_dirs)
|
||||
if reading.daemon == "systemd-timesyncd":
|
||||
return timesyncd_servers(reading.timesync)
|
||||
if reading.daemon == "ntpd" and reading.ntpd is not None:
|
||||
return ntpd_servers(reading.ntpd.lines)
|
||||
if reading.daemon == "openntpd" and reading.openntpd is not None:
|
||||
return openntpd_servers(reading.openntpd.lines)
|
||||
return []
|
||||
|
||||
|
||||
def _dhcp_servers(reading: Reading, dhcp_dirs: tuple[str, ...]) -> list[str] | None:
|
||||
"""What DHCP brought: chrony's and timesyncd's; None where ntpd or OpenNTPD runs,
|
||||
whose DHCP hooks keep a configuration of their own that is not read."""
|
||||
if reading.daemon == "chrony":
|
||||
return chrony_dhcp_servers(reading.chrony, dhcp_dirs)
|
||||
if reading.daemon == "systemd-timesyncd":
|
||||
return timesyncd_link_servers(reading.timesync)
|
||||
if reading.daemon is None:
|
||||
return []
|
||||
return None
|
||||
|
||||
|
||||
def _synchronized(reading: Reading) -> bool | None:
|
||||
if reading.daemon == "chrony":
|
||||
chrony = chrony_synchronized(reading.chronyc)
|
||||
if chrony is not None:
|
||||
return chrony
|
||||
return {"yes": True, "no": False}.get(reading.timedatectl.get("NTPSynchronized", ""))
|
||||
|
||||
|
||||
def time_sync_of(reading: Reading, dhcp_dirs: tuple[str, ...] = CHRONY_DHCP_DIRS) -> TimeSyncDict:
|
||||
"""What a reading says, as ``get_time_sync`` returns it; *dhcp_dirs* are the DHCP
|
||||
client's ``sourcedir`` of chrony."""
|
||||
return {
|
||||
"ntp_servers": _servers(reading, dhcp_dirs),
|
||||
"timezone": _time_zone(reading),
|
||||
"synchronized": _synchronized(reading),
|
||||
"daemon": reading.daemon,
|
||||
"dhcp_ntp_servers": _dhcp_servers(reading, dhcp_dirs),
|
||||
}
|
||||
|
||||
|
||||
def parse_time_sync(text: str, dhcp_dirs: tuple[str, ...] = CHRONY_DHCP_DIRS) -> TimeSyncDict:
|
||||
"""The reading's output as ``get_time_sync`` returns it.
|
||||
|
||||
:raises ValueError: for a reading cut short.
|
||||
"""
|
||||
return time_sync_of(parse_reading(text), dhcp_dirs)
|
||||
@@ -0,0 +1,238 @@
|
||||
"""
|
||||
Abstract base class for backup server drivers.
|
||||
|
||||
Usage::
|
||||
|
||||
from napalm_device_types import BackupServerDriver, StorageDriver
|
||||
|
||||
class ProxmoxBackupDriver(BackupServerDriver, StorageDriver):
|
||||
def get_backup_datastores(self):
|
||||
...
|
||||
"""
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from napalm_device_types.base import DeviceTypeDriver
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - for type checkers only
|
||||
from napalm_device_types.backups import (
|
||||
BackupDatastoreDict,
|
||||
BackupGroupDict,
|
||||
BackupJobKind,
|
||||
BackupRemoteDict,
|
||||
BackupServerJobDict,
|
||||
BackupSnapshotDict,
|
||||
BackupSnapshotRefDict,
|
||||
BackupTaskDict,
|
||||
BackupTaskStatusDict,
|
||||
)
|
||||
|
||||
|
||||
class BackupServerDriver(DeviceTypeDriver):
|
||||
"""
|
||||
Abstract intermediate driver for backup servers (e.g. Proxmox Backup
|
||||
Server): the device hypervisors and hosts write their backups to.
|
||||
|
||||
A backup server keeps its backups in datastores, one group per source (a
|
||||
VM, a container, a host) and one snapshot per backup run, and keeps them
|
||||
healthy with scheduled jobs: garbage collection, verify, prune, sync with
|
||||
other servers and tape. None of that is a storage pool, a volume or a
|
||||
replication job, so it is not modelled through ``StorageDriver``. A server
|
||||
that also manages its own disks and pools lists ``StorageDriver`` as a
|
||||
second role, after this one.
|
||||
|
||||
The actions start a task on the server and return at once with its id;
|
||||
:meth:`get_backup_task` follows it.
|
||||
"""
|
||||
|
||||
#: Stable key netOrk exposes as ``device_class``. The order in which a
|
||||
#: driver lists its role bases is the ranking; see
|
||||
#: :func:`napalm_device_types.roles.primary_role_of`.
|
||||
ROLE: str = "backup_server"
|
||||
TYPE_LABEL: str = "Backup Server"
|
||||
|
||||
if TYPE_CHECKING:
|
||||
# ------------------------------------------------------------------
|
||||
# Read
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def get_backup_datastores(self) -> list[BackupDatastoreDict]:
|
||||
"""
|
||||
Returns the server's datastores with their usage.
|
||||
|
||||
Example::
|
||||
|
||||
[
|
||||
{
|
||||
"name": "store1",
|
||||
"path": "/mnt/datastore/store1",
|
||||
"backend": "local",
|
||||
"total": 4000787030016,
|
||||
"used": 1529008357376,
|
||||
"available": 2471778672640,
|
||||
"estimated_full": 1798761600,
|
||||
"maintenance": None,
|
||||
"comment": None,
|
||||
},
|
||||
]
|
||||
|
||||
:raises Exception: when the server could not be read; never an
|
||||
empty list for a failed read.
|
||||
"""
|
||||
...
|
||||
|
||||
def get_backup_groups(self, datastore: str = "") -> list[BackupGroupDict]:
|
||||
"""
|
||||
Returns the backup groups of every namespace of *datastore*, or of
|
||||
every datastore when it is empty, each with its newest snapshot
|
||||
only. :meth:`get_backup_snapshots` lists all of a group's.
|
||||
|
||||
Example::
|
||||
|
||||
[
|
||||
{
|
||||
"datastore": "store1",
|
||||
"namespace": "",
|
||||
"type": "vm",
|
||||
"id": "101",
|
||||
"owner": "backup@pbs",
|
||||
"count": 14,
|
||||
"newest": {
|
||||
"time": 1793660400,
|
||||
"size": 34359738368,
|
||||
"verify": "ok",
|
||||
"protected": False,
|
||||
"comment": "web-01",
|
||||
},
|
||||
"oldest_time": 1792537200,
|
||||
"comment": None,
|
||||
},
|
||||
]
|
||||
|
||||
:raises Exception: when the server could not be read.
|
||||
"""
|
||||
...
|
||||
|
||||
def get_backup_snapshots(
|
||||
self, datastore: str, namespace: str, group_type: str, group_id: str
|
||||
) -> list[BackupSnapshotDict]:
|
||||
"""
|
||||
Returns every snapshot of one group, newest first.
|
||||
|
||||
:param namespace: ``""`` for the datastore's root.
|
||||
:raises ValueError: when the datastore or the group does not exist.
|
||||
"""
|
||||
...
|
||||
|
||||
def get_backup_jobs(self) -> list[BackupServerJobDict]:
|
||||
"""
|
||||
Returns the server's scheduled jobs of every kind, garbage
|
||||
collection included (one per datastore, ``id`` = the datastore).
|
||||
|
||||
Example::
|
||||
|
||||
[
|
||||
{
|
||||
"id": "store1",
|
||||
"kind": "gc",
|
||||
"datastore": "store1",
|
||||
"namespace": None,
|
||||
"schedule": "daily",
|
||||
"enabled": True,
|
||||
"last_run": 1793664000,
|
||||
"last_state": "ok",
|
||||
"last_error": None,
|
||||
"next_run": 1793750400,
|
||||
"remote": None,
|
||||
"direction": None,
|
||||
"comment": None,
|
||||
},
|
||||
{
|
||||
"id": "offsite",
|
||||
"kind": "sync",
|
||||
"datastore": "store1",
|
||||
"namespace": "",
|
||||
"schedule": "sat 03:00",
|
||||
"enabled": True,
|
||||
"last_run": 1793502000,
|
||||
"last_state": "error",
|
||||
"last_error": "connection refused",
|
||||
"next_run": 1794106800,
|
||||
"remote": "pbs-offsite",
|
||||
"direction": "push",
|
||||
"comment": None,
|
||||
},
|
||||
]
|
||||
|
||||
:raises Exception: when the server could not be read.
|
||||
"""
|
||||
...
|
||||
|
||||
def get_backup_remotes(self) -> list[BackupRemoteDict]:
|
||||
"""
|
||||
Returns the other backup servers this one syncs with, without their
|
||||
secrets.
|
||||
|
||||
:raises Exception: when the server could not be read.
|
||||
"""
|
||||
...
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Actions
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def start_garbage_collection(self, datastore: str) -> BackupTaskDict:
|
||||
"""
|
||||
Starts garbage collection on *datastore*.
|
||||
|
||||
:raises ValueError: when the datastore does not exist.
|
||||
:raises RuntimeError: when the server refuses (e.g. one is running).
|
||||
"""
|
||||
...
|
||||
|
||||
def run_backup_job(self, kind: BackupJobKind, job_id: str) -> BackupTaskDict:
|
||||
"""
|
||||
Runs a scheduled job now, outside its schedule.
|
||||
|
||||
:param kind: a :data:`~napalm_device_types.backups.BackupJobKind`;
|
||||
``"gc"`` runs the datastore named *job_id*'s garbage collection.
|
||||
:raises ValueError: when there is no such job.
|
||||
:raises RuntimeError: when the server refuses.
|
||||
"""
|
||||
...
|
||||
|
||||
def verify_backup_snapshot(self, snapshot: BackupSnapshotRefDict) -> BackupTaskDict:
|
||||
"""
|
||||
Verifies one snapshot now.
|
||||
|
||||
:raises ValueError: when the snapshot does not exist.
|
||||
"""
|
||||
...
|
||||
|
||||
def set_backup_snapshot_protected(
|
||||
self, snapshot: BackupSnapshotRefDict, protected: bool
|
||||
) -> None:
|
||||
"""
|
||||
Protects a snapshot from pruning and deletion, or lifts that.
|
||||
|
||||
:raises ValueError: when the snapshot does not exist.
|
||||
"""
|
||||
...
|
||||
|
||||
def delete_backup_snapshot(self, snapshot: BackupSnapshotRefDict) -> None:
|
||||
"""
|
||||
Deletes one snapshot. The data is gone once garbage collection has
|
||||
run; there is no undo.
|
||||
|
||||
:raises ValueError: when the snapshot does not exist.
|
||||
:raises RuntimeError: when it is protected.
|
||||
"""
|
||||
...
|
||||
|
||||
def get_backup_task(self, task: str) -> BackupTaskStatusDict:
|
||||
"""
|
||||
Returns the state and the log so far of a task an action started.
|
||||
|
||||
:raises ValueError: when the server does not know the task.
|
||||
"""
|
||||
...
|
||||
@@ -0,0 +1,119 @@
|
||||
"""The shapes of a backup server (``BackupServerDriver``).
|
||||
|
||||
Apart from ``models.py``, which is over the file size limit. A value the
|
||||
platform does not report is None; it is never guessed.
|
||||
"""
|
||||
|
||||
from typing import Literal, TypedDict, get_args
|
||||
|
||||
#: What a backup server's jobs do. Garbage collection is one of them: it has a
|
||||
#: schedule and a last run like the others.
|
||||
BackupJobKind = Literal["gc", "verify", "prune", "sync", "tape"]
|
||||
BACKUP_JOB_KINDS: tuple[str, ...] = get_args(BackupJobKind)
|
||||
|
||||
#: How a job's or a task's run ended; "running" while it has not.
|
||||
BackupRunState = Literal["ok", "warning", "error", "running"]
|
||||
|
||||
#: The outcome of the last verification of a snapshot.
|
||||
BackupVerifyState = Literal["ok", "failed"]
|
||||
|
||||
|
||||
class BackupDatastoreDict(TypedDict):
|
||||
"""A datastore: where a backup server keeps its backups."""
|
||||
|
||||
name: str
|
||||
path: str | None # local path; for an object-store backend the local cache, if any
|
||||
backend: str # "local", "s3", "removable", ... as the platform names it
|
||||
total: int | None # bytes
|
||||
used: int | None # bytes
|
||||
available: int | None # bytes
|
||||
estimated_full: int | None # unix seconds the platform expects it to be full
|
||||
maintenance: str | None # "read-only", "offline", ...; None when in service
|
||||
comment: str | None
|
||||
|
||||
|
||||
class BackupSnapshotDict(TypedDict):
|
||||
"""One backup of a group: the state of its source at one time."""
|
||||
|
||||
time: int # unix seconds the backup was taken
|
||||
size: int | None # bytes
|
||||
verify: BackupVerifyState | None # None: never verified
|
||||
protected: bool # exempt from pruning and deletion
|
||||
comment: str | None
|
||||
|
||||
|
||||
class BackupGroupDict(TypedDict):
|
||||
"""The backups of one source (a VM, a container, a host) in one namespace."""
|
||||
|
||||
datastore: str
|
||||
namespace: str # "" for the datastore's root
|
||||
type: str # "vm", "ct", "host"
|
||||
id: str # the guest's id ("101") or the host's name
|
||||
owner: str | None # the account that writes the group's backups
|
||||
count: int # snapshots in the group
|
||||
newest: BackupSnapshotDict | None # None for a group without a snapshot
|
||||
oldest_time: int | None # unix seconds of the oldest snapshot
|
||||
comment: str | None
|
||||
|
||||
|
||||
class BackupSnapshotRefDict(TypedDict):
|
||||
"""Names one snapshot, for an action on it."""
|
||||
|
||||
datastore: str
|
||||
namespace: str # "" for the datastore's root
|
||||
group_type: str # BackupGroupDict.type
|
||||
group_id: str # BackupGroupDict.id
|
||||
time: int # BackupSnapshotDict.time
|
||||
|
||||
|
||||
class BackupServerJobDict(TypedDict):
|
||||
"""A scheduled job of a backup server."""
|
||||
|
||||
id: str # for "gc", the datastore's name
|
||||
kind: BackupJobKind
|
||||
datastore: str | None
|
||||
namespace: str | None
|
||||
schedule: str | None # as the platform writes it: "daily", "sat 02:00", "*:0/30"
|
||||
enabled: bool
|
||||
last_run: int | None # unix seconds the last run ended (or started, while running)
|
||||
last_state: BackupRunState | None # None: never ran
|
||||
last_error: str | None
|
||||
next_run: int | None # unix seconds
|
||||
remote: str | None # a sync job's BackupRemoteDict.name
|
||||
direction: Literal["pull", "push"] | None # a sync job's
|
||||
comment: str | None
|
||||
|
||||
|
||||
class BackupRemoteDict(TypedDict):
|
||||
"""Another backup server this one syncs with. Never its secret."""
|
||||
|
||||
name: str
|
||||
host: str
|
||||
port: int | None
|
||||
fingerprint: str | None # of the remote's TLS certificate
|
||||
auth_id: str | None # the account the sync logs in as
|
||||
comment: str | None
|
||||
|
||||
|
||||
class BackupTaskDict(TypedDict):
|
||||
"""A task an action started on the server, to follow with ``get_backup_task``."""
|
||||
|
||||
task: str # the platform's task id
|
||||
|
||||
|
||||
class BackupTaskStatusDict(TypedDict):
|
||||
task: str
|
||||
running: bool
|
||||
state: BackupRunState | None # "running" while it runs
|
||||
error: str | None
|
||||
log: list[str] # the task's log lines so far
|
||||
|
||||
|
||||
class BackupServerRefDict(TypedDict):
|
||||
"""Which backup server and datastore a hypervisor's backup storage writes to."""
|
||||
|
||||
server: str # host name or address, as the hypervisor has it configured
|
||||
port: int | None
|
||||
datastore: str
|
||||
namespace: str | None
|
||||
fingerprint: str | None # of the server's TLS certificate, as the hypervisor pins it
|
||||
@@ -5,6 +5,8 @@ Apart from ``models.py``, which is over the file size limit.
|
||||
|
||||
from typing import NotRequired, TypedDict
|
||||
|
||||
from napalm_device_types.backups import BackupServerRefDict
|
||||
|
||||
|
||||
class ClusterNodeDict(TypedDict):
|
||||
"""One host of a cluster, as its cluster sees it. A value the platform does not
|
||||
@@ -34,6 +36,9 @@ class ClusterStorageDict(TypedDict):
|
||||
nodes: list[str] # the nodes it is available on
|
||||
total: int | None # bytes; None where the platform does not report it
|
||||
used: int | None # bytes
|
||||
# A backup storage that writes to a backup server: which one. Left out for every
|
||||
# other storage; None where the platform does not say.
|
||||
backup_server: NotRequired[BackupServerRefDict | None]
|
||||
|
||||
|
||||
class ClusterNetworkDict(TypedDict):
|
||||
|
||||
@@ -0,0 +1,203 @@
|
||||
"""Cron expressions: what they mean, when they fire next, and as systemd ``OnCalendar``.
|
||||
|
||||
A scheduled job is given as a cron expression on every host, and a host with
|
||||
systemd runs it as a timer. So the expression has to mean the same in three
|
||||
places -- on the host's cron, in a preview of the next runs, and in the
|
||||
``OnCalendar`` lines a timer is written with -- and the meaning lives here once.
|
||||
|
||||
**What cron means is Vixie cron's.** Five fields (minute, hour, day of month,
|
||||
month, day of week), each ``*``, a value, a range ``a-b``, a step ``*/n`` or
|
||||
``a-b/n``, or a list of those; month and weekday names in any case; ``7`` is
|
||||
Sunday as well as ``0``; the ``@hourly`` … ``@yearly`` aliases. When both day
|
||||
fields are restricted -- neither starts with ``*`` -- a day matches if *either*
|
||||
does; otherwise both must (``*/2`` counts as starting with ``*``).
|
||||
|
||||
**``OnCalendar`` by expansion.** Every field is written as its values, so a range
|
||||
with a step needs no systemd syntax of its own, and a timer fires on exactly the
|
||||
minutes cron would. The *either* rule is two ``OnCalendar`` lines, which systemd
|
||||
ORs. ``@reboot`` has no calendar; a timer takes it as ``OnBootSec``.
|
||||
|
||||
**Next runs are in the host's time zone.** A local time the clock skips when
|
||||
summer time starts is left out; cron implementations disagree on it, and a
|
||||
preview that promised it would promise too much.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import date, datetime, timedelta
|
||||
from typing import NamedTuple
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
REBOOT = "@reboot"
|
||||
|
||||
ALIASES = {
|
||||
"@yearly": "0 0 1 1 *",
|
||||
"@annually": "0 0 1 1 *",
|
||||
"@monthly": "0 0 1 * *",
|
||||
"@weekly": "0 0 * * 0",
|
||||
"@daily": "0 0 * * *",
|
||||
"@midnight": "0 0 * * *",
|
||||
"@hourly": "0 * * * *",
|
||||
}
|
||||
|
||||
_MONTHS = ("jan", "feb", "mar", "apr", "may", "jun", "jul", "aug", "sep", "oct", "nov", "dec")
|
||||
_WEEKDAYS = ("sun", "mon", "tue", "wed", "thu", "fri", "sat")
|
||||
_SYSTEMD_WEEKDAYS = ("Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat")
|
||||
|
||||
#: (lowest, highest, names) per field; a name stands for its index plus the lowest.
|
||||
_FIELDS = (
|
||||
(0, 59, ()),
|
||||
(0, 23, ()),
|
||||
(1, 31, ()),
|
||||
(1, 12, _MONTHS),
|
||||
(0, 7, _WEEKDAYS),
|
||||
)
|
||||
|
||||
#: How far ahead a next run is looked for: a 29 February comes within eight years.
|
||||
_HORIZON_DAYS = 8 * 366
|
||||
|
||||
|
||||
class CronSchedule(NamedTuple):
|
||||
"""An expression's values per field, and whether a day matches on either day field."""
|
||||
|
||||
minutes: frozenset[int]
|
||||
hours: frozenset[int]
|
||||
days: frozenset[int]
|
||||
months: frozenset[int]
|
||||
weekdays: frozenset[int] # 0 = Sunday
|
||||
either_day: bool
|
||||
|
||||
|
||||
def is_reboot(expr: str) -> bool:
|
||||
"""Whether *expr* is ``@reboot``: once at boot, no calendar."""
|
||||
return expr.strip().lower() == REBOOT
|
||||
|
||||
|
||||
def _value(token: str, low: int, names: tuple[str, ...]) -> int:
|
||||
if token.isdigit():
|
||||
return int(token)
|
||||
if token.lower() in names:
|
||||
return names.index(token.lower()) + low
|
||||
raise ValueError(f"not a value: {token!r}")
|
||||
|
||||
|
||||
def _part(part: str, low: int, high: int, names: tuple[str, ...]) -> set[int]:
|
||||
span, _, step_text = part.partition("/")
|
||||
step = 1
|
||||
if step_text:
|
||||
if not step_text.isdigit() or int(step_text) == 0:
|
||||
raise ValueError(f"not a step: {part!r}")
|
||||
step = int(step_text)
|
||||
if span == "*":
|
||||
first, last = low, high
|
||||
elif "-" in span:
|
||||
a, _, b = span.partition("-")
|
||||
first, last = _value(a, low, names), _value(b, low, names)
|
||||
else:
|
||||
first = _value(span, low, names)
|
||||
last = high if step_text else first
|
||||
if not low <= first <= last <= high:
|
||||
raise ValueError(f"out of range {low}-{high}: {part!r}")
|
||||
return set(range(first, last + 1, step))
|
||||
|
||||
|
||||
def _field(text: str, low: int, high: int, names: tuple[str, ...]) -> frozenset[int]:
|
||||
values: set[int] = set()
|
||||
for part in text.split(","):
|
||||
if not part:
|
||||
raise ValueError(f"empty list item in {text!r}")
|
||||
values |= _part(part, low, high, names)
|
||||
return frozenset(values)
|
||||
|
||||
|
||||
def parse_cron(expr: str) -> CronSchedule:
|
||||
"""The values *expr* stands for.
|
||||
|
||||
:raises ValueError: for anything Vixie cron would not take, and for
|
||||
``@reboot``, which names no time.
|
||||
"""
|
||||
text = expr.strip()
|
||||
if is_reboot(text):
|
||||
raise ValueError("@reboot names no time; it runs once at boot")
|
||||
if text.startswith("@"):
|
||||
if text.lower() not in ALIASES:
|
||||
raise ValueError(f"unknown alias {text!r}")
|
||||
text = ALIASES[text.lower()]
|
||||
fields = text.split()
|
||||
if len(fields) != len(_FIELDS):
|
||||
raise ValueError(f"a cron expression has five fields, not {len(fields)}: {expr!r}")
|
||||
minutes, hours, days, months, weekdays = (
|
||||
_field(f, low, high, names) for f, (low, high, names) in zip(fields, _FIELDS, strict=True)
|
||||
)
|
||||
if 7 in weekdays:
|
||||
weekdays = (weekdays - {7}) | {0}
|
||||
either = not fields[2].startswith("*") and not fields[4].startswith("*")
|
||||
return CronSchedule(minutes, hours, days, months, frozenset(weekdays), either)
|
||||
|
||||
|
||||
def _day_matches(s: CronSchedule, day: date) -> bool:
|
||||
if day.month not in s.months:
|
||||
return False
|
||||
on_day = day.day in s.days
|
||||
on_weekday = (day.isoweekday() % 7) in s.weekdays
|
||||
return (on_day or on_weekday) if s.either_day else (on_day and on_weekday)
|
||||
|
||||
|
||||
def _local(day: date, hour: int, minute: int, tz: ZoneInfo) -> datetime | None:
|
||||
"""*day* at *hour*:*minute* in *tz*; None when the clock skips that time."""
|
||||
naive = datetime(day.year, day.month, day.day, hour, minute)
|
||||
aware = naive.replace(tzinfo=tz)
|
||||
if aware.astimezone(ZoneInfo("UTC")).astimezone(tz).replace(tzinfo=None) != naive:
|
||||
return None
|
||||
return aware
|
||||
|
||||
|
||||
def next_runs(expr: str, after: datetime, tz: ZoneInfo, count: int = 5) -> list[datetime]:
|
||||
"""The next *count* times *expr* fires after *after*, in *tz*.
|
||||
|
||||
``@reboot`` and a day that never comes (31 February) give none.
|
||||
|
||||
:raises ValueError: for an expression :func:`parse_cron` refuses.
|
||||
"""
|
||||
if is_reboot(expr):
|
||||
return []
|
||||
s = parse_cron(expr)
|
||||
start = after.astimezone(tz)
|
||||
runs: list[datetime] = []
|
||||
for offset in range(_HORIZON_DAYS):
|
||||
day = start.date() + timedelta(days=offset)
|
||||
if not _day_matches(s, day):
|
||||
continue
|
||||
for hour in sorted(s.hours):
|
||||
for minute in sorted(s.minutes):
|
||||
at = _local(day, hour, minute, tz)
|
||||
if at is None or at <= after:
|
||||
continue
|
||||
runs.append(at)
|
||||
if len(runs) == count:
|
||||
return runs
|
||||
return runs
|
||||
|
||||
|
||||
def _values(values: frozenset[int], low: int, high: int) -> str:
|
||||
if values == frozenset(range(low, high + 1)):
|
||||
return "*"
|
||||
return ",".join(f"{v:02d}" for v in sorted(values))
|
||||
|
||||
|
||||
def to_on_calendar(expr: str) -> list[str]:
|
||||
"""``OnCalendar`` lines that fire exactly when *expr* does.
|
||||
|
||||
:raises ValueError: for an expression :func:`parse_cron` refuses, ``@reboot``
|
||||
among them -- a timer takes that as ``OnBootSec``.
|
||||
"""
|
||||
s = parse_cron(expr)
|
||||
time = f"{_values(s.hours, 0, 23)}:{_values(s.minutes, 0, 59)}:00"
|
||||
months = _values(s.months, 1, 12)
|
||||
days = _values(s.days, 1, 31)
|
||||
weekdays = ",".join(_SYSTEMD_WEEKDAYS[d] for d in (1, 2, 3, 4, 5, 6, 0) if d in s.weekdays)
|
||||
every_weekday = s.weekdays == frozenset(range(7))
|
||||
if s.either_day:
|
||||
return [f"*-{months}-{days} {time}", f"{weekdays} *-{months}-* {time}"]
|
||||
prefix = "" if every_weekday else f"{weekdays} "
|
||||
return [f"{prefix}*-{months}-{days} {time}"]
|
||||
+55
-16
@@ -58,6 +58,19 @@ _DHCP_SUBNET_COMPARE_FIELDS = (
|
||||
_HEX_ONLY = re.compile(r"[^0-9a-f]")
|
||||
|
||||
|
||||
class DhcpApplyError(RuntimeError):
|
||||
"""Some reservations of an applied set were refused by the device.
|
||||
|
||||
Raised after the rest were written and committed: ``failed`` of ``attempted``
|
||||
changes did not go through, and each was named in a ``[failed]`` progress line.
|
||||
"""
|
||||
|
||||
def __init__(self, failed: int, attempted: int) -> None:
|
||||
self.failed = failed
|
||||
self.attempted = attempted
|
||||
super().__init__(f"{failed} of {attempted} DHCP reservation change(s) were refused")
|
||||
|
||||
|
||||
def normalize_mac(mac: str | None) -> str:
|
||||
"""Reduces a MAC address to lowercase colon-separated form.
|
||||
|
||||
@@ -298,29 +311,55 @@ class DhcpServerMixin:
|
||||
committing reloads the DHCP daemon and drops in-flight requests, which
|
||||
is too high a price for a no-op run.
|
||||
|
||||
A change the device refuses (say, an address in a subnet its DHCP server
|
||||
does not manage) does not stop the others: it gets a ``[failed]`` line,
|
||||
the rest are written and committed, and ``DhcpApplyError`` is raised at
|
||||
the end. Any non-empty diff commits, even one whose every write was
|
||||
refused, so that writes an earlier, interrupted run saved take effect.
|
||||
|
||||
:param desired: The complete desired reservation set.
|
||||
:yields: Progress lines, one per applied add/update, plus a final
|
||||
commit line.
|
||||
:yields: Progress lines, one per applied, updated or refused change,
|
||||
plus a final commit line.
|
||||
:raises DhcpApplyError: after the commit, when a change was refused.
|
||||
"""
|
||||
diff = self.diff_dhcp_reservations(desired)
|
||||
|
||||
for reservation in diff["add"]:
|
||||
self.apply_dhcp_reservation(reservation)
|
||||
yield f"[add] {reservation['mac']} -> {reservation['ip']}"
|
||||
|
||||
for entry in diff["update"]:
|
||||
self.apply_dhcp_reservation(entry["reservation"], uuid=entry["uuid"])
|
||||
fields = ", ".join(entry["changed_fields"])
|
||||
yield (
|
||||
f"[update] {entry['reservation']['mac']} -> {entry['reservation']['ip']} ({fields})"
|
||||
)
|
||||
|
||||
if not diff["add"] and not diff["update"]:
|
||||
attempted = len(diff["add"]) + len(diff["update"])
|
||||
if not attempted:
|
||||
yield "[commit] no changes"
|
||||
return
|
||||
|
||||
added = updated = refused = 0
|
||||
|
||||
for reservation in diff["add"]:
|
||||
line, ok = self._write_reservation(reservation, None, "[add]")
|
||||
added, refused = added + ok, refused + (not ok)
|
||||
yield line
|
||||
|
||||
for entry in diff["update"]:
|
||||
fields = ", ".join(entry["changed_fields"])
|
||||
line, ok = self._write_reservation(entry["reservation"], entry["uuid"], "[update]")
|
||||
updated, refused = updated + ok, refused + (not ok)
|
||||
yield f"{line} ({fields})" if ok else line
|
||||
|
||||
# Committed even when every write was refused: a run that stopped before its
|
||||
# commit left saved writes behind, which the device reports as configured, so
|
||||
# no later diff would ever commit them.
|
||||
self.commit_dhcp_reservations()
|
||||
yield (f"[commit] applied {len(diff['add'])} add(s), {len(diff['update'])} update(s)")
|
||||
yield f"[commit] applied {added} add(s), {updated} update(s)"
|
||||
|
||||
if refused:
|
||||
raise DhcpApplyError(refused, attempted)
|
||||
|
||||
def _write_reservation(
|
||||
self, reservation: DhcpReservationDict, uuid: str | None, tag: str
|
||||
) -> tuple[str, bool]:
|
||||
"""Write one reservation; its progress line, and whether the device took it."""
|
||||
target = f"{reservation['mac']} -> {reservation['ip']}"
|
||||
try:
|
||||
self.apply_dhcp_reservation(reservation, uuid=uuid)
|
||||
except Exception as exc:
|
||||
return f"[failed] {target}: {exc}", False
|
||||
return f"{tag} {target}", True
|
||||
|
||||
def diff_dhcp_subnets(self, desired: list[DhcpSubnetDict]) -> DhcpSubnetDiffDict:
|
||||
"""
|
||||
|
||||
@@ -43,7 +43,9 @@ exits quietly every day (NetOrk/netork#738). While dpkg, apt, aptitude or
|
||||
unattended-upgrade is running, packages are halfway on purpose -- also between
|
||||
two of apt's dpkg calls -- so the answer is then ``None``, as on a host without
|
||||
dpkg. unattended-upgrade is told from Ubuntu's idle shutdown helper by its
|
||||
command line (:data:`BUSY_CHECK`).
|
||||
command line (:data:`BUSY_CHECK`). On a host without dpkg, dnf or yum, apk and opkg
|
||||
are read the same way (:mod:`napalm_device_types.package_managers`,
|
||||
NetOrk/netork#851), and ``package_manager`` names the one the answer is about.
|
||||
|
||||
**Filesystems** are how full ``/``, ``/boot`` and ``/var`` are, from POSIX
|
||||
``df -Pk``: where an upgrade unpacks a kernel, builds its initrd and keeps the
|
||||
@@ -65,6 +67,11 @@ from napalm_device_types.models import (
|
||||
HostStatusDict,
|
||||
InterruptedPackageDict,
|
||||
)
|
||||
from napalm_device_types.package_managers import (
|
||||
PACKAGE_MANAGERS_COMMAND,
|
||||
PackageManagerReading,
|
||||
parse_package_managers,
|
||||
)
|
||||
from napalm_device_types.terminal import strip_terminal_codes
|
||||
|
||||
_BEGIN = "HSTAT_BEGIN"
|
||||
@@ -117,6 +124,7 @@ HOST_STATUS_COMMAND = (
|
||||
f"echo '[dpkg-journal]'; ls -1 {_DPKG_JOURNAL} 2>/dev/null | head -n 20; "
|
||||
"echo '[dpkg-audit]'; dpkg-query -W -f='${db:Status-Abbrev} ${Package}\\n' 2>/dev/null "
|
||||
"| awk 'substr($0, 2, 1) ~ /[HUFWt]/ || substr($0, 3, 1) == \"R\"' | head -n 50; fi; "
|
||||
f"{PACKAGE_MANAGERS_COMMAND}"
|
||||
f"echo '[filesystems]'; df -Pk {_FILESYSTEMS}; "
|
||||
f"echo '[timers]'; for u in {_APT_TIMER} {' '.join(_DNF_TIMERS)}; do "
|
||||
'printf \'%s %s\\n\' "$u" "$(systemctl is-enabled "$u" 2>/dev/null)"; done; '
|
||||
@@ -287,12 +295,24 @@ def _halfway(line: str) -> InterruptedPackageDict | None:
|
||||
return {"name": name, "state": state} if name and state else None
|
||||
|
||||
|
||||
def _dpkg(sections: dict[str, list[str]]) -> tuple[bool | None, list[InterruptedPackageDict]]:
|
||||
if "dpkg-audit" not in sections or "dpkg-busy" in sections:
|
||||
return None, []
|
||||
def _dpkg(sections: dict[str, list[str]]) -> PackageManagerReading | None:
|
||||
if "dpkg-audit" not in sections:
|
||||
return None
|
||||
if "dpkg-busy" in sections:
|
||||
return PackageManagerReading("dpkg", None, [])
|
||||
packages = [p for p in map(_halfway, sections["dpkg-audit"]) if p is not None]
|
||||
journal = any(_JOURNAL_ENTRY.fullmatch(f) for f in sections.get("dpkg-journal") or [])
|
||||
return bool(packages) or journal, packages
|
||||
return PackageManagerReading("dpkg", bool(packages) or journal, packages)
|
||||
|
||||
|
||||
def _package_manager(sections: dict[str, list[str]]) -> PackageManagerReading:
|
||||
"""The reading that says most: one left halfway, then one that is not, then a busy one."""
|
||||
readings = [r for r in (_dpkg(sections), *parse_package_managers(sections)) if r is not None]
|
||||
for interrupted in (True, False):
|
||||
found = next((r for r in readings if r.interrupted is interrupted), None)
|
||||
if found is not None:
|
||||
return found
|
||||
return readings[0] if readings else PackageManagerReading(None, None, [])
|
||||
|
||||
|
||||
def _filesystems(sections: dict[str, list[str]]) -> list[FilesystemUsageDict] | None:
|
||||
@@ -318,15 +338,16 @@ def parse_host_status(output: str) -> HostStatusDict:
|
||||
"""
|
||||
sections = _sections(output)
|
||||
required, reason = _reboot(sections)
|
||||
interrupted, packages = _dpkg(sections)
|
||||
manager = _package_manager(sections)
|
||||
return {
|
||||
"reboot_required": required,
|
||||
"reboot_reason": reason,
|
||||
"auto_updates": _auto_updates(sections),
|
||||
"auto_updates_last_success": _last_success(sections),
|
||||
"auto_updates_interval": _interval(sections),
|
||||
"package_manager_interrupted": interrupted,
|
||||
"interrupted_packages": packages,
|
||||
"package_manager": manager.manager,
|
||||
"package_manager_interrupted": manager.interrupted,
|
||||
"interrupted_packages": manager.packages,
|
||||
"filesystems": _filesystems(sections),
|
||||
}
|
||||
|
||||
@@ -356,6 +377,7 @@ class HostStatusMixin:
|
||||
* auto_updates (bool or None)
|
||||
* auto_updates_last_success (Unix time or None)
|
||||
* auto_updates_interval (seconds or None)
|
||||
* package_manager (dpkg, dnf, yum, apk, opkg, or None)
|
||||
* package_manager_interrupted (bool or None)
|
||||
* interrupted_packages (list of {name, state})
|
||||
* filesystems (list of {mountpoint, total, used, available} in bytes, or None)
|
||||
|
||||
@@ -5,6 +5,14 @@ with their public key and, where they are to administer the host, may use
|
||||
sudo with their own password. When they leave, the account is locked, not
|
||||
deleted: their key goes, sudo goes, the login is refused, and the home stays.
|
||||
|
||||
**What the person changes follows them** (NetOrk/netork#2013). A key they
|
||||
replace goes from the account as the new one comes in (``replaces``); a key
|
||||
they delete goes without a new one (``remove_local_account_keys``). Either way
|
||||
only the keys named go: one the person added on the host stays, and the
|
||||
account stays open. A password they remove goes from the account too
|
||||
(``clear_password``): it logs in with a key only and has no sudo, as the sudo
|
||||
rule asks for that password.
|
||||
|
||||
:class:`LocalAccountsMixin` holds what is the same on every POSIX host -- the
|
||||
checks, reading an account back, the order of the steps, the key file and the
|
||||
sudo rule. A driver supplies the commands that differ between operating
|
||||
@@ -48,7 +56,7 @@ from typing import TYPE_CHECKING, ClassVar
|
||||
|
||||
from typing_extensions import TypedDict
|
||||
|
||||
from napalm_device_types.fixes import run_fix_steps
|
||||
from napalm_device_types.fixes import FixStep, run_fix_steps
|
||||
from napalm_device_types.models import FixResultDict, FixStepDict, FixStepOutcomeDict
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - for type checkers only
|
||||
@@ -179,6 +187,18 @@ def check_public_key(key: str) -> None:
|
||||
raise ValueError("not a public key: expected '<type> <base64> [comment]' on one line")
|
||||
|
||||
|
||||
def _checked_keys(keys: Sequence[str]) -> list[str]:
|
||||
"""*keys*, each one stripped.
|
||||
|
||||
:raises ValueError: when *keys* is one string, or one of them no public key.
|
||||
"""
|
||||
if isinstance(keys, str):
|
||||
raise ValueError("expected a list of public keys, not one string")
|
||||
for key in keys:
|
||||
check_public_key(key)
|
||||
return [key.strip() for key in keys]
|
||||
|
||||
|
||||
def _command(argv: Sequence[str]) -> str:
|
||||
return shlex.join(argv)
|
||||
|
||||
@@ -225,9 +245,11 @@ class LocalAccountsMixin:
|
||||
|
||||
# -- what a driver supplies ------------------------------------------------------
|
||||
#
|
||||
# ``_account_*`` are the driver's: the five below it must supply, the key
|
||||
# store (``_account_key_files``, ``_account_read_keys``, ``_account_add_key``,
|
||||
# ``_account_remove_keys``) it may replace. ``_accounts_*`` are this mixin's own.
|
||||
# ``_account_*`` are the driver's: the five below it must supply, the two after
|
||||
# them (``_account_create_with_uid``, ``_account_clear_password``) it may, and
|
||||
# the key store (``_account_key_files``, ``_account_read_keys``,
|
||||
# ``_account_add_key``, ``_account_drop_keys``, ``_account_remove_keys``) it may
|
||||
# replace. ``_accounts_*`` are this mixin's own.
|
||||
|
||||
def _account_create(self, name: str, password_hash: str | None) -> list[list[str]]:
|
||||
"""The commands that create *name* with a home and :attr:`ACCOUNT_SHELL`.
|
||||
@@ -265,6 +287,16 @@ class LocalAccountsMixin:
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
def _account_clear_password(self, name: str) -> list[list[str]]:
|
||||
"""The commands that take *name*'s password away; its key login stays as it is.
|
||||
|
||||
Not an empty password -- PAM's ``nullok`` lets anyone in without one --
|
||||
and not a lock or an expiry, which sshd or PAM apply to a key too.
|
||||
Only ``add_local_account(clear_password=True)`` and
|
||||
``remove_local_account_keys(clear_password=True)`` ask for it.
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
# -- reading ---------------------------------------------------------------------
|
||||
|
||||
def get_local_accounts(self, names: Iterable[str]) -> list[LocalAccountDict]:
|
||||
@@ -339,6 +371,8 @@ class LocalAccountsMixin:
|
||||
*,
|
||||
password_hash: str | None = None,
|
||||
admin: bool = False,
|
||||
clear_password: bool = False,
|
||||
replaces: Sequence[str] = (),
|
||||
) -> LocalAccountChangeDict:
|
||||
"""Give *name* an account that logs in with *public_key*, or open it again.
|
||||
|
||||
@@ -347,30 +381,82 @@ class LocalAccountsMixin:
|
||||
:attr:`ACCOUNT_PASSWORD_SCHEME`) becomes its password; *admin* gives it
|
||||
the sudo rule, which asks for that password -- so an administrator
|
||||
needs one. Without *admin*, the rule and :attr:`ACCOUNT_ADMIN_GROUPS`
|
||||
are taken away.
|
||||
are taken away. Without *password_hash* the account keeps the password
|
||||
it has, unless *clear_password* takes it away: the account then logs
|
||||
in with a key only, and can have no sudo.
|
||||
|
||||
Steps: ``account``, then ``sudo`` and ``keys``, each of which runs when
|
||||
the other failed.
|
||||
*replaces* are keys *public_key* takes the place of -- the person's
|
||||
old ones. They go from the account, matched by their base64 whatever
|
||||
a line's options or comment; every other key stays.
|
||||
|
||||
:raises ValueError: for a name, key or hash no host takes, the login
|
||||
user's own account, or a system account.
|
||||
Steps: ``account``, then ``sudo``, ``password`` (with
|
||||
*clear_password*), ``keys`` and ``old keys`` (with *replaces*), each of
|
||||
which runs when another failed: an old key goes even when the new one
|
||||
could not be added.
|
||||
|
||||
:raises ValueError: for a name, key or hash no host takes, a replaced
|
||||
key that is none, *clear_password* with a *password_hash* or with
|
||||
*admin*, the login user's own account, or a system account.
|
||||
"""
|
||||
check_account_name(name)
|
||||
check_public_key(public_key)
|
||||
if password_hash is not None:
|
||||
self._accounts_check_hash(password_hash)
|
||||
new = _key_blob(public_key)
|
||||
old = [key for key in _checked_keys(replaces) if _key_blob(key) != new]
|
||||
self._accounts_check_password(password_hash, admin=admin, clear=clear_password)
|
||||
existing = self._accounts_changeable(name)
|
||||
first = run_fix_steps(
|
||||
[("account", lambda: self._accounts_open_or_create(name, existing, password_hash))]
|
||||
)
|
||||
parts = [first]
|
||||
if first["success"]:
|
||||
parts += [
|
||||
run_fix_steps([("sudo", lambda: self._accounts_set_sudo(name, admin=admin))]),
|
||||
run_fix_steps([("keys", lambda: self._accounts_add_key(name, public_key))]),
|
||||
]
|
||||
steps = self._accounts_after_open(name, public_key, admin, clear_password, old)
|
||||
parts += [run_fix_steps([step]) for step in steps]
|
||||
return self._accounts_result(name, parts)
|
||||
|
||||
def _accounts_after_open(
|
||||
self, name: str, public_key: str, admin: bool, clear_password: bool, old: list[str]
|
||||
) -> list[FixStep]:
|
||||
"""The steps of ``add_local_account`` once the account is there, each on its own."""
|
||||
steps: list[FixStep] = [("sudo", lambda: self._accounts_set_sudo(name, admin=admin))]
|
||||
if clear_password:
|
||||
steps.append(("password", lambda: self._accounts_clear_password(name)))
|
||||
steps.append(("keys", lambda: self._accounts_add_key(name, public_key)))
|
||||
if old:
|
||||
steps.append(("old keys", lambda: self._accounts_drop_keys(name, old)))
|
||||
return steps
|
||||
|
||||
def remove_local_account_keys(
|
||||
self, name: str, keys: Sequence[str], *, clear_password: bool = False
|
||||
) -> LocalAccountChangeDict:
|
||||
"""Take *keys* off *name*'s account, for a person who deleted them.
|
||||
|
||||
Only those keys go, matched by their base64 whatever a line's options
|
||||
or comment; the other keys stay, and the account stays open -- to get
|
||||
a new key once there is one. A locked account has none left to
|
||||
remove: the lock took them all. Without *clear_password* the password
|
||||
and sudo stay too; with it the password goes, and so do the sudo rule
|
||||
and :attr:`ACCOUNT_ADMIN_GROUPS`, as sudo asks for that password -- for
|
||||
a person whose password went with their key.
|
||||
|
||||
Steps: ``keys``, then ``password`` and ``sudo`` (with *clear_password*),
|
||||
each of which runs when another failed.
|
||||
|
||||
:raises ValueError: for a name no host takes, no keys or one that is
|
||||
none, the login user's own account, or a system account.
|
||||
"""
|
||||
check_account_name(name)
|
||||
wanted = _checked_keys(keys)
|
||||
if not wanted:
|
||||
raise ValueError("no public key to remove")
|
||||
if self._accounts_changeable(name) is None:
|
||||
step: FixStepDict = {"step": "keys", "ok": False, "output": f"no account {name}"}
|
||||
return {**_joined([step]), "account": None}
|
||||
steps: list[FixStep] = [("keys", lambda: self._accounts_drop_keys(name, wanted))]
|
||||
if clear_password:
|
||||
steps.append(("password", lambda: self._accounts_clear_password(name)))
|
||||
steps.append(("sudo", lambda: self._accounts_set_sudo(name, admin=False)))
|
||||
return self._accounts_result(name, [run_fix_steps([step]) for step in steps])
|
||||
|
||||
def lock_local_account(self, name: str) -> LocalAccountChangeDict:
|
||||
"""Lock *name*: its keys go, every login is refused, sudo goes; its home stays.
|
||||
|
||||
@@ -490,6 +576,20 @@ class LocalAccountsMixin:
|
||||
raise RuntimeError(f"cannot ask who has uid {uid}: {_said(result)}")
|
||||
return result.stdout.split(":", 1)[0].strip() or None
|
||||
|
||||
def _accounts_check_password(
|
||||
self, password_hash: str | None, *, admin: bool, clear: bool
|
||||
) -> None:
|
||||
""":raises ValueError: for a hash the host does not take, or a password to
|
||||
clear that is to be set, or that sudo would ask for."""
|
||||
if clear and password_hash is not None:
|
||||
raise ValueError(
|
||||
"a password to clear cannot be set too: clear_password or password_hash"
|
||||
)
|
||||
if clear and admin:
|
||||
raise ValueError("an account without a password cannot use sudo: its rule asks for it")
|
||||
if password_hash is not None:
|
||||
self._accounts_check_hash(password_hash)
|
||||
|
||||
def _accounts_check_hash(self, password_hash: str) -> None:
|
||||
prefixes = PASSWORD_SCHEMES[self.ACCOUNT_PASSWORD_SCHEME]
|
||||
if not _CRYPT_RE.fullmatch(password_hash) or not password_hash.startswith(prefixes):
|
||||
@@ -550,6 +650,16 @@ class LocalAccountsMixin:
|
||||
return {"ok": False, "output": _said(failed)}
|
||||
return {"ok": True, "output": done + (", password set" if password_hash else "")}
|
||||
|
||||
def _accounts_clear_password(self, name: str) -> FixStepOutcomeDict:
|
||||
try:
|
||||
commands = self._account_clear_password(name)
|
||||
except NotImplementedError:
|
||||
return {"ok": False, "output": "this driver cannot remove a password"}
|
||||
failed = self._accounts_run(commands)
|
||||
if failed is not None:
|
||||
return {"ok": False, "output": _said(failed)}
|
||||
return {"ok": True, "output": f"{name} has no password: it logs in with a key only"}
|
||||
|
||||
def _accounts_lock(self, name: str) -> FixStepOutcomeDict:
|
||||
passwd = self._accounts_passwd(name)
|
||||
if passwd is None:
|
||||
@@ -683,9 +793,23 @@ class LocalAccountsMixin:
|
||||
mine = self._accounts_key_file(name, path)
|
||||
if mine is None:
|
||||
return {"ok": False, "output": f"cannot read {path}"}
|
||||
staged = self._accounts_stage("".join(f"{line}\n" for line in [*mine, public_key.strip()]))
|
||||
failed = self._accounts_write_key_file(name, home, path, [*mine, public_key.strip()])
|
||||
if failed is not None:
|
||||
return {"ok": False, "output": f"{path}: {_said(failed)}"}
|
||||
return {"ok": True, "output": f"key added to {path}"}
|
||||
|
||||
def _accounts_write_key_file(
|
||||
self, name: str, home: str, path: str, lines: list[str]
|
||||
) -> CommandResult | None:
|
||||
"""*lines* as *name*'s key file *path*, written as *name*; None, or what failed.
|
||||
|
||||
Without lines the file goes: nothing is staged from an empty stdin.
|
||||
"""
|
||||
if not lines:
|
||||
return self._accounts_run_as(name, [["rm", "-f", "--", path]])
|
||||
staged = self._accounts_stage("".join(f"{line}\n" for line in lines))
|
||||
try:
|
||||
failed = self._accounts_run_as(
|
||||
return self._accounts_run_as(
|
||||
name,
|
||||
[
|
||||
["mkdir", "-p", "-m", "700", "--", f"{home}/.ssh"],
|
||||
@@ -694,9 +818,49 @@ class LocalAccountsMixin:
|
||||
)
|
||||
finally:
|
||||
self._accounts_unstage(staged)
|
||||
|
||||
def _accounts_drop_from(
|
||||
self, name: str, home: str, path: str, blobs: set[str | None]
|
||||
) -> FixStepOutcomeDict:
|
||||
"""The keys of *blobs* out of *name*'s key file *path*; no output when none was in it."""
|
||||
lines = self._accounts_key_file(name, path)
|
||||
if lines is None:
|
||||
return {"ok": False, "output": f"cannot read {path}"}
|
||||
kept = [line for line in lines if _key_blob(line) not in blobs]
|
||||
if len(kept) == len(lines):
|
||||
return {"ok": True, "output": ""}
|
||||
failed = self._accounts_write_key_file(name, home, path, kept)
|
||||
if failed is not None:
|
||||
return {"ok": False, "output": f"{path}: {_said(failed)}"}
|
||||
return {"ok": True, "output": f"key added to {path}"}
|
||||
return {"ok": True, "output": f"{len(lines) - len(kept)} removed from {path}"}
|
||||
|
||||
def _accounts_drop_keys(self, name: str, keys: list[str]) -> FixStepOutcomeDict:
|
||||
home = self._accounts_home(name)
|
||||
if home is None:
|
||||
return {"ok": False, "output": f"no account {name}"}
|
||||
return self._account_drop_keys(name, home, keys)
|
||||
|
||||
def _account_drop_keys(self, name: str, home: str, keys: list[str]) -> FixStepOutcomeDict:
|
||||
"""Remove exactly *keys* from *name*'s key files in *home*, as *name*.
|
||||
|
||||
A key is matched by its base64, whatever a line's options or comment;
|
||||
every other line stays. Afterwards none of *keys* may be left in
|
||||
:meth:`_account_key_files`. A driver whose host keeps keys elsewhere
|
||||
overrides this, with :meth:`_account_add_key`.
|
||||
"""
|
||||
blobs = {_key_blob(key) for key in keys}
|
||||
said: list[str] = []
|
||||
for path in authorized_keys_paths(home):
|
||||
outcome = self._accounts_drop_from(name, home, path, blobs)
|
||||
if not outcome["ok"]:
|
||||
return outcome
|
||||
said += [outcome["output"]] if outcome["output"] else []
|
||||
left = self._account_read_keys(name, home)
|
||||
if left is None:
|
||||
return {"ok": False, "output": f"cannot read the key files of {name}"}
|
||||
if any(_key_blob(line) in blobs for line in left):
|
||||
return {"ok": False, "output": f"{name} still has a key that was to go"}
|
||||
return {"ok": True, "output": "\n".join(said) or f"none of the keys is in {home}/.ssh"}
|
||||
|
||||
def _accounts_key_file(self, name: str, path: str) -> list[str] | None:
|
||||
there = self.run_command(_command(["test", "-e", path]), privileged=True, timeout=30)
|
||||
|
||||
@@ -0,0 +1,318 @@
|
||||
"""Where a host's logs go beyond it: what rsyslog forwards, which files it reads
|
||||
in, and whether it gets the journal (NetOrk/netork#2053).
|
||||
|
||||
A log source (:mod:`napalm_device_types.log_sources`) says who writes a log;
|
||||
this says who reads it on. It is read in the same command, so a reading of
|
||||
the logs a host writes also says which of them leave it.
|
||||
|
||||
**rsyslog's configuration, read as text.** Both of its syntaxes are read:
|
||||
the legacy one (``authpriv.* @@logs:514``, ``& @host``, ``:programname, isequal,
|
||||
"sshd" @host``, ``$InputFileName``) and RainerScript (``action(type="omfwd" …)``,
|
||||
``input(type="imfile" …)``, ``ruleset(…) { … }``, ``if … then``). A rule is
|
||||
*complete* when it takes every message of its ruleset -- ``*.*`` with no
|
||||
condition around it. Anything narrower is partial, and keeps its selector or
|
||||
condition for the reader to see: which facility a unit logs to is not
|
||||
computed (decided in #2053).
|
||||
|
||||
**The journal reaches rsyslog** through ``imjournal``, or through ``imuxsock``
|
||||
when journald forwards to syslog (``ForwardToSyslog``). With no setting of its
|
||||
own a host runs on its distribution's default, which ``journald.conf`` ships
|
||||
commented out: ``#ForwardToSyslog=yes``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from typing import TypedDict
|
||||
|
||||
|
||||
class ForwardRuleDict(TypedDict):
|
||||
"""One rule sending messages to another host."""
|
||||
|
||||
protocol: str # "udp", "tcp" or "relp"
|
||||
target: str # the host, as configured
|
||||
port: int | None # None: the protocol's default
|
||||
selector: str # "*.*", "authpriv.*", or the condition around it
|
||||
complete: bool # every message of its ruleset goes
|
||||
ruleset: str | None # None: the default ruleset, which gets the journal and syslog
|
||||
|
||||
|
||||
class FileInputDict(TypedDict):
|
||||
"""A file rsyslog reads in (``imfile``), and the ruleset it hands it to."""
|
||||
|
||||
path: str # as configured: may hold a glob
|
||||
ruleset: str | None
|
||||
|
||||
|
||||
class RsyslogDict(TypedDict):
|
||||
rules: list[ForwardRuleDict]
|
||||
file_inputs: list[FileInputDict]
|
||||
imjournal: bool
|
||||
imuxsock: bool
|
||||
|
||||
|
||||
class LogForwardingDict(TypedDict):
|
||||
"""Where a host's logs go beyond it.
|
||||
|
||||
``journal`` is None when it cannot be told whether the journal reaches
|
||||
rsyslog: ``imuxsock`` is loaded but ``journald.conf`` says nothing.
|
||||
"""
|
||||
|
||||
rsyslog: bool # rsyslogd runs
|
||||
journal: bool | None # the journal reaches rsyslog
|
||||
rules: list[ForwardRuleDict]
|
||||
file_inputs: list[FileInputDict]
|
||||
|
||||
|
||||
#: Selectors that take every message.
|
||||
_EVERYTHING = {"*.*", "*.debug"}
|
||||
_PARAM_RE = re.compile(r'([A-Za-z][\w.]*)\s*=\s*"([^"]*)"')
|
||||
_CALL_RE = re.compile(
|
||||
r"^(?P<before>.*?)\b(?P<name>action|input|module|ruleset)\s*\((?P<args>.*)\)\s*;?$", re.S
|
||||
)
|
||||
_IF_RE = re.compile(r"^(?:else\s+)?if\s+(?P<cond>.*?)\s+then\b\s*(?P<rest>.*)$", re.S)
|
||||
_PROPERTY_RE = re.compile(r'^(?P<sel>:[^,]+,\s*!?[\w-]+,\s*"(?:[^"\\]|\\.)*")\s+(?P<action>.+)$')
|
||||
_LEGACY_RE = re.compile(r"^(?P<sel>\S+)\s+(?P<action>.+)$")
|
||||
_FORWARD_RE = re.compile(
|
||||
r"^(?:(?P<at>@@?)(?:\([^)]*\))?|:omrelp:)"
|
||||
r"(?:\[(?P<v6>[^\]]+)\]|(?P<host>[^:;\s]+))(?::(?P<port>\d+))?"
|
||||
)
|
||||
_FORWARD_TYPES = {"omfwd", "omrelp"}
|
||||
_TRUE = {"yes", "true", "1", "on"}
|
||||
|
||||
|
||||
def _uncomment(text: str) -> str:
|
||||
"""*text* without its comments: ``#`` to the end of the line, outside quotes."""
|
||||
out: list[str] = []
|
||||
quoted = commented = False
|
||||
for char in text:
|
||||
if char == "\n":
|
||||
quoted = commented = False
|
||||
elif commented:
|
||||
continue
|
||||
elif char == '"':
|
||||
quoted = not quoted
|
||||
elif char == "#" and not quoted:
|
||||
commented = True
|
||||
continue
|
||||
out.append(char)
|
||||
return "".join(out)
|
||||
|
||||
|
||||
class _Pieces:
|
||||
"""Cuts a configuration into statements, ``{`` and ``}``: a line runs on
|
||||
while a bracket is open, and a brace outside one stands as a piece of its own."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.pieces: list[str] = []
|
||||
self.current: list[str] = []
|
||||
self.depth = 0
|
||||
self.quoted = False
|
||||
|
||||
def close(self) -> None:
|
||||
piece = "".join(self.current).strip()
|
||||
if piece:
|
||||
self.pieces.append(piece)
|
||||
self.current.clear()
|
||||
|
||||
def bracket(self, char: str) -> None:
|
||||
if char in "([":
|
||||
self.depth += 1
|
||||
elif char in ")]":
|
||||
self.depth = max(self.depth - 1, 0)
|
||||
|
||||
def feed(self, char: str) -> None:
|
||||
if char == '"':
|
||||
self.quoted = not self.quoted
|
||||
elif not self.quoted:
|
||||
self.bracket(char)
|
||||
if self.depth == 0 and char in "{}\n":
|
||||
self.close()
|
||||
if char != "\n":
|
||||
self.pieces.append(char)
|
||||
return
|
||||
self.current.append(char)
|
||||
|
||||
|
||||
def _pieces(text: str) -> list[str]:
|
||||
"""Statements, ``{`` and ``}`` in order (:class:`_Pieces`)."""
|
||||
cutter = _Pieces()
|
||||
for char in text:
|
||||
cutter.feed(char)
|
||||
cutter.close()
|
||||
return cutter.pieces
|
||||
|
||||
|
||||
def _params(args: str) -> dict[str, str]:
|
||||
return {key.lower(): value for key, value in _PARAM_RE.findall(args)}
|
||||
|
||||
|
||||
def _port(value: str | None) -> int | None:
|
||||
return int(value) if value and value.isdigit() else None
|
||||
|
||||
|
||||
class _Reader:
|
||||
"""Walks the pieces of a configuration, keeping the blocks it is in."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.parsed: RsyslogDict = {
|
||||
"rules": [],
|
||||
"file_inputs": [],
|
||||
"imjournal": False,
|
||||
"imuxsock": False,
|
||||
}
|
||||
self.stack: list[tuple[str, str | None]] = []
|
||||
self.pending: tuple[str, str | None] | None = None
|
||||
self.legacy_ruleset: str | None = None
|
||||
self.last_selector = "*.*"
|
||||
|
||||
def ruleset(self) -> str | None:
|
||||
names = [value for kind, value in self.stack if kind == "ruleset"]
|
||||
return names[-1] if names else self.legacy_ruleset
|
||||
|
||||
def condition(self, once: str | None) -> str | None:
|
||||
conditions = [value for kind, value in self.stack if kind == "if"]
|
||||
return once or (conditions[-1] if conditions else None)
|
||||
|
||||
def forward(
|
||||
self, protocol: str, target: str, port: int | None, selector: str, once: str | None
|
||||
) -> None:
|
||||
condition = self.condition(once)
|
||||
self.parsed["rules"].append(
|
||||
{
|
||||
"protocol": protocol,
|
||||
"target": target,
|
||||
"port": port,
|
||||
"selector": condition or selector,
|
||||
"complete": condition is None and selector in _EVERYTHING,
|
||||
"ruleset": self.ruleset(),
|
||||
}
|
||||
)
|
||||
|
||||
def piece(self, piece: str) -> None:
|
||||
if piece == "{":
|
||||
self.stack.append(self.pending or ("block", None))
|
||||
self.pending = None
|
||||
return
|
||||
if piece == "}":
|
||||
if self.stack:
|
||||
self.stack.pop()
|
||||
return
|
||||
if piece == "else":
|
||||
self.pending = ("if", "else")
|
||||
return
|
||||
# A condition whose action follows on its own: the action is under it.
|
||||
once = self.pending[1] if self.pending and self.pending[0] == "if" else None
|
||||
self.pending = None
|
||||
match = _IF_RE.match(piece)
|
||||
if match:
|
||||
if match["rest"]:
|
||||
self.statement(match["rest"], once=match["cond"])
|
||||
else:
|
||||
self.pending = ("if", match["cond"])
|
||||
return
|
||||
self.statement(piece, once=once)
|
||||
|
||||
def statement(self, text: str, once: str | None) -> None:
|
||||
call = _CALL_RE.match(text)
|
||||
if call:
|
||||
self.call(call["name"], _params(call["args"]), call["before"].strip(), once)
|
||||
elif text.startswith("$"):
|
||||
self.directive(text)
|
||||
else:
|
||||
self.legacy(text, once)
|
||||
|
||||
def call(self, name: str, params: dict[str, str], before: str, once: str | None) -> None:
|
||||
if name == "module":
|
||||
self.module(params.get("load", ""))
|
||||
elif name == "ruleset":
|
||||
self.pending = ("ruleset", params.get("name"))
|
||||
elif name == "input" and params.get("type") == "imfile" and params.get("file"):
|
||||
self.parsed["file_inputs"].append(
|
||||
{"path": params["file"], "ruleset": params.get("ruleset")}
|
||||
)
|
||||
elif name == "action":
|
||||
self.action(params, before, once)
|
||||
|
||||
def action(self, params: dict[str, str], before: str, once: str | None) -> None:
|
||||
selector = self.last_selector if before == "&" else (before or "*.*")
|
||||
self.last_selector = selector
|
||||
kind = params.get("type", "")
|
||||
if kind in _FORWARD_TYPES and params.get("target"):
|
||||
protocol = "relp" if kind == "omrelp" else params.get("protocol", "udp").lower()
|
||||
self.forward(protocol, params["target"], _port(params.get("port")), selector, once)
|
||||
|
||||
def module(self, name: str) -> None:
|
||||
if name == "imjournal":
|
||||
self.parsed["imjournal"] = True
|
||||
elif name == "imuxsock":
|
||||
self.parsed["imuxsock"] = True
|
||||
|
||||
def directive(self, text: str) -> None:
|
||||
name, _, value = text.partition(" ")
|
||||
name, value = name.lower(), value.strip()
|
||||
if name == "$modload":
|
||||
self.module(value)
|
||||
elif name == "$inputfilename" and value:
|
||||
self.parsed["file_inputs"].append({"path": value, "ruleset": self.legacy_ruleset})
|
||||
elif name == "$ruleset":
|
||||
self.legacy_ruleset = None if value == "RSYSLOG_DefaultRuleset" else value
|
||||
|
||||
def legacy(self, text: str, once: str | None) -> None:
|
||||
match = _PROPERTY_RE.match(text) or _LEGACY_RE.match(text)
|
||||
if not match:
|
||||
return
|
||||
selector = self.last_selector if match["sel"] == "&" else match["sel"]
|
||||
self.last_selector = selector
|
||||
forward = _FORWARD_RE.match(match["action"].strip())
|
||||
if forward:
|
||||
protocol = {"@": "udp", "@@": "tcp"}.get(forward["at"] or "", "relp")
|
||||
self.forward(
|
||||
protocol, forward["v6"] or forward["host"], _port(forward["port"]), selector, once
|
||||
)
|
||||
|
||||
|
||||
def parse_rsyslog(text: str) -> RsyslogDict:
|
||||
"""rsyslog's configuration -- ``rsyslog.conf`` and ``rsyslog.d`` as one text --
|
||||
read for its forwarding rules, its file inputs, and the inputs that take
|
||||
the journal."""
|
||||
reader = _Reader()
|
||||
for piece in _pieces(_uncomment(text)):
|
||||
reader.piece(piece)
|
||||
return reader.parsed
|
||||
|
||||
|
||||
def journal_to_syslog(lines: list[str]) -> bool | None:
|
||||
"""Whether journald forwards to syslog, from its ``ForwardToSyslog`` lines in
|
||||
the order journald reads them: the last setting wins; with none, the
|
||||
default the distribution ships commented out; None when nothing says."""
|
||||
setting = shipped = None
|
||||
for line in lines:
|
||||
commented = line.lstrip().startswith("#")
|
||||
_, _, value = line.lstrip("# \t").partition("=")
|
||||
if commented:
|
||||
shipped = value.strip().lower() in _TRUE
|
||||
else:
|
||||
setting = value.strip().lower() in _TRUE
|
||||
return setting if setting is not None else shipped
|
||||
|
||||
|
||||
def log_forwarding(
|
||||
running: bool, rsyslog: RsyslogDict, journald_lines: list[str]
|
||||
) -> LogForwardingDict:
|
||||
"""Where a host's logs go: *rsyslog* is its configuration, read whether it runs or not."""
|
||||
if not running:
|
||||
return {"rsyslog": False, "journal": False, "rules": [], "file_inputs": []}
|
||||
journal: bool | None
|
||||
if rsyslog["imjournal"]:
|
||||
journal = True
|
||||
elif rsyslog["imuxsock"]:
|
||||
journal = journal_to_syslog(journald_lines)
|
||||
else:
|
||||
journal = False
|
||||
return {
|
||||
"rsyslog": True,
|
||||
"journal": journal,
|
||||
"rules": rsyslog["rules"],
|
||||
"file_inputs": rsyslog["file_inputs"],
|
||||
}
|
||||
@@ -0,0 +1,365 @@
|
||||
"""Log sources: which logs a host writes, and who writes each of them.
|
||||
|
||||
Before anything can watch a host's logs -- a person, CrowdSec, a syslog
|
||||
forwarder -- it has to know which logs there are. That is read the same way on
|
||||
every Linux host, so the command and its parse live here once, and a driver
|
||||
only carries the command across (NetOrk/netork#2049).
|
||||
|
||||
**Three kinds.** journald names every unit it holds entries for
|
||||
(``journalctl -F _SYSTEMD_UNIT``); only services count, not login sessions.
|
||||
``/proc/<pid>/fd`` shows the files running processes hold open, a log being a
|
||||
``*.log`` / ``*.log.N`` file or anything under ``/var/log``; ``stat -L`` on the
|
||||
descriptor gives its size and last write, also for a file inside a container,
|
||||
whose path is the container's own. The engine's own file of a container's
|
||||
output (``/var/lib/docker/containers/<id>/…``) is that container's stream, not
|
||||
a file of ``dockerd``.
|
||||
|
||||
**Who writes it.** Only a process holding the file open for writing writes
|
||||
it: the access mode in ``/proc/<pid>/fdinfo/<fd>``'s ``flags`` is ``O_WRONLY`` or
|
||||
``O_RDWR``. A reader -- CrowdSec, Promtail, rsyslog's ``imfile``, a ``tail -F``
|
||||
-- holds it too, and is no producer (NetOrk/netork#2092). ``/proc/<pid>/cgroup``
|
||||
says which systemd service or container the first writer runs in, as for
|
||||
listening sockets (:mod:`napalm_device_types.listening`). A file only readers
|
||||
hold is listed without a producer; one only readers in a container hold is left
|
||||
out, its path being the reader's view of a mount. journald's own files, files
|
||||
already deleted and a database's write-ahead log (LevelDB's ``000004.log``) are
|
||||
left out.
|
||||
|
||||
**Where they go** is read in the same round trip
|
||||
(:mod:`napalm_device_types.log_forwarding`, NetOrk/netork#2053): whether
|
||||
rsyslog runs, its configuration -- each line behind ``| ``, so that none reads
|
||||
as a section -- and journald's ``ForwardToSyslog`` lines in the order journald
|
||||
reads them.
|
||||
|
||||
**One round trip, root, and without it.** As for listening sockets the script
|
||||
goes as one ``sh -c`` argument. Without root only the login user's processes
|
||||
show their files; the command then runs again unprivileged, and the reading
|
||||
says it is not ``attributed``. A report whose end is missing raises: a list cut
|
||||
short must never read as logs that went away, and so must a host whose
|
||||
``find`` cannot list open files (busybox): it raises
|
||||
:class:`LogSourcesUnavailable`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from datetime import UTC, datetime
|
||||
from shlex import quote
|
||||
from typing import TYPE_CHECKING, TypedDict
|
||||
|
||||
from napalm_device_types.listening import _cgroup_paths, _container, _sections, _unit
|
||||
from napalm_device_types.log_forwarding import LogForwardingDict, log_forwarding, parse_rsyslog
|
||||
from napalm_device_types.terminal import strip_terminal_codes
|
||||
|
||||
|
||||
class LogSourceDict(TypedDict):
|
||||
"""A log the host writes: a unit's journal, a file a process holds open, or a
|
||||
container's stream as the engine keeps it.
|
||||
|
||||
``path`` is as its writer sees it: a file a container writes is named inside
|
||||
that container. Whether anything reads the log is the consumer's question.
|
||||
"""
|
||||
|
||||
kind: str # "journald", "file" or "container"
|
||||
path: str | None # the file; None for a journal
|
||||
unit: str | None # the journal's unit, or the file writer's service; without ".service"
|
||||
container_id: str | None # the full container ID: the writer's, or the stream's
|
||||
process: str | None # the first process writing the file; None when only readers hold it
|
||||
size: int | None # bytes; None for a journal, or when the file could not be read
|
||||
modified: str | None # the last write, ISO 8601 UTC; None as for size
|
||||
|
||||
|
||||
class LogSourcesDict(TypedDict):
|
||||
"""What ``LogSourcesMixin.get_log_sources`` read.
|
||||
|
||||
``attributed`` is False when the reading ran without root: only the login
|
||||
user's own processes show their files then, so a missing log means "not
|
||||
told", not "not there". ``journald`` is False on a host without systemd's
|
||||
journal, where no unit has one.
|
||||
"""
|
||||
|
||||
attributed: bool
|
||||
journald: bool
|
||||
sources: list[LogSourceDict]
|
||||
forwarding: LogForwardingDict | None # where the logs go beyond the host
|
||||
|
||||
|
||||
_BEGIN = "LOGS_BEGIN"
|
||||
_END = "LOGS_END"
|
||||
|
||||
#: One line, POSIX ``sh`` with GNU ``find`` (``-lname``, ``-printf``), read-only.
|
||||
#: The frame markers are printed in two halves so that a transport which echoes
|
||||
#: the command does not show them early. ``-printf '%p %l'`` puts the descriptor
|
||||
#: before its target, so a target with a space in it stays whole.
|
||||
LOG_SOURCES_COMMAND = (
|
||||
"PATH=$PATH:/usr/sbin:/sbin; "
|
||||
"printf '%s%s\\n' LOGS_ BEGIN; "
|
||||
"if [ -d /run/systemd/system ] && command -v journalctl >/dev/null 2>&1; then "
|
||||
"j=journalctl; command -v timeout >/dev/null 2>&1 && j='timeout 30 journalctl'; "
|
||||
"echo '[journald]'; $j -F _SYSTEMD_UNIT 2>/dev/null; fi; "
|
||||
"if find / -maxdepth 0 -lname x -printf '' >/dev/null 2>&1; then "
|
||||
"o=$(find /proc/[0-9]*/fd -mindepth 1 -maxdepth 1 -type l "
|
||||
"\\( -lname '*.log' -o -lname '*.log.[0-9]*' -o -lname '/var/log/*' \\) "
|
||||
"-printf '%p %l\\n' 2>/dev/null); "
|
||||
"echo '[open]'; printf '%s\\n' \"$o\"; "
|
||||
"echo '[stat]'; printf '%s\\n' \"$o\" | cut -d' ' -f1 "
|
||||
"| xargs -r stat -L -c '%n %s %Y' 2>/dev/null; "
|
||||
"echo '[flags]'; printf '%s\\n' \"$o\" | cut -d' ' -f1 | sed 's|/fd/|/fdinfo/|' "
|
||||
"| xargs -r grep -H '^flags:' 2>/dev/null; "
|
||||
"pids=$(printf '%s\\n' \"$o\" | cut -d/ -f3 | sort -un); "
|
||||
"echo '[cgroups]'; for p in $pids; do sed \"s|^|$p |\" /proc/$p/cgroup 2>/dev/null; done; "
|
||||
"echo '[comm]'; for p in $pids; do c=$(cat /proc/$p/comm 2>/dev/null) && echo \"$p $c\"; done; "
|
||||
"else echo '[no-find]'; fi; "
|
||||
"echo '[rsyslog]'; { pidof rsyslogd || pgrep -x rsyslogd; } >/dev/null 2>&1 && echo running; "
|
||||
"echo '[rsyslog-conf]'; cat /etc/rsyslog.conf /etc/rsyslog.d/*.conf 2>/dev/null "
|
||||
"| sed 's/^/| /'; "
|
||||
"echo '[journald-conf]'; cat /etc/systemd/journald.conf "
|
||||
"/usr/lib/systemd/journald.conf.d/*.conf /run/systemd/journald.conf.d/*.conf "
|
||||
"/etc/systemd/journald.conf.d/*.conf 2>/dev/null "
|
||||
"| grep -i '^[#[:space:]]*ForwardToSyslog'; "
|
||||
"printf '%s%s\\n' LOGS_ END"
|
||||
)
|
||||
|
||||
#: ``/proc/900/fd/5`` -- the process and its descriptor.
|
||||
_FD_RE = re.compile(r"^/proc/(\d+)/fd/\d+$")
|
||||
#: ``/proc/900/fdinfo/5:flags:<tab>0102001`` -- the descriptor's open flags, octal.
|
||||
_FLAGS_RE = re.compile(r"^/proc/(\d+)/fdinfo/(\d+):flags:\s*([0-7]+)$")
|
||||
#: The access mode bits of the open flags: 0 is O_RDONLY, 1 O_WRONLY, 2 O_RDWR.
|
||||
_ACCESS_MODE = 0o3
|
||||
#: The engine's file of a container's output: json-file and local drivers.
|
||||
_STREAM_RE = re.compile(r"^/var/lib/docker/containers/([0-9a-f]{64})/")
|
||||
#: Files that are no log to read: journald's store, a file already deleted, and
|
||||
#: the write-ahead log LevelDB and RocksDB name ``000004.log``.
|
||||
_NOT_A_SOURCE = re.compile(r"^/var/log/journal/|\.journal~?$| \(deleted\)$|/\d{6}\.log$")
|
||||
_SERVICE_SUFFIX = ".service"
|
||||
_NO_FIND = "no-find"
|
||||
|
||||
|
||||
class LogSourcesUnavailable(NotImplementedError):
|
||||
"""The host's ``find`` cannot list open files (busybox); nothing to read, nothing to retry."""
|
||||
|
||||
|
||||
def _frame(output: str) -> list[str]:
|
||||
lines = [line.strip() for line in strip_terminal_codes(output).splitlines()]
|
||||
try:
|
||||
start = lines.index(_BEGIN)
|
||||
end = lines.index(_END, start)
|
||||
except ValueError:
|
||||
raise ValueError("no intact log source report in the output") from None
|
||||
return [line for line in lines[start + 1 : end] if line]
|
||||
|
||||
|
||||
def _stats(lines: list[str]) -> dict[str, tuple[int, str]]:
|
||||
"""Each descriptor's size and last write: ``/proc/900/fd/5 2048 1762250200``."""
|
||||
stats: dict[str, tuple[int, str]] = {}
|
||||
for line in lines:
|
||||
fd, _, rest = line.partition(" ")
|
||||
size, _, mtime = rest.partition(" ")
|
||||
if size.isdigit() and mtime.isdigit():
|
||||
stats[fd] = (int(size), datetime.fromtimestamp(int(mtime), UTC).isoformat())
|
||||
return stats
|
||||
|
||||
|
||||
def _names(lines: list[str]) -> dict[int, str]:
|
||||
names: dict[int, str] = {}
|
||||
for line in lines:
|
||||
pid, _, name = line.partition(" ")
|
||||
if pid.isdigit() and name:
|
||||
names[int(pid)] = name
|
||||
return names
|
||||
|
||||
|
||||
def _journals(units: list[str]) -> list[LogSourceDict]:
|
||||
services = {u[: -len(_SERVICE_SUFFIX)] for u in units if u.endswith(_SERVICE_SUFFIX)}
|
||||
return [
|
||||
{
|
||||
"kind": "journald",
|
||||
"path": None,
|
||||
"unit": unit,
|
||||
"container_id": None,
|
||||
"process": None,
|
||||
"size": None,
|
||||
"modified": None,
|
||||
}
|
||||
for unit in sorted(services)
|
||||
]
|
||||
|
||||
|
||||
def _held(lines: list[str]) -> list[tuple[int, str, str]]:
|
||||
"""(pid, descriptor, target) for every log file a process holds, by pid."""
|
||||
held = []
|
||||
for line in lines:
|
||||
fd, _, target = line.partition(" ")
|
||||
match = _FD_RE.match(fd)
|
||||
if match and target.startswith("/") and not _NOT_A_SOURCE.search(target):
|
||||
held.append((int(match.group(1)), fd, target))
|
||||
return sorted(held)
|
||||
|
||||
|
||||
def _writes(lines: list[str]) -> dict[str, bool]:
|
||||
"""Whether each descriptor is open for writing, by its ``/proc/<pid>/fd/<fd>``."""
|
||||
writes: dict[str, bool] = {}
|
||||
for line in lines:
|
||||
match = _FLAGS_RE.match(line)
|
||||
if match:
|
||||
pid, fd, flags = match.groups()
|
||||
writes[f"/proc/{pid}/fd/{fd}"] = int(flags, 8) & _ACCESS_MODE != 0
|
||||
return writes
|
||||
|
||||
|
||||
_Key = tuple[str, str | None, str]
|
||||
|
||||
|
||||
def _holders(lines: list[str], paths: dict[int, str]) -> dict[_Key, list[tuple[int, str]]]:
|
||||
"""The (pid, descriptor) pairs holding each log, by kind, container and path."""
|
||||
holders: dict[_Key, list[tuple[int, str]]] = {}
|
||||
for pid, fd, target in _held(lines):
|
||||
stream = _STREAM_RE.match(target)
|
||||
kind, container = (
|
||||
("container", stream.group(1)) if stream else ("file", _container(paths.get(pid)))
|
||||
)
|
||||
holders.setdefault((kind, container, target), []).append((pid, fd))
|
||||
return holders
|
||||
|
||||
|
||||
def _writer(held: list[tuple[int, str]], writes: dict[str, bool] | None) -> int | None:
|
||||
"""The first process writing; the first holding for a report without flags,
|
||||
as an older command printed it."""
|
||||
if writes is None:
|
||||
return held[0][0]
|
||||
return next((pid for pid, fd in held if writes.get(fd)), None)
|
||||
|
||||
|
||||
def _producer(
|
||||
kind: str, writer: int | None, paths: dict[int, str], names: dict[int, str]
|
||||
) -> tuple[str | None, str | None]:
|
||||
"""The writer's service and name; a stream's service is its container."""
|
||||
if writer is None:
|
||||
return None, None
|
||||
return (_unit(paths.get(writer)) if kind == "file" else None), names.get(writer)
|
||||
|
||||
|
||||
def _files(sections: dict[str, list[str]]) -> list[LogSourceDict]:
|
||||
stats = _stats(sections.get("stat", []))
|
||||
paths = _cgroup_paths(sections.get("cgroups", []))
|
||||
names = _names(sections.get("comm", []))
|
||||
writes = _writes(sections["flags"]) if "flags" in sections else None
|
||||
found: list[LogSourceDict] = []
|
||||
for (kind, container, target), held in _holders(sections.get("open", []), paths).items():
|
||||
writer = _writer(held, writes)
|
||||
if writer is None and kind == "file" and container is not None:
|
||||
continue # a reader's view of a mount, inside its container
|
||||
unit, process = _producer(kind, writer, paths, names)
|
||||
size, modified = next((stats[fd] for _, fd in held if fd in stats), (None, None))
|
||||
found.append(
|
||||
{
|
||||
"kind": kind,
|
||||
"path": target,
|
||||
"unit": unit,
|
||||
"container_id": container,
|
||||
"process": process,
|
||||
"size": size,
|
||||
"modified": modified,
|
||||
}
|
||||
)
|
||||
return found
|
||||
|
||||
|
||||
def _order(source: LogSourceDict) -> tuple[str, str, str]:
|
||||
return (source["kind"], source["container_id"] or "", source["path"] or source["unit"] or "")
|
||||
|
||||
|
||||
def parse_log_sources(output: str) -> tuple[bool, list[LogSourceDict]]:
|
||||
"""Parse what :data:`LOG_SOURCES_COMMAND` printed: whether the host has a
|
||||
journal, and every source, sorted by kind, container and name.
|
||||
|
||||
:raises LogSourcesUnavailable: when the host has no GNU ``find``.
|
||||
:raises ValueError: when the output carries no intact report.
|
||||
"""
|
||||
sections = _sections(_frame(output))
|
||||
if _NO_FIND in sections:
|
||||
raise LogSourcesUnavailable("the host's find cannot list open files (no GNU find)")
|
||||
journald = "journald" in sections
|
||||
sources = _journals(sections.get("journald", [])) + _files(sections)
|
||||
return journald, sorted(sources, key=_order)
|
||||
|
||||
|
||||
def _reading(output: str, *, attributed: bool) -> LogSourcesDict:
|
||||
journald, sources = parse_log_sources(output)
|
||||
forwarding = parse_log_forwarding(output)
|
||||
return {
|
||||
"attributed": attributed,
|
||||
"journald": journald,
|
||||
"sources": sources,
|
||||
"forwarding": forwarding,
|
||||
}
|
||||
|
||||
|
||||
def parse_log_forwarding(output: str) -> LogForwardingDict | None:
|
||||
"""Parse where the logs go from what :data:`LOG_SOURCES_COMMAND` printed;
|
||||
None for a report without the sections, as an older command printed it.
|
||||
|
||||
:raises ValueError: when the output carries no intact report.
|
||||
"""
|
||||
sections = _sections(_frame(output))
|
||||
if "rsyslog" not in sections:
|
||||
return None
|
||||
config = "\n".join(
|
||||
line[2:] if line.startswith("| ") else line[1:] for line in sections.get("rsyslog-conf", [])
|
||||
)
|
||||
running = "running" in sections["rsyslog"]
|
||||
return log_forwarding(running, parse_rsyslog(config), sections.get("journald-conf", []))
|
||||
|
||||
|
||||
class LogSourcesMixin:
|
||||
"""Adds :meth:`get_log_sources` to a driver that can run a command on a Linux host.
|
||||
|
||||
The template form (README, "Function classes"): the command and its parse
|
||||
are the same everywhere, so they are concrete here, and a driver supplies
|
||||
only :meth:`_run_log_sources_command` -- how a command reaches its host, and
|
||||
how it gains root there. Mixed in by the drivers that can, so
|
||||
``hasattr(driver, "get_log_sources")`` stays truthful.
|
||||
"""
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def _run_log_sources_command(self, command: str, *, privileged: bool) -> str:
|
||||
"""Run *command* on the host and return what it printed; as root
|
||||
when *privileged*. The command is a single ``sh -c`` invocation."""
|
||||
...
|
||||
|
||||
def get_log_sources(self) -> LogSourcesDict:
|
||||
"""
|
||||
Returns every log the host writes, with the service or container writing it.
|
||||
|
||||
* attributed (bool) - read as root, so every process's files are listed
|
||||
* journald (bool) - the host keeps systemd's journal
|
||||
* sources (list) - see :class:`~napalm_device_types.models.LogSourceDict`
|
||||
* forwarding (dict) - where the logs go beyond the host, see
|
||||
:class:`~napalm_device_types.log_forwarding.LogForwardingDict`
|
||||
|
||||
Example::
|
||||
|
||||
{
|
||||
"attributed": True,
|
||||
"journald": True,
|
||||
"sources": [
|
||||
{"kind": "journald", "path": None, "unit": "nginx",
|
||||
"container_id": None, "process": None, "size": None, "modified": None},
|
||||
{"kind": "file", "path": "/var/log/nginx/access.log", "unit": "nginx",
|
||||
"container_id": None, "process": "nginx", "size": 2048,
|
||||
"modified": "2026-11-04T09:56:40+00:00"},
|
||||
],
|
||||
}
|
||||
|
||||
:raises LogSourcesUnavailable: if the host has no GNU ``find``.
|
||||
:raises ValueError: if neither reading carried an intact report.
|
||||
"""
|
||||
command = f"sh -c {quote(LOG_SOURCES_COMMAND)}"
|
||||
try:
|
||||
output = self._run_log_sources_command(command, privileged=True)
|
||||
return _reading(output, attributed=True)
|
||||
except ValueError:
|
||||
pass
|
||||
output = self._run_log_sources_command(command, privileged=False)
|
||||
return _reading(output, attributed=False)
|
||||
@@ -85,8 +85,8 @@ class InterruptedPackageDict(TypedDict):
|
||||
"""A package an interrupted package manager run left halfway."""
|
||||
|
||||
name: str
|
||||
#: dpkg's word for it: "half-installed", "unpacked", "half-configured",
|
||||
#: "triggers-awaited", "triggers-pending", or "reinstall-required".
|
||||
#: The manager's word for it: dpkg's and opkg's ("unpacked", "half-configured", ...),
|
||||
#: rpm's "duplicate", apk's "broken-files" and "broken-script" (``package_managers``).
|
||||
state: str
|
||||
|
||||
|
||||
@@ -111,9 +111,9 @@ class HostStatusDict(TypedDict):
|
||||
reboot_reason: str | None
|
||||
#: True when the host installs updates on its own (unattended-upgrades, dnf-automatic).
|
||||
auto_updates: bool | None
|
||||
#: True when a package manager run was cut off and has to be finished
|
||||
#: (``dpkg --configure -a``) before anything else installs; None when it
|
||||
#: cannot tell, or a package manager is running right now.
|
||||
#: The package manager read (dpkg, dnf, yum, apk, opkg; None: none) and whether a run
|
||||
#: of it was cut off, left halfway (None: cannot tell, or one is running right now).
|
||||
package_manager: NotRequired[str | None]
|
||||
package_manager_interrupted: NotRequired[bool | None]
|
||||
#: The packages that run left halfway; empty when only its journal says so.
|
||||
interrupted_packages: NotRequired[list[InterruptedPackageDict]]
|
||||
@@ -1038,15 +1038,6 @@ class ProcessDict(TypedDict):
|
||||
command: str # full command line
|
||||
|
||||
|
||||
class CronJobDict(TypedDict):
|
||||
"""A scheduled cron task."""
|
||||
|
||||
user: str # owner of the crontab entry
|
||||
schedule: str # five-field cron expression, e.g. "0 * * * *"
|
||||
command: str # shell command to execute
|
||||
description: NotRequired[str] # optional inline comment
|
||||
|
||||
|
||||
class ApplyUpdatesResultDict(TypedDict):
|
||||
"""Result of an ``apply_updates()`` call."""
|
||||
|
||||
|
||||
@@ -15,7 +15,6 @@ from typing import TYPE_CHECKING
|
||||
from napalm_device_types.base import DeviceTypeDriver
|
||||
from napalm_device_types.health_metrics import HealthMetricsMixin
|
||||
from napalm_device_types.models import (
|
||||
CronJobDict,
|
||||
DeviceActionResultDict,
|
||||
ProcessDict,
|
||||
SNMPConfigDict,
|
||||
@@ -131,34 +130,6 @@ class OSDriver(
|
||||
"""
|
||||
...
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Cron jobs
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def get_cron_jobs(self) -> list[CronJobDict]:
|
||||
"""
|
||||
Returns scheduled cron tasks from all user crontabs and ``/etc/cron.d``.
|
||||
|
||||
Each entry contains:
|
||||
|
||||
* user (string) - owner of the crontab entry
|
||||
* schedule (string) - five-field cron expression (e.g. ``"0 * * * *"``)
|
||||
* command (string) - shell command to execute
|
||||
* description (string, optional) - inline comment text if present
|
||||
|
||||
Example::
|
||||
|
||||
[
|
||||
{
|
||||
"user": "root",
|
||||
"schedule": "0 4 * * *",
|
||||
"command": "/usr/local/bin/backup.sh",
|
||||
"description": "nightly backup",
|
||||
},
|
||||
]
|
||||
"""
|
||||
...
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# SNMP
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
@@ -0,0 +1,194 @@
|
||||
"""Whether dnf or yum, apk or opkg was left halfway: the host status's answer for
|
||||
a host without dpkg (NetOrk/netork#851).
|
||||
|
||||
Read the same way as dpkg's state (:mod:`napalm_device_types.host_status`): in the
|
||||
one host status command, without root, and changing nothing. Each manager is asked
|
||||
only where it exists, and while one of its processes runs -- from
|
||||
``/proc/<pid>/comm``, which every Linux has and BusyBox's ``ps`` cannot narrow to --
|
||||
its packages are halfway on purpose and the answer is unknown.
|
||||
|
||||
**dnf and yum** (rpm). A transaction cut off after rpm installed the new version
|
||||
of a package and before it erased the old one leaves both in the rpm database:
|
||||
what ``dnf check --duplicates`` and ``package-cleanup --dupes`` report. Packages
|
||||
dnf installs side by side on purpose do not count: its default ``installonlypkgs``
|
||||
(``kernel``, ``installonlypkg(kernel)`` and the rest), plus any the host adds in
|
||||
``dnf.conf``, asked of rpm by what they provide; and ``gpg-pubkey``, one entry
|
||||
per imported key. Also interrupted is a transaction that never ended: dnf 4's
|
||||
history (libdnf's ``trans`` table) holds its row with ``state`` 0 until
|
||||
``finish()`` writes 1 (done) or 2 (error), so the newest row at 0 is one cut off;
|
||||
yum leaves a ``transaction-all.*`` journal in ``/var/lib/yum``, which
|
||||
``yum-complete-transaction`` reads (one renamed ``.disabled`` is given up). The
|
||||
history is read with Python's sqlite3 module in read-only mode, as a stock EL host
|
||||
has no ``sqlite3`` program and a ``dnf`` command loads every plugin and may reach
|
||||
for its repositories. dnf 5's history lives elsewhere and is not read; its
|
||||
duplicates are.
|
||||
|
||||
**apk** (Alpine, OpenWrt from 25.12). apk marks an installed package whose files
|
||||
or scripts failed with ``f:`` in ``/lib/apk/db/installed`` -- ``f`` broken files,
|
||||
``s`` a broken script; ``x`` (xattrs) and ``S`` (the checksum kind) are no fault --
|
||||
and those are exactly the packages ``apk fix`` without arguments picks. A run
|
||||
killed outright leaves no such mark; its next run usually repeats what it left.
|
||||
|
||||
**opkg** (OpenWrt before 25.12). ``/usr/lib/opkg/status`` holds
|
||||
``Status: <want> <flags> <state>`` per package. A run cut off, or a postinst that
|
||||
failed, leaves ``unpacked`` (or ``half-configured``, ``half-installed``), and the
|
||||
``reinstreq`` flag asks for a reinstall; ``opkg configure`` finishes every unpacked
|
||||
package.
|
||||
|
||||
What to do about it is not decided here.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import NamedTuple
|
||||
|
||||
from napalm_device_types.models import InterruptedPackageDict
|
||||
|
||||
#: dnf's default ``installonlypkgs``: names and what such packages provide.
|
||||
_INSTALLONLY = (
|
||||
"kernel",
|
||||
"kernel-PAE",
|
||||
"installonlypkg(kernel)",
|
||||
"installonlypkg(kernel-module)",
|
||||
"installonlypkg(vm)",
|
||||
"multiversion(kernel)",
|
||||
)
|
||||
#: Where a host adds its own installonly packages, dnf's then yum's.
|
||||
_DNF_CONF = "/etc/dnf/dnf.conf /etc/yum.conf"
|
||||
DNF_HISTORY = "/var/lib/dnf/history.sqlite"
|
||||
YUM_DIR = "/var/lib/yum"
|
||||
APK_INSTALLED = "/lib/apk/db/installed"
|
||||
OPKG_STATUS = "/usr/lib/opkg/status"
|
||||
|
||||
#: manager -> the process names that leave its packages halfway while they run.
|
||||
_RUNNING = {
|
||||
"rpm": frozenset({"rpm", "dnf", "dnf5", "yum", "dnf-automatic", "packagekitd"}),
|
||||
"apk": frozenset({"apk"}),
|
||||
"opkg": frozenset({"opkg"}),
|
||||
}
|
||||
_UNFINISHED = "0"
|
||||
_YUM_JOURNAL = "transaction-all."
|
||||
_YUM_GIVEN_UP = ".disabled"
|
||||
_APK_FLAGS = (("f", "broken-files"), ("s", "broken-script"))
|
||||
_OPKG_HALFWAY = frozenset({"unpacked", "half-configured", "half-installed"})
|
||||
_OPKG_REINSTALL = "reinstreq"
|
||||
|
||||
#: The newest transaction in dnf's history as ``<id> <state>``; nothing when unreadable.
|
||||
#: One literal, with :data:`DNF_HISTORY` spelt out: a fixed query, nothing put into it.
|
||||
_DNF_HISTORY_PY = (
|
||||
"import sqlite3; "
|
||||
"c = sqlite3.connect('file:/var/lib/dnf/history.sqlite?mode=ro', uri=True); "
|
||||
"r = c.execute('SELECT id, state FROM trans ORDER BY id DESC LIMIT 1').fetchone(); "
|
||||
"r and print(*r)"
|
||||
)
|
||||
|
||||
_RUNNING_PART = (
|
||||
"echo '[pm-running]'; cat /proc/[0-9]*/comm 2>/dev/null | grep -xE '"
|
||||
+ "|".join(sorted(set().union(*_RUNNING.values())))
|
||||
+ "' | sort -u; "
|
||||
)
|
||||
_RPM_PART = (
|
||||
"if command -v rpm >/dev/null 2>&1 && { command -v dnf || command -v yum; } >/dev/null 2>&1; "
|
||||
"then echo '[rpm-frontend]'; { command -v dnf || command -v yum; } | sed 's#.*/##'; "
|
||||
f"echo '[rpm-installonly]'; io=\"{' '.join(_INSTALLONLY)} "
|
||||
f"$(sed -n 's/^[[:space:]]*installonlypkgs[[:space:]]*=//p' {_DNF_CONF} 2>/dev/null "
|
||||
"| tr ',' ' ')\"; rpm -q --whatprovides $io --qf '%{NAME}\\n' 2>/dev/null "
|
||||
"| grep -v ' ' | sort -u; "
|
||||
"echo '[rpm-dupes]'; rpm -qa --qf '%{NAME} %{ARCH}\\n' 2>/dev/null "
|
||||
"| grep -v '^gpg-pubkey ' | sort | uniq -d | head -n 50; "
|
||||
f"echo '[yum-journal]'; ls -1 {YUM_DIR} 2>/dev/null | grep '^transaction-all\\.' "
|
||||
"| head -n 20; "
|
||||
"echo '[dnf-history]'; for py in /usr/libexec/platform-python python3; do "
|
||||
f'if command -v $py >/dev/null 2>&1; then $py -c "{_DNF_HISTORY_PY}" 2>/dev/null; '
|
||||
"break; fi; done; fi; "
|
||||
)
|
||||
_APK_PART = (
|
||||
f"if command -v apk >/dev/null 2>&1 && [ -r {APK_INSTALLED} ]; then echo '[apk-audit]'; "
|
||||
"awk '/^P:/ {p = substr($0, 3)} /^f:/ {f = substr($0, 3)} "
|
||||
'/^$/ {if (f ~ /[fs]/) print f, p; p = ""; f = ""} END {if (f ~ /[fs]/) print f, p}\' '
|
||||
f"{APK_INSTALLED} | head -n 50; fi; "
|
||||
)
|
||||
_OPKG_PART = (
|
||||
f"if command -v opkg >/dev/null 2>&1 && [ -r {OPKG_STATUS} ]; then echo '[opkg-audit]'; "
|
||||
"awk '/^Package:/ {p = $2} /^Status:/ && "
|
||||
"($4 ~ /^(unpacked|half-configured|half-installed)$/ || $3 ~ /reinstreq/) "
|
||||
f"{{print $3, $4, p}}' {OPKG_STATUS} | head -n 50; fi; "
|
||||
)
|
||||
|
||||
#: The part of the host status command that reads them; POSIX ``sh``, BusyBox too.
|
||||
PACKAGE_MANAGERS_COMMAND = _RUNNING_PART + _RPM_PART + _APK_PART + _OPKG_PART
|
||||
|
||||
|
||||
class PackageManagerReading(NamedTuple):
|
||||
"""One package manager's state: halfway or not, None while it is running."""
|
||||
|
||||
manager: str | None
|
||||
interrupted: bool | None
|
||||
packages: list[InterruptedPackageDict]
|
||||
|
||||
|
||||
def _unfinished_transaction(sections: dict[str, list[str]]) -> bool:
|
||||
newest = (sections.get("dnf-history") or [""])[0].split()
|
||||
if len(newest) == 2 and newest[1] == _UNFINISHED:
|
||||
return True
|
||||
return any(
|
||||
name.startswith(_YUM_JOURNAL) and not name.endswith(_YUM_GIVEN_UP)
|
||||
for name in sections.get("yum-journal") or []
|
||||
)
|
||||
|
||||
|
||||
def _rpm(sections: dict[str, list[str]], running: set[str]) -> PackageManagerReading | None:
|
||||
if "rpm-dupes" not in sections:
|
||||
return None
|
||||
manager = (sections.get("rpm-frontend") or ["dnf"])[0]
|
||||
if running & _RUNNING["rpm"]:
|
||||
return PackageManagerReading(manager, None, [])
|
||||
installonly = set(sections.get("rpm-installonly") or [])
|
||||
names = dict.fromkeys(line.split()[0] for line in sections["rpm-dupes"])
|
||||
packages: list[InterruptedPackageDict] = [
|
||||
{"name": name, "state": "duplicate"} for name in names if name not in installonly
|
||||
]
|
||||
return PackageManagerReading(
|
||||
manager, bool(packages) or _unfinished_transaction(sections), packages
|
||||
)
|
||||
|
||||
|
||||
def _apk(sections: dict[str, list[str]], running: set[str]) -> PackageManagerReading | None:
|
||||
if "apk-audit" not in sections:
|
||||
return None
|
||||
if running & _RUNNING["apk"]:
|
||||
return PackageManagerReading("apk", None, [])
|
||||
packages: list[InterruptedPackageDict] = []
|
||||
for line in sections["apk-audit"]:
|
||||
flags, _, name = line.partition(" ")
|
||||
name = name.strip()
|
||||
packages += [
|
||||
{"name": name, "state": state} for flag, state in _APK_FLAGS if name and flag in flags
|
||||
]
|
||||
return PackageManagerReading("apk", bool(packages), packages)
|
||||
|
||||
|
||||
def _opkg_package(line: str) -> InterruptedPackageDict | None:
|
||||
parts = line.split()
|
||||
if len(parts) != 3:
|
||||
return None
|
||||
flags, state, name = parts
|
||||
if _OPKG_REINSTALL in flags.split(","):
|
||||
return {"name": name, "state": "reinstall-required"}
|
||||
return {"name": name, "state": state} if state in _OPKG_HALFWAY else None
|
||||
|
||||
|
||||
def _opkg(sections: dict[str, list[str]], running: set[str]) -> PackageManagerReading | None:
|
||||
if "opkg-audit" not in sections:
|
||||
return None
|
||||
if running & _RUNNING["opkg"]:
|
||||
return PackageManagerReading("opkg", None, [])
|
||||
packages = [p for p in map(_opkg_package, sections["opkg-audit"]) if p is not None]
|
||||
return PackageManagerReading("opkg", bool(packages), packages)
|
||||
|
||||
|
||||
def parse_package_managers(sections: dict[str, list[str]]) -> list[PackageManagerReading]:
|
||||
"""What :data:`PACKAGE_MANAGERS_COMMAND` found, one reading per manager it asked."""
|
||||
running = set(sections.get("pm-running") or [])
|
||||
readings = (read(sections, running) for read in (_rpm, _apk, _opkg))
|
||||
return [reading for reading in readings if reading is not None]
|
||||
@@ -0,0 +1,176 @@
|
||||
"""A time zone as a device takes it: an IANA name, a POSIX TZ rule, or the rule's parts.
|
||||
|
||||
An application names a zone the IANA way (``Europe/Berlin``). Not every device
|
||||
can: OpenWrt hands ``system.timezone`` to libc as ``TZ``, which wants the rule
|
||||
itself (``CET-1CEST,M3.5.0,M10.5.0/3``), and a switch wants a UTC offset and the
|
||||
summer-time rule. A driver translates with these helpers.
|
||||
|
||||
The rule is the footer of the zone's TZif file (version 2 and later): the line
|
||||
after the last transition that says how the zone goes on. OpenWrt's LuCI table is
|
||||
built from the same footers. The files come from the ``tzdata`` package, a
|
||||
declared dependency, so every driver reads the same database whatever its host
|
||||
ships.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from functools import lru_cache
|
||||
from importlib.resources import files
|
||||
from typing import TypedDict
|
||||
|
||||
#: An IANA name: letters, digits, ``_+-`` in ``/``-separated parts. No dots, so
|
||||
#: nothing reaches outside the database.
|
||||
_ZONE_NAME = re.compile(r"[A-Za-z0-9_+-]+(?:/[A-Za-z0-9_+-]+)*")
|
||||
|
||||
_ABBR = r"(?:[A-Za-z]{3,}|<[A-Za-z0-9+-]{3,}>)"
|
||||
_OFFSET = r"[+-]?\d{1,2}(?::\d{1,2}){0,2}"
|
||||
_DATE = r"(?:J\d{1,3}|\d{1,3}|M\d{1,2}\.\d\.\d)(?:/[+-]?\d{1,3}(?::\d{1,2}){0,2})?"
|
||||
#: ``std offset [dst [offset] [,start,end]]`` (POSIX.1, 8.3), and a bare ``UTC`` or
|
||||
#: ``GMT``, which OpenWrt itself writes by default.
|
||||
_POSIX_TZ = re.compile(
|
||||
rf"(?:UTC|GMT)|{_ABBR}{_OFFSET}(?:{_ABBR}(?:{_OFFSET})?(?:,{_DATE},{_DATE})?)?"
|
||||
)
|
||||
|
||||
_PARTS = re.compile(
|
||||
rf"(?P<std>{_ABBR})(?P<offset>{_OFFSET})"
|
||||
rf"(?:(?P<dst>{_ABBR})(?P<dst_offset>{_OFFSET})?"
|
||||
r"(?:,(?P<start>[^,]+),(?P<end>[^,]+))?)?"
|
||||
)
|
||||
_TRANSITION = re.compile(r"M(\d{1,2})\.([1-5])\.([0-6])(?:/([+-]?\d{1,3}(?::\d{1,2}){0,2}))?")
|
||||
#: When a rule names no time of day for a transition: 02:00 local time.
|
||||
_DEFAULT_TRANSITION_TIME = 7200
|
||||
|
||||
|
||||
class DstTransitionDict(TypedDict):
|
||||
"""When summer time starts or ends: a weekday of a week of a month, and a time."""
|
||||
|
||||
month: int # 1-12
|
||||
week: int # 1-5; 5 is the month's last such weekday
|
||||
weekday: int # 0 = Sunday ... 6 = Saturday
|
||||
time: int # seconds after local midnight, in the time then in force; may be < 0 or > 24h
|
||||
|
||||
|
||||
class PosixTzDict(TypedDict):
|
||||
"""A POSIX TZ rule in parts (:func:`parse_posix_tz`)."""
|
||||
|
||||
std_name: str # "CET"; a quoted name without its <> ("-03")
|
||||
utc_offset: int # seconds east of UTC in standard time: CET 3600, EST -18000
|
||||
dst_name: str | None # "CEST"; None for a zone without summer time
|
||||
dst_offset: int | None # seconds east of UTC in summer time; None without it
|
||||
dst_start: DstTransitionDict | None # None without summer time or without a rule for it
|
||||
dst_end: DstTransitionDict | None
|
||||
|
||||
|
||||
class UnknownTimezone(ValueError):
|
||||
"""A zone that is not in the zone database (nor, where one is taken, a POSIX TZ rule)."""
|
||||
|
||||
def __init__(self, zone: str) -> None:
|
||||
super().__init__(zone)
|
||||
self.zone = zone
|
||||
|
||||
def __str__(self) -> str:
|
||||
return (
|
||||
f"unknown time zone {self.zone!r}: not a name of the IANA time zone database"
|
||||
" such as Europe/Berlin"
|
||||
)
|
||||
|
||||
|
||||
def is_posix_tz(value: str) -> bool:
|
||||
"""Whether *value* reads as a POSIX TZ string -- the form libc takes in ``TZ``."""
|
||||
return bool(_POSIX_TZ.fullmatch(value))
|
||||
|
||||
|
||||
@lru_cache(maxsize=512)
|
||||
def _footer(zone: str) -> str | None:
|
||||
"""The POSIX rule at the end of *zone*'s TZif file; None for no such zone."""
|
||||
if not _ZONE_NAME.fullmatch(zone):
|
||||
return None
|
||||
tzfile = files("tzdata").joinpath("zoneinfo", *zone.split("/"))
|
||||
if not tzfile.is_file():
|
||||
return None
|
||||
data = tzfile.read_bytes()
|
||||
if not data.startswith(b"TZif") or data[4:5] not in (b"2", b"3", b"4") or data[-1:] != b"\n":
|
||||
return None
|
||||
footer = data[:-1].rsplit(b"\n", 1)[-1].decode("ascii", "replace")
|
||||
return footer if is_posix_tz(footer) else None
|
||||
|
||||
|
||||
def is_iana_zone(zone: str) -> bool:
|
||||
"""Whether *zone* names a zone of the database (``Europe/Berlin``, ``UTC``)."""
|
||||
return _footer(zone) is not None
|
||||
|
||||
|
||||
def posix_tz(zone: str) -> str | None:
|
||||
"""The POSIX TZ string for *zone*: an IANA name's rule, a POSIX string as it is.
|
||||
|
||||
None when *zone* is neither -- a device given it would run on UTC.
|
||||
"""
|
||||
return _footer(zone) or (zone if is_posix_tz(zone) else None)
|
||||
|
||||
|
||||
def _seconds(value: str) -> int:
|
||||
"""``[+-]hh[:mm[:ss]]`` in seconds."""
|
||||
sign = -1 if value.startswith("-") else 1
|
||||
parts = [int(p) for p in value.lstrip("+-").split(":")]
|
||||
hours, minutes, seconds = (parts + [0, 0])[:3]
|
||||
return sign * (hours * 3600 + minutes * 60 + seconds)
|
||||
|
||||
|
||||
def _transition(value: str) -> DstTransitionDict | None:
|
||||
match = _TRANSITION.fullmatch(value)
|
||||
if match is None:
|
||||
return None
|
||||
month, week, weekday, time = match.groups()
|
||||
return {
|
||||
"month": int(month),
|
||||
"week": int(week),
|
||||
"weekday": int(weekday),
|
||||
"time": _DEFAULT_TRANSITION_TIME if time is None else _seconds(time),
|
||||
}
|
||||
|
||||
|
||||
def parse_posix_tz(rule: str) -> PosixTzDict | None:
|
||||
"""*rule* in parts; None for what is not a rule, or one with Julian-day transitions.
|
||||
|
||||
POSIX counts offsets west of UTC (``CET-1``); the parts count them east, as a
|
||||
device's "UTC+01:00" does. Summer time without an offset of its own is one hour
|
||||
ahead of standard time.
|
||||
"""
|
||||
if rule in ("UTC", "GMT"):
|
||||
rule += "0"
|
||||
match = _PARTS.fullmatch(rule)
|
||||
if match is None or not is_posix_tz(rule):
|
||||
return None
|
||||
utc_offset = -_seconds(match["offset"])
|
||||
parts: PosixTzDict = {
|
||||
"std_name": match["std"].strip("<>"),
|
||||
"utc_offset": utc_offset,
|
||||
"dst_name": None,
|
||||
"dst_offset": None,
|
||||
"dst_start": None,
|
||||
"dst_end": None,
|
||||
}
|
||||
if match["dst"] is None:
|
||||
return parts
|
||||
parts["dst_name"] = match["dst"].strip("<>")
|
||||
parts["dst_offset"] = (
|
||||
utc_offset + 3600 if match["dst_offset"] is None else -_seconds(match["dst_offset"])
|
||||
)
|
||||
if match["start"] is not None:
|
||||
start, end = _transition(match["start"]), _transition(match["end"])
|
||||
if start is None or end is None:
|
||||
return None
|
||||
parts["dst_start"], parts["dst_end"] = start, end
|
||||
return parts
|
||||
|
||||
|
||||
__all__ = [
|
||||
"DstTransitionDict",
|
||||
"PosixTzDict",
|
||||
"UnknownTimezone",
|
||||
"is_iana_zone",
|
||||
"is_posix_tz",
|
||||
"parse_posix_tz",
|
||||
"posix_tz",
|
||||
]
|
||||
@@ -0,0 +1,69 @@
|
||||
"""The backups running on a device right now.
|
||||
|
||||
A restart aborts every backup that runs on the device it restarts: a
|
||||
hypervisor's backup of its guests, a backup server's backup, sync, verify or
|
||||
garbage collection, a host's own file-level backup (restic, borg and the like).
|
||||
A guest's restart breaks the backup its hypervisor is taking of it. So an
|
||||
application that restarts a device asks first, right before the restart.
|
||||
|
||||
Declared under ``if TYPE_CHECKING``: a driver mixes this in when it can tell;
|
||||
``hasattr`` tells a caller whether it did. No role base carries it, and a driver
|
||||
without it cannot tell -- which is not the same as "nothing runs".
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING, Literal, TypedDict, get_args
|
||||
|
||||
#: What a running backup does: the backup itself, or one of a backup server's
|
||||
#: jobs (``napalm_device_types.backups.BackupJobKind``).
|
||||
RunningBackupKind = Literal["backup", "sync", "verify", "gc", "prune", "tape"]
|
||||
RUNNING_BACKUP_KINDS: tuple[str, ...] = get_args(RunningBackupKind)
|
||||
|
||||
|
||||
class RunningBackupDict(TypedDict):
|
||||
"""One backup running on the device (``get_running_backups``)."""
|
||||
|
||||
kind: RunningBackupKind
|
||||
job: str | None # the job it runs for, as the platform names it; None: started by hand
|
||||
task: str | None # this run's id on the platform: a task id, a unit, a process id
|
||||
guests: list[str] # the guests (get_vms' ids) it is backing up now; [] for the host's own
|
||||
since: int | None # unix seconds it started; None when the platform does not say
|
||||
|
||||
|
||||
class RunningBackupsMixin:
|
||||
if TYPE_CHECKING:
|
||||
|
||||
def get_running_backups(self) -> list[RunningBackupDict]:
|
||||
"""
|
||||
Returns the backups running on the device now; ``[]`` when none is.
|
||||
|
||||
Every entry runs on this device: restarting the device aborts it.
|
||||
A hypervisor names in ``guests`` the guests a backup is taking right
|
||||
now -- a guest that is locked for its backup counts, with or without
|
||||
a task the platform still shows -- so the application can tell a
|
||||
guest's own restart not to break it.
|
||||
|
||||
Example::
|
||||
|
||||
[
|
||||
{
|
||||
"kind": "backup",
|
||||
"job": "backup-3f2a9c1e-77d1",
|
||||
"task": "UPID:pve1:0001F2A3:00C4B1D2:6731A0C0:vzdump::root@pam:",
|
||||
"guests": ["101"],
|
||||
"since": 1793660400,
|
||||
},
|
||||
{
|
||||
"kind": "gc",
|
||||
"job": "store1",
|
||||
"task": "UPID:localhost:000004D2:...:garbage_collection:store1:root@pam:",
|
||||
"guests": [],
|
||||
"since": 1793664000,
|
||||
},
|
||||
]
|
||||
|
||||
:raises Exception: when the device could not be read; never an
|
||||
empty list for a failed read.
|
||||
"""
|
||||
...
|
||||
@@ -0,0 +1,196 @@
|
||||
"""Scheduled jobs: what a host runs on its own schedule, and switching it on, off and now.
|
||||
|
||||
A host runs work on its own: cron lines, run-parts scripts, anacron, systemd
|
||||
timers, FreeBSD's ``periodic``, a firewall's or a NAS's own scheduled jobs, the
|
||||
Windows Task Scheduler. :class:`ScheduledJobsMixin` reads all of them as one
|
||||
shape, :class:`~napalm_device_types.models.ScheduledJobDict`, and acts on them.
|
||||
Declared under ``if TYPE_CHECKING`` like a role: nothing exists at runtime until
|
||||
a driver implements it. :class:`~napalm_device_types.scheduled_jobs_posix.PosixScheduledJobsMixin`
|
||||
is the implementation for a host reached through the command channel.
|
||||
|
||||
**Who owns a definition** is on every job (``managed_by``): ``package`` (installed
|
||||
by one), ``admin`` (written on the host), ``platform`` (generated by the device's
|
||||
own configuration -- OPNsense, OpenMediaVault, QTS, Windows), ``application`` (written
|
||||
through :meth:`put_scheduled_job`, under the ``name`` the application gave it).
|
||||
What the application makes of that is the application's.
|
||||
|
||||
**What may be done is on every job too** (``actions``): every job a driver can
|
||||
switch takes ``enable``, ``disable`` and ``run``; only an ``application`` job takes
|
||||
``edit`` and ``delete``. A driver never edits or removes a job it did not write.
|
||||
|
||||
**Never over a change it did not see.** Every write names the ``digest`` the caller
|
||||
saw. The driver reads the job again right before it writes and raises
|
||||
:class:`ScheduledJobChanged` when the definition differs.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import re
|
||||
from typing import TYPE_CHECKING, Any, NotRequired, TypedDict
|
||||
|
||||
from napalm_device_types.cron import is_reboot, parse_cron
|
||||
|
||||
#: What :meth:`ScheduledJobsMixin.manage_scheduled_job` takes.
|
||||
ACTIONS = ("enable", "disable", "run")
|
||||
|
||||
#: What a job's ``actions`` may list beyond :data:`ACTIONS`.
|
||||
EDIT = "edit"
|
||||
DELETE = "delete"
|
||||
|
||||
KINDS = ("cron", "run_parts", "anacron", "systemd_timer", "periodic", "task")
|
||||
MANAGED_BY = ("package", "admin", "platform", "application")
|
||||
|
||||
#: Seconds a run now may take on the host before ``timeout`` stops it.
|
||||
RUN_TIMEOUT = 300
|
||||
|
||||
#: A job's name becomes a file and a unit name: lower case, digits, dashes.
|
||||
_NAME = re.compile(r"[a-z0-9][a-z0-9-]{0,62}")
|
||||
#: A user name as useradd and the BSDs take it.
|
||||
_USER = re.compile(r"[A-Za-z_][A-Za-z0-9_.-]{0,31}\$?")
|
||||
|
||||
#: The parts of a job that are its definition; what the host says of its runs is not.
|
||||
_DEFINITION = ("kind", "source", "run_as", "schedule", "command", "enabled")
|
||||
|
||||
|
||||
class ScheduledJobDict(TypedDict):
|
||||
"""A job the host runs on its own schedule (``get_scheduled_jobs``).
|
||||
|
||||
A cron line, a run-parts script, an anacron job, a systemd timer, a FreeBSD
|
||||
``periodic`` script, a platform's own job, a Windows task -- one shape for all
|
||||
of them (``napalm_device_types.scheduled_jobs``).
|
||||
"""
|
||||
|
||||
key: str # stable name on this host: "timer:apt-daily.timer", "cron:/etc/crontab:1a2b3c4d"
|
||||
kind: str # cron | run_parts | anacron | systemd_timer | periodic | task
|
||||
source: str # the file, unit file or configuration it is defined in
|
||||
run_as: str # the user it runs as
|
||||
schedule: str # as the host has it: "*/5 * * * *", "OnCalendar=daily", "daily"
|
||||
schedule_kind: str # cron | calendar | monotonic | period | triggers
|
||||
command: str # as the host has it
|
||||
enabled: bool
|
||||
managed_by: str # package | admin | platform | application
|
||||
digest: str # job_digest(): the definition as it is now
|
||||
actions: list[str] # what the driver can do to this job: enable, disable, run, edit, delete
|
||||
name: NotRequired[str] # the name an application gave a job it wrote (put_scheduled_job)
|
||||
description: NotRequired[str]
|
||||
last_run: NotRequired[str | None] # ISO 8601, UTC; where the host knows it
|
||||
last_result: NotRequired[str | None] # "success", or what the host says went wrong
|
||||
next_run: NotRequired[str | None] # ISO 8601, UTC; where the host knows it
|
||||
|
||||
|
||||
class ScheduledJobsReadingDict(TypedDict):
|
||||
"""Every scheduled job of a host, and the time zone its schedules are in."""
|
||||
|
||||
jobs: list[ScheduledJobDict]
|
||||
time_zone: str | None # IANA name ("Europe/Berlin"); None when the host does not say
|
||||
|
||||
|
||||
class ScheduledJobSpecDict(TypedDict):
|
||||
"""A job an application wants on the host (``put_scheduled_job``)."""
|
||||
|
||||
name: str # the job's name on the host, chosen by the application: [a-z0-9][a-z0-9-]*
|
||||
schedule: str # a cron expression (napalm_device_types.cron), "@reboot" included
|
||||
run_as: str
|
||||
command: str # one or more lines of POSIX sh; a platform may take its own (OPNsense: configd)
|
||||
enabled: bool
|
||||
description: NotRequired[str]
|
||||
|
||||
|
||||
class ScheduledJobResultDict(TypedDict):
|
||||
"""What an action on a scheduled job did."""
|
||||
|
||||
success: bool
|
||||
output: str # what the host said; for a run, what the job printed
|
||||
|
||||
|
||||
class ScheduledJobChanged(Exception): # noqa: N818 - a state, as the callers word it
|
||||
"""The job's definition differs from the digest the caller saw: nothing was written."""
|
||||
|
||||
|
||||
class ScheduledJobRefused(ValueError):
|
||||
"""The job cannot take what was asked of it, or the request itself is not valid."""
|
||||
|
||||
|
||||
def job_digest(job: dict[str, Any], *, content: str = "") -> str:
|
||||
"""The definition of *job* in 16 hex characters.
|
||||
|
||||
Over kind, source, run_as, schedule, command and enabled -- so enabling a job
|
||||
changes it -- and *content*: what the definition points at that the job's
|
||||
fields do not show, such as a run-parts script's checksum.
|
||||
"""
|
||||
parts = [job.get(field) for field in _DEFINITION] + [content]
|
||||
return hashlib.sha256(json.dumps(parts).encode()).hexdigest()[:16]
|
||||
|
||||
|
||||
def check_job_name(name: str) -> None:
|
||||
""":raises ScheduledJobRefused: for a name a file or a unit could not carry safely."""
|
||||
if not _NAME.fullmatch(name or ""):
|
||||
raise ScheduledJobRefused(f"not a job name: {name!r} (a-z, 0-9 and -, at most 63)")
|
||||
|
||||
|
||||
def check_spec(spec: ScheduledJobSpecDict) -> None:
|
||||
""":raises ScheduledJobRefused: for a spec no host could be given as it is."""
|
||||
check_job_name(spec["name"])
|
||||
if not _USER.fullmatch(spec.get("run_as") or ""):
|
||||
raise ScheduledJobRefused(f"not a user name: {spec.get('run_as')!r}")
|
||||
if not (spec.get("command") or "").strip():
|
||||
raise ScheduledJobRefused("a job needs a command")
|
||||
if "\n" in (spec.get("description") or ""):
|
||||
raise ScheduledJobRefused("a description is one line")
|
||||
if not is_reboot(spec["schedule"]):
|
||||
try:
|
||||
parse_cron(spec["schedule"])
|
||||
except ValueError as exc:
|
||||
raise ScheduledJobRefused(str(exc)) from exc
|
||||
|
||||
|
||||
class ScheduledJobsMixin:
|
||||
"""Declares the scheduled-jobs contract; a driver that can implements it.
|
||||
|
||||
A verb a driver cannot do at all is ``None`` on the driver, as ``manage_service``
|
||||
is; a verb one job cannot take is missing from that job's ``actions``.
|
||||
"""
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def get_scheduled_jobs(self) -> ScheduledJobsReadingDict:
|
||||
"""Every scheduled job of the host.
|
||||
|
||||
:raises ValueError: when the reading came back cut short -- never a
|
||||
shorter list.
|
||||
"""
|
||||
...
|
||||
|
||||
def manage_scheduled_job(
|
||||
self, key: str, action: str, *, digest: str
|
||||
) -> ScheduledJobResultDict:
|
||||
"""Enable, disable or run the job *key* now.
|
||||
|
||||
:raises ScheduledJobChanged: when the job is no longer as *digest* says.
|
||||
:raises ScheduledJobRefused: when the job does not take *action*.
|
||||
:raises LookupError: when the host has no job *key*.
|
||||
"""
|
||||
...
|
||||
|
||||
def put_scheduled_job(
|
||||
self, spec: ScheduledJobSpecDict, *, digest: str | None = None
|
||||
) -> ScheduledJobResultDict:
|
||||
"""Create the job ``spec["name"]`` (*digest* None) or change it.
|
||||
|
||||
:raises ScheduledJobChanged: when the job is no longer as *digest*
|
||||
says, or when a job of that name is there although *digest* is None.
|
||||
:raises ScheduledJobRefused: for a spec :func:`check_spec` refuses, or
|
||||
one the host cannot take.
|
||||
"""
|
||||
...
|
||||
|
||||
def delete_scheduled_job(self, key: str, *, digest: str) -> ScheduledJobResultDict:
|
||||
"""Delete the job *key*, which an application wrote.
|
||||
|
||||
:raises ScheduledJobChanged: when the job is no longer as *digest* says.
|
||||
:raises ScheduledJobRefused: when the job is not one an application wrote.
|
||||
:raises LookupError: when the host has no job *key*.
|
||||
"""
|
||||
...
|
||||
@@ -0,0 +1,493 @@
|
||||
"""Scheduled jobs on a POSIX host, over the command channel: cron and systemd timers.
|
||||
|
||||
:class:`PosixScheduledJobsMixin` implements
|
||||
:class:`~napalm_device_types.scheduled_jobs.ScheduledJobsMixin` for a driver that has
|
||||
:meth:`~napalm_device_types.channel.CommandChannelMixin.run_command`. The driver
|
||||
mixes it in and sets the paths its host keeps jobs in; the defaults are Debian's.
|
||||
|
||||
**Reading** is one framed report as root (:mod:`napalm_device_types._scheduled_reading`).
|
||||
|
||||
**Switching** a cron or anacron line comments it out behind ``#napalm-disabled: ``
|
||||
and takes exactly that marker away again; a run-parts script loses or gets its
|
||||
execute bit; a timer is ``systemctl disable --now`` / ``enable --now``. A timer that
|
||||
cannot be switched (``static``) only runs.
|
||||
|
||||
**Writing** stages the new text as the login user and installs it as root with one
|
||||
fixed command: ``crontab -u <user> <file>`` for a user's crontab, ``install`` with
|
||||
the file's own mode and owner for a system file. Root never reads stdin. Before any
|
||||
write the job is read again and compared with the digest the caller saw.
|
||||
|
||||
**An application's own job** (:meth:`put_scheduled_job`) is a timer on a host with
|
||||
systemd -- ``<name>.timer`` and ``<name>.service`` in :attr:`UNIT_DIR`, the command
|
||||
in :attr:`JOB_SCRIPT_DIR` ``/<name>`` behind a marker and its cron expression --
|
||||
else a file in :attr:`OWN_CRON_D_DIR`, else marked lines in the user's crontab.
|
||||
|
||||
**Running now** is bounded by :data:`~napalm_device_types.scheduled_jobs.RUN_TIMEOUT`:
|
||||
a timer's unit by ``systemctl start`` and its journal since then, a cron line's
|
||||
command as its user the way cron runs it (``_crontab.run_script``).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import fnmatch
|
||||
import shlex
|
||||
from collections.abc import Sequence
|
||||
from typing import TYPE_CHECKING, ClassVar
|
||||
|
||||
from napalm_device_types._crontab import (
|
||||
JOB_MARKER,
|
||||
CronLine,
|
||||
marked_lines,
|
||||
parse_anacrontab,
|
||||
parse_crontab,
|
||||
run_script,
|
||||
switched,
|
||||
with_job,
|
||||
without_job,
|
||||
)
|
||||
from napalm_device_types._scheduled_reading import (
|
||||
OWNERS_SH,
|
||||
SCHEDULE_HEADER,
|
||||
TIME_ZONE_SH,
|
||||
Paths,
|
||||
Reading,
|
||||
parse_reading,
|
||||
reading_command,
|
||||
)
|
||||
from napalm_device_types.cron import is_reboot, to_on_calendar
|
||||
from napalm_device_types.scheduled_jobs import (
|
||||
ACTIONS,
|
||||
RUN_TIMEOUT,
|
||||
ScheduledJobChanged,
|
||||
ScheduledJobRefused,
|
||||
ScheduledJobResultDict,
|
||||
ScheduledJobsMixin,
|
||||
ScheduledJobSpecDict,
|
||||
ScheduledJobsReadingDict,
|
||||
check_spec,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from napalm_device_types.channel import CommandResult
|
||||
|
||||
_STAGE = 't=$(mktemp) && chmod 644 "$t" && cat > "$t" && printf %s "$t"'
|
||||
#: What a run keeps of the job's output.
|
||||
_OUTPUT_LIMIT = 64 * 1024
|
||||
_READ_TIMEOUT = 120
|
||||
|
||||
|
||||
def _command(argv: Sequence[str]) -> str:
|
||||
return shlex.join(argv)
|
||||
|
||||
|
||||
def _said(result: CommandResult) -> str:
|
||||
return "\n".join(p for p in (result.stdout.strip(), result.stderr.strip()) if p)[
|
||||
-_OUTPUT_LIMIT:
|
||||
]
|
||||
|
||||
|
||||
def _result(result: CommandResult, done: str) -> ScheduledJobResultDict:
|
||||
if result.exit_code != 0:
|
||||
return {"success": False, "output": _said(result) or f"exit {result.exit_code}"}
|
||||
return {"success": True, "output": _said(result) or done}
|
||||
|
||||
|
||||
class PosixScheduledJobsMixin(ScheduledJobsMixin):
|
||||
"""Scheduled jobs of a host reached through ``run_command``; the paths are the driver's."""
|
||||
|
||||
CRON_SYSTEM_FILES: ClassVar[tuple[str, ...]] = ("/etc/crontab",)
|
||||
CRON_D_DIRS: ClassVar[tuple[str, ...]] = ("/etc/cron.d",)
|
||||
CRON_SPOOL_DIRS: ClassVar[tuple[str, ...]] = (
|
||||
"/var/spool/cron/crontabs",
|
||||
"/var/spool/cron",
|
||||
"/etc/crontabs",
|
||||
)
|
||||
RUN_PARTS_DIRS: ClassVar[tuple[tuple[str, str], ...]] = (
|
||||
("hourly", "/etc/cron.hourly"),
|
||||
("daily", "/etc/cron.daily"),
|
||||
("weekly", "/etc/cron.weekly"),
|
||||
("monthly", "/etc/cron.monthly"),
|
||||
)
|
||||
ANACRONTAB: ClassVar[str] = "/etc/anacrontab"
|
||||
JOB_SCRIPT_DIR: ClassVar[str] = "/usr/local/libexec/scheduled-jobs"
|
||||
UNIT_DIR: ClassVar[str] = "/etc/systemd/system"
|
||||
#: Where a host without systemd takes an application's job as a file, if the directory
|
||||
#: is there; else, or None, the user's crontab.
|
||||
OWN_CRON_D_DIR: ClassVar[str | None] = "/etc/cron.d"
|
||||
ROOT_GROUP: ClassVar[str] = "root"
|
||||
#: Prints "<octal mode> <owner> <group>" of the path appended.
|
||||
STAT_ARGV: ClassVar[tuple[str, ...]] = ("stat", "-c", "%a %U %G", "--")
|
||||
#: ``install`` and how it makes missing directories (FreeBSD's ``-D`` is a destdir).
|
||||
INSTALL: ClassVar[tuple[str, ...]] = ("install", "-D")
|
||||
#: ``sh`` that reads definition paths on stdin and prints ``<package>: <path>``.
|
||||
OWNERS_SH: ClassVar[str] = OWNERS_SH
|
||||
#: ``sh`` that prints the time zone: an IANA name, or a path ending in ``zoneinfo/<name>``.
|
||||
TIME_ZONE_SH: ClassVar[str] = TIME_ZONE_SH
|
||||
#: ``fnmatch`` patterns on a job's ``source``: what the platform generates is ``platform``.
|
||||
PLATFORM_SOURCES: ClassVar[tuple[str, ...]] = ()
|
||||
#: Run as root after a user's crontab was written; for a host that keeps it elsewhere too.
|
||||
CRONTAB_AFTER_WRITE: ClassVar[tuple[str, ...] | None] = None
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def run_command(
|
||||
self,
|
||||
command: str,
|
||||
*,
|
||||
privileged: bool = False,
|
||||
timeout: float = 60,
|
||||
stdin: bytes | None = None,
|
||||
) -> CommandResult: ...
|
||||
|
||||
# -- reading ---------------------------------------------------------------------------------
|
||||
|
||||
def _sj_paths(self) -> Paths:
|
||||
return Paths(
|
||||
system_files=self.CRON_SYSTEM_FILES,
|
||||
cron_d_dirs=self.CRON_D_DIRS,
|
||||
spool_dirs=self.CRON_SPOOL_DIRS,
|
||||
run_parts=self.RUN_PARTS_DIRS,
|
||||
anacrontab=self.ANACRONTAB,
|
||||
script_dir=self.JOB_SCRIPT_DIR,
|
||||
owners_sh=self.OWNERS_SH,
|
||||
time_zone_sh=self.TIME_ZONE_SH,
|
||||
)
|
||||
|
||||
def _sj_read(self) -> Reading:
|
||||
result = self.run_command(
|
||||
reading_command(self._sj_paths()), privileged=True, timeout=_READ_TIMEOUT
|
||||
)
|
||||
reading = parse_reading(result.stdout)
|
||||
for job in reading.jobs:
|
||||
source = str(job["source"])
|
||||
if job["managed_by"] != "application" and any(
|
||||
fnmatch.fnmatchcase(source, p) for p in self.PLATFORM_SOURCES
|
||||
):
|
||||
job["managed_by"] = "platform"
|
||||
return reading
|
||||
|
||||
def get_scheduled_jobs(self) -> ScheduledJobsReadingDict:
|
||||
"""Every scheduled job of the host, in one round trip.
|
||||
|
||||
:raises ValueError: when the report came back cut short.
|
||||
"""
|
||||
reading = self._sj_read()
|
||||
return {"jobs": reading.jobs, "time_zone": reading.time_zone} # type: ignore[typeddict-item]
|
||||
|
||||
def _sj_job(self, reading: Reading, key: str, digest: str) -> dict[str, object]:
|
||||
job = next((j for j in reading.jobs if j["key"] == key), None)
|
||||
if job is None:
|
||||
raise LookupError(f"no scheduled job {key}")
|
||||
if job["digest"] != digest:
|
||||
raise ScheduledJobChanged(f"{key} changed on the host since it was read")
|
||||
return job
|
||||
|
||||
# -- commands --------------------------------------------------------------------------------
|
||||
|
||||
def _sj_root(self, argv: Sequence[str], timeout: float = 60) -> CommandResult:
|
||||
return self.run_command(_command(argv), privileged=True, timeout=timeout)
|
||||
|
||||
def _sj_checked(self, argv: Sequence[str]) -> CommandResult:
|
||||
result = self._sj_root(argv)
|
||||
if result.exit_code != 0:
|
||||
raise RuntimeError(f"{_command(argv)}: {_said(result) or result.exit_code}")
|
||||
return result
|
||||
|
||||
def _sj_stage(self, content: str) -> str:
|
||||
result = self.run_command(_STAGE, timeout=30, stdin=content.encode())
|
||||
path = result.stdout.strip().splitlines()[-1:] or [""]
|
||||
if result.exit_code != 0 or not path[0].startswith("/"):
|
||||
raise RuntimeError(f"cannot stage a file: {_said(result)}")
|
||||
return path[0]
|
||||
|
||||
def _sj_install(self, content: str, path: str, mode: str, owner: str, group: str) -> None:
|
||||
staged = self._sj_stage(content)
|
||||
try:
|
||||
self._sj_checked(
|
||||
[*self.INSTALL, "-m", mode, "-o", owner, "-g", group, "--", staged, path]
|
||||
)
|
||||
finally:
|
||||
self.run_command(_command(["rm", "-f", "--", staged]), timeout=30)
|
||||
|
||||
# -- crontab files ---------------------------------------------------------------------------
|
||||
|
||||
def _sj_spool_user(self, path: str) -> str | None:
|
||||
"""The user whose crontab *path* is; None for a system file."""
|
||||
directory, _, user = path.rpartition("/")
|
||||
return user if directory in self.CRON_SPOOL_DIRS else None
|
||||
|
||||
def _sj_crontab(self, user: str) -> str:
|
||||
result = self._sj_root(["crontab", "-u", user, "-l"])
|
||||
if result.exit_code != 0:
|
||||
if "no crontab" in (result.stderr + result.stdout).lower():
|
||||
return ""
|
||||
raise RuntimeError(f"cannot read the crontab of {user}: {_said(result)}")
|
||||
return result.stdout
|
||||
|
||||
def _sj_text(self, path: str) -> str:
|
||||
user = self._sj_spool_user(path)
|
||||
if user is not None:
|
||||
return self._sj_crontab(user)
|
||||
return self._sj_checked(["cat", "--", path]).stdout
|
||||
|
||||
def _sj_write_crontab(self, user: str, text: str) -> None:
|
||||
staged = self._sj_stage(text)
|
||||
try:
|
||||
self._sj_checked(["crontab", "-u", user, staged])
|
||||
finally:
|
||||
self.run_command(_command(["rm", "-f", "--", staged]), timeout=30)
|
||||
if self.CRONTAB_AFTER_WRITE:
|
||||
self._sj_checked(self.CRONTAB_AFTER_WRITE)
|
||||
|
||||
def _sj_write(self, path: str, text: str) -> None:
|
||||
user = self._sj_spool_user(path)
|
||||
if user is not None:
|
||||
self._sj_write_crontab(user, text)
|
||||
return
|
||||
# Through a symbolic link to the file it names, with that file's mode: the
|
||||
# link's own mode is 777, and cron ignores a file anyone may write.
|
||||
real = self._sj_checked(["readlink", "-f", "--", path]).stdout.strip() or path
|
||||
mode, owner, group = self._sj_checked([*self.STAT_ARGV, real]).stdout.split()[:3]
|
||||
self._sj_install(text, real, mode, owner, group)
|
||||
|
||||
def _sj_line(self, job: dict[str, object], text: str) -> CronLine:
|
||||
"""The job's line in *text* as it is now, read again for the write."""
|
||||
path = str(job["source"])
|
||||
if job["kind"] == "anacron":
|
||||
found = parse_anacrontab(text)
|
||||
else:
|
||||
user = self._sj_spool_user(path)
|
||||
found = parse_crontab(path, text, system=user is None, user=user or "")
|
||||
line = next((ln for ln in found if ln.key == job["key"]), None)
|
||||
if line is None:
|
||||
raise ScheduledJobChanged(f"{job['key']} is no longer in {path}")
|
||||
return line
|
||||
|
||||
# -- switching and running -------------------------------------------------------------------
|
||||
|
||||
def _sj_timer_unit(self, job: dict[str, object]) -> str:
|
||||
key = str(job["key"])
|
||||
return f"{job['name']}.timer" if key.startswith("job:") else key.removeprefix("timer:")
|
||||
|
||||
def _sj_switch(self, job: dict[str, object], enabled: bool) -> ScheduledJobResultDict:
|
||||
word = "enabled" if enabled else "disabled"
|
||||
if job["kind"] == "systemd_timer":
|
||||
verb = "enable" if enabled else "disable"
|
||||
return _result(
|
||||
self._sj_root(["systemctl", verb, "--now", "--", self._sj_timer_unit(job)]), word
|
||||
)
|
||||
if job["kind"] == "run_parts":
|
||||
return _result(
|
||||
self._sj_root(["chmod", "+x" if enabled else "-x", "--", str(job["source"])]), word
|
||||
)
|
||||
path = str(job["source"])
|
||||
text = self._sj_text(path)
|
||||
self._sj_write(path, switched(text, self._sj_line(job, text), enabled=enabled))
|
||||
return {"success": True, "output": f"{job['key']} {word}"}
|
||||
|
||||
def _sj_as(self, user: str, script: str) -> list[str]:
|
||||
"""*script* run by ``sh`` as *user*, from root."""
|
||||
if user == "root":
|
||||
return ["sh", "-c", script]
|
||||
return ["su", "-s", "/bin/sh", user, "-c", script]
|
||||
|
||||
def _sj_run(self, job: dict[str, object], reading: Reading) -> ScheduledJobResultDict:
|
||||
bound = ["timeout", str(RUN_TIMEOUT)]
|
||||
wait = RUN_TIMEOUT + 30
|
||||
if job["kind"] == "systemd_timer":
|
||||
unit = reading.units[str(job["key"])]
|
||||
since = self.run_command("date +%s", timeout=30).stdout.strip()
|
||||
started = self._sj_root([*bound, "systemctl", "start", "--", unit], timeout=wait)
|
||||
journal = self._sj_root(
|
||||
[
|
||||
"journalctl",
|
||||
"-u",
|
||||
unit,
|
||||
"--since",
|
||||
f"@{since}",
|
||||
"--no-pager",
|
||||
"-o",
|
||||
"cat",
|
||||
"-n",
|
||||
"200",
|
||||
]
|
||||
)
|
||||
output = "\n".join(p for p in (_said(started), journal.stdout.strip()) if p)
|
||||
return {"success": started.exit_code == 0, "output": output or f"{unit} ran"}
|
||||
if job["kind"] == "run_parts":
|
||||
return _result(self._sj_root([*bound, str(job["source"])], timeout=wait), "ran")
|
||||
path = str(job["source"])
|
||||
line = self._sj_line(job, self._sj_text(path))
|
||||
script = run_script(line.command, line.env)
|
||||
return _result(
|
||||
self._sj_root([*bound, *self._sj_as(line.run_as, script)], timeout=wait), "ran"
|
||||
)
|
||||
|
||||
def manage_scheduled_job(self, key: str, action: str, *, digest: str) -> ScheduledJobResultDict:
|
||||
"""Enable, disable or run the job *key* now; see :class:`ScheduledJobsMixin`."""
|
||||
if action not in ACTIONS:
|
||||
raise ScheduledJobRefused(f"unknown action {action!r}; allowed: {list(ACTIONS)}")
|
||||
reading = self._sj_read()
|
||||
job = self._sj_job(reading, key, digest)
|
||||
if action not in job["actions"]: # type: ignore[operator]
|
||||
raise ScheduledJobRefused(f"{key} cannot be told to {action} on this host")
|
||||
if action == "run":
|
||||
return self._sj_run(job, reading)
|
||||
return self._sj_switch(job, action == "enable")
|
||||
|
||||
# -- an application's own jobs ---------------------------------------------------------------
|
||||
|
||||
def _sj_own(
|
||||
self, reading: Reading, spec: ScheduledJobSpecDict, digest: str | None
|
||||
) -> dict[str, object] | None:
|
||||
current = next((j for j in reading.jobs if j["key"] == f"job:{spec['name']}"), None)
|
||||
if digest is None:
|
||||
if current is not None:
|
||||
raise ScheduledJobChanged(f"a job {spec['name']} is there already")
|
||||
return None
|
||||
if current is None:
|
||||
raise LookupError(f"no scheduled job job:{spec['name']}")
|
||||
if current["digest"] != digest:
|
||||
raise ScheduledJobChanged(f"job:{spec['name']} changed on the host since it was read")
|
||||
return current
|
||||
|
||||
def _sj_group(self, user: str) -> str:
|
||||
result = self.run_command(_command(["id", "-gn", "--", user]), timeout=30)
|
||||
if result.exit_code != 0:
|
||||
raise ScheduledJobRefused(f"the host has no user {user}")
|
||||
return result.stdout.strip()
|
||||
|
||||
def _sj_calendar(self, schedule: str) -> list[str]:
|
||||
if is_reboot(schedule):
|
||||
return ["OnBootSec=1min"]
|
||||
lines = to_on_calendar(schedule)
|
||||
for line in lines:
|
||||
checked = self.run_command(
|
||||
_command(["systemd-analyze", "calendar", "--", line]), timeout=30
|
||||
)
|
||||
if checked.exit_code != 0:
|
||||
raise ScheduledJobRefused(f"systemd does not take {line!r}: {_said(checked)}")
|
||||
return [f"OnCalendar={line}" for line in lines]
|
||||
|
||||
def _sj_put_timer(self, spec: ScheduledJobSpecDict) -> None:
|
||||
name, description = spec["name"], spec.get("description") or spec["name"]
|
||||
script = f"{self.JOB_SCRIPT_DIR}/{name}"
|
||||
unit_description = description.replace("%", "%%")
|
||||
timer = "\n".join(
|
||||
[
|
||||
"[Unit]",
|
||||
f"Description={unit_description}",
|
||||
"",
|
||||
"[Timer]",
|
||||
*self._sj_calendar(spec["schedule"]),
|
||||
f"Unit={name}.service",
|
||||
"",
|
||||
"[Install]",
|
||||
"WantedBy=timers.target",
|
||||
"",
|
||||
]
|
||||
)
|
||||
service = "\n".join(
|
||||
[
|
||||
"[Unit]",
|
||||
f"Description={unit_description}",
|
||||
"",
|
||||
"[Service]",
|
||||
"Type=oneshot",
|
||||
f"User={spec['run_as']}",
|
||||
f"ExecStart={script}",
|
||||
"",
|
||||
]
|
||||
)
|
||||
body = f"#!/bin/sh\n{JOB_MARKER}{name} {spec.get('description') or ''}".rstrip()
|
||||
body += f"\n{SCHEDULE_HEADER}{spec['schedule']}\n{spec['command'].strip()}\n"
|
||||
self._sj_install(body, script, "750", "root", self._sj_group(spec["run_as"]))
|
||||
self._sj_install(service, f"{self.UNIT_DIR}/{name}.service", "644", "root", self.ROOT_GROUP)
|
||||
self._sj_install(timer, f"{self.UNIT_DIR}/{name}.timer", "644", "root", self.ROOT_GROUP)
|
||||
self._sj_checked(["systemctl", "daemon-reload"])
|
||||
verb = "enable" if spec["enabled"] else "disable"
|
||||
self._sj_checked(["systemctl", verb, "--now", "--", f"{name}.timer"])
|
||||
|
||||
def _sj_cron_text(self, spec: ScheduledJobSpecDict, *, system: bool) -> list[str]:
|
||||
if "\n" in spec["command"].strip():
|
||||
raise ScheduledJobRefused("a cron line holds one line of command")
|
||||
owner = f" {spec['run_as']}" if system else ""
|
||||
line = f"{spec['schedule']}{owner} {spec['command'].strip()}"
|
||||
return marked_lines(
|
||||
spec["name"], spec.get("description") or "", line, enabled=spec["enabled"]
|
||||
)
|
||||
|
||||
def _sj_cron_d(self) -> str | None:
|
||||
""":attr:`OWN_CRON_D_DIR` where the host has it; BusyBox crond reads none."""
|
||||
directory = self.OWN_CRON_D_DIR
|
||||
if directory is None:
|
||||
return None
|
||||
found = self.run_command(_command(["test", "-d", directory]), timeout=30)
|
||||
return directory if found.exit_code == 0 else None
|
||||
|
||||
def _sj_put_cron(self, spec: ScheduledJobSpecDict, current: dict[str, object] | None) -> None:
|
||||
if self._sj_cron_d() is not None:
|
||||
self._sj_group(spec["run_as"])
|
||||
text = "\n".join(self._sj_cron_text(spec, system=True)) + "\n"
|
||||
self._sj_install(
|
||||
text, f"{self.OWN_CRON_D_DIR}/{spec['name']}", "644", "root", self.ROOT_GROUP
|
||||
)
|
||||
return
|
||||
if current is not None and current["run_as"] != spec["run_as"]:
|
||||
self._sj_delete_cron(current)
|
||||
user = spec["run_as"]
|
||||
text = self._sj_crontab(user)
|
||||
jobs = parse_crontab("", text, system=False, user=user)
|
||||
self._sj_write_crontab(
|
||||
user, with_job(text, jobs, spec["name"], self._sj_cron_text(spec, system=False))
|
||||
)
|
||||
|
||||
def put_scheduled_job(
|
||||
self, spec: ScheduledJobSpecDict, *, digest: str | None = None
|
||||
) -> ScheduledJobResultDict:
|
||||
"""Create or change the application's own job; see :class:`ScheduledJobsMixin`."""
|
||||
check_spec(spec)
|
||||
reading = self._sj_read()
|
||||
current = self._sj_own(reading, spec, digest)
|
||||
if reading.systemd:
|
||||
self._sj_put_timer(spec)
|
||||
else:
|
||||
self._sj_put_cron(spec, current)
|
||||
return {
|
||||
"success": True,
|
||||
"output": f"job:{spec['name']} {'changed' if current else 'created'}",
|
||||
}
|
||||
|
||||
def _sj_delete_cron(self, job: dict[str, object]) -> None:
|
||||
path = str(job["source"])
|
||||
if self._sj_spool_user(path) is None:
|
||||
self._sj_checked(["rm", "-f", "--", path])
|
||||
return
|
||||
text = self._sj_text(path)
|
||||
jobs = parse_crontab(path, text, system=False, user=str(job["run_as"]))
|
||||
self._sj_write(path, without_job(text, jobs, str(job["name"])))
|
||||
|
||||
def delete_scheduled_job(self, key: str, *, digest: str) -> ScheduledJobResultDict:
|
||||
"""Delete the application's job *key*; see :class:`ScheduledJobsMixin`."""
|
||||
job = self._sj_job(self._sj_read(), key, digest)
|
||||
if job["managed_by"] != "application":
|
||||
raise ScheduledJobRefused(
|
||||
f"{key} was not written by an application; it is only switched"
|
||||
)
|
||||
name = str(job["name"])
|
||||
if job["kind"] != "systemd_timer":
|
||||
self._sj_delete_cron(job)
|
||||
return {"success": True, "output": f"{key} deleted"}
|
||||
self._sj_root(["systemctl", "disable", "--now", "--", f"{name}.timer"])
|
||||
self._sj_checked(
|
||||
[
|
||||
"rm",
|
||||
"-f",
|
||||
"--",
|
||||
f"{self.UNIT_DIR}/{name}.timer",
|
||||
f"{self.UNIT_DIR}/{name}.service",
|
||||
f"{self.JOB_SCRIPT_DIR}/{name}",
|
||||
]
|
||||
)
|
||||
self._sj_checked(["systemctl", "daemon-reload"])
|
||||
return {"success": True, "output": f"{key} deleted"}
|
||||
@@ -15,7 +15,9 @@ accident.
|
||||
``/etc/ssh/sshd_config.d/*.conf`` -- Debian and Ubuntu do, at the top -- the
|
||||
lists go to :data:`DROPIN_PATH`, which sorts first and so wins. sshd_config
|
||||
stays as the distribution shipped it, and removing the file undoes the change.
|
||||
Only without such an ``Include`` is sshd_config itself rewritten, after a copy.
|
||||
Only without such an ``Include`` is sshd_config itself rewritten, after a copy --
|
||||
and not at all on a platform that generates it
|
||||
(:attr:`OpenSshServerMixin.SSHD_CONFIG_GENERATED`).
|
||||
|
||||
**Nobody is locked out, by three checks.**
|
||||
|
||||
@@ -51,6 +53,8 @@ from typing import (
|
||||
TYPE_CHECKING,
|
||||
)
|
||||
|
||||
from napalm_device_types.ssh_server_auth import SSHD_EFFECTIVE_COMMAND, SshServerAuthMixin
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - for type checkers only
|
||||
from napalm_device_types.channel import CommandResult
|
||||
|
||||
@@ -418,7 +422,7 @@ def _unchanged(
|
||||
return result
|
||||
|
||||
|
||||
class OpenSshServerMixin:
|
||||
class OpenSshServerMixin(SshServerAuthMixin):
|
||||
"""Adds :meth:`harden_ssh_server` to a driver that can run commands as root on an OpenSSH host.
|
||||
|
||||
Mixed in by the drivers whose host runs a stock OpenSSH, not by a base class,
|
||||
@@ -426,9 +430,19 @@ class OpenSshServerMixin:
|
||||
whose platform writes sshd_config itself (an appliance that regenerates it
|
||||
from its own database) must not inherit this, or must opt out with
|
||||
``harden_ssh_server = None``: its file would be overwritten on the next
|
||||
apply.
|
||||
apply. Where that generated file still includes the drop-in directory, the
|
||||
driver sets :attr:`SSHD_CONFIG_GENERATED` instead and keeps the hardening.
|
||||
|
||||
It carries :class:`~napalm_device_types.ssh_server_auth.SshServerAuthMixin`:
|
||||
a host whose sshd can be hardened can also say how it lets people log in.
|
||||
"""
|
||||
|
||||
#: True where the platform writes sshd_config from its own database
|
||||
#: (OpenMediaVault): only the drop-in directory is written, and a
|
||||
#: sshd_config that does not include it is left alone, since the platform
|
||||
#: would write over a change to it on its next apply.
|
||||
SSHD_CONFIG_GENERATED: bool = False
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def run_command(
|
||||
@@ -504,7 +518,7 @@ class OpenSshServerMixin:
|
||||
return parse_sshd_effective(self._sshd_effective_text())
|
||||
|
||||
def _sshd_effective_text(self) -> str:
|
||||
result = self._ssh_run("PATH=$PATH:/usr/sbin:/sbin sshd -T")
|
||||
result = self._ssh_run(SSHD_EFFECTIVE_COMMAND)
|
||||
if result.exit_code != 0:
|
||||
raise RuntimeError(f"sshd -T failed: {(result.stderr or result.stdout).strip()}")
|
||||
return result.stdout
|
||||
@@ -565,6 +579,13 @@ class OpenSshServerMixin:
|
||||
main = self._ssh_read(MAIN_CONFIG)
|
||||
if main is None:
|
||||
raise RuntimeError(f"cannot read {MAIN_CONFIG}")
|
||||
if self.SSHD_CONFIG_GENERATED and not has_dropin_include(main):
|
||||
result.message = (
|
||||
f"Not changed: {MAIN_CONFIG} does not include {DROPIN_DIR}, and this"
|
||||
" platform writes it from its own settings."
|
||||
)
|
||||
say(result.message)
|
||||
return result
|
||||
restore = self._write_change(plan, main, result, say)
|
||||
return self._apply_or_restore(
|
||||
result,
|
||||
|
||||
@@ -0,0 +1,281 @@
|
||||
"""How an OpenSSH server lets people log in: the effective settings and sshd's PAM stack.
|
||||
|
||||
Whether a host takes a password, only a key, or a key *and* a second factor is
|
||||
the same question on every host that runs OpenSSH, so the reading lives here,
|
||||
and a driver only mixes in :class:`SshServerAuthMixin` and supplies
|
||||
``run_command``.
|
||||
|
||||
**What sshd uses, not what a file says.** ``sshd -T`` prints the settings after
|
||||
sshd's defaults, ``sshd_config``, every ``Include`` and first-match-wins -- the
|
||||
same reading :class:`~napalm_device_types.ssh_server.OpenSshServerMixin` plans
|
||||
its hardening on, by the same fixed command (:data:`SSHD_EFFECTIVE_COMMAND`). It
|
||||
needs root: without it sshd cannot load its host keys and exits.
|
||||
|
||||
**PAM, as far as it can be read.** With ``UsePAM yes``, keyboard-interactive
|
||||
asks whatever PAM's ``auth`` stack for sshd asks: a password from
|
||||
``pam_unix.so``, a one-time code from an OTP module, or both. The stack is read
|
||||
from ``/etc/pam.d`` in one listing as the login user (the files are
|
||||
world-readable), and followed through ``@include``, ``include`` and
|
||||
``substack``. Each module is reported with its control and arguments as
|
||||
written; which of them is a second factor, and whether ``nullok`` lets a user
|
||||
without one through, is the consumer's call.
|
||||
|
||||
Not read: ``Match`` blocks (``sshd -T`` without ``-C`` prints the global
|
||||
settings), and what a PAM module's own configuration asks for.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from collections.abc import Mapping
|
||||
from typing import TYPE_CHECKING, TypedDict
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - for type checkers only
|
||||
from napalm_device_types.channel import CommandResult
|
||||
|
||||
#: sshd's effective configuration. ``sshd`` lives in ``/usr/sbin``, outside a login
|
||||
#: user's ``PATH``. One fixed command, run as root.
|
||||
SSHD_EFFECTIVE_COMMAND = "PATH=$PATH:/usr/sbin:/sbin sshd -T"
|
||||
|
||||
PAM_DIR = "/etc/pam.d"
|
||||
#: Every file in ``/etc/pam.d``, each after a ``==> path <==`` header (GNU, BSD and
|
||||
#: busybox ``tail`` alike). One fixed command, as the login user.
|
||||
PAM_FILES_COMMAND = f"tail -n +1 -- {PAM_DIR}/*"
|
||||
#: The PAM service sshd runs as.
|
||||
PAM_SERVICE = "sshd"
|
||||
|
||||
_TIMEOUT = 30
|
||||
_HEADER_RE = re.compile(rf"^==> {re.escape(PAM_DIR)}/(?P<name>[^/]+) <==$")
|
||||
#: A service name an include may name: a file in ``/etc/pam.d``, never a path out of it.
|
||||
_SERVICE_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]*$")
|
||||
_INCLUDE_CONTROLS = ("include", "substack")
|
||||
|
||||
|
||||
class PamAuthEntryDict(TypedDict):
|
||||
"""One line of a PAM ``auth`` stack: ``auth required pam_unix.so nullok``."""
|
||||
|
||||
#: The module's file name, without its directory: ``pam_unix.so``.
|
||||
module: str
|
||||
#: ``required``, ``requisite``, ``sufficient``, ``optional``, or a bracketed
|
||||
#: ``[value=action ...]`` as written.
|
||||
control: str
|
||||
args: list[str]
|
||||
|
||||
|
||||
class SshServerAuthDict(TypedDict):
|
||||
"""How an OpenSSH server lets people log in, as ``get_ssh_server_auth`` read it.
|
||||
|
||||
The global settings ``sshd -T`` prints. A ``Match`` block can change them for
|
||||
some users or addresses; ``sshd -T`` without ``-C`` does not show that.
|
||||
|
||||
``authentication_methods`` is ``AuthenticationMethods``: each inner list is one
|
||||
way in, every method of it required in turn (``publickey,keyboard-interactive``
|
||||
-> ``["publickey", "keyboard-interactive"]``); empty for ``any``, where any one
|
||||
enabled method is enough. A method keeps its submethod
|
||||
(``keyboard-interactive:pam``).
|
||||
|
||||
``use_pam`` is None where sshd has no PAM (OpenBSD). ``pam_auth_stack`` is the
|
||||
``auth`` stack PAM runs for sshd, includes and substacks followed; None when
|
||||
PAM is off or its files gave nothing for sshd.
|
||||
"""
|
||||
|
||||
password_authentication: bool
|
||||
#: ``KbdInteractiveAuthentication``; before OpenSSH 8.7, either it or
|
||||
#: ``ChallengeResponseAuthentication``.
|
||||
kbd_interactive_authentication: bool
|
||||
pubkey_authentication: bool
|
||||
hostbased_authentication: bool
|
||||
#: False also where sshd was built without GSSAPI.
|
||||
gssapi_authentication: bool
|
||||
authentication_methods: list[list[str]]
|
||||
#: ``yes``, ``prohibit-password`` (``without-password``), ``forced-commands-only`` or ``no``.
|
||||
permit_root_login: str
|
||||
use_pam: bool | None
|
||||
pam_auth_stack: list[PamAuthEntryDict] | None
|
||||
|
||||
|
||||
def _flag(settings: Mapping[str, str], keyword: str) -> bool:
|
||||
return settings.get(keyword, "no") == "yes"
|
||||
|
||||
|
||||
def _settings(text: str) -> dict[str, str]:
|
||||
"""``keyword value`` per line, the first one of each keyword."""
|
||||
found: dict[str, str] = {}
|
||||
for raw in text.splitlines():
|
||||
parts = raw.strip().split(None, 1)
|
||||
if len(parts) == 2:
|
||||
found.setdefault(parts[0].lower(), parts[1].strip())
|
||||
return found
|
||||
|
||||
|
||||
def _methods(value: str) -> list[list[str]]:
|
||||
if value == "any":
|
||||
return []
|
||||
return [[m for m in alternative.split(",") if m] for alternative in value.split()]
|
||||
|
||||
|
||||
def parse_sshd_auth(text: str) -> SshServerAuthDict:
|
||||
"""The authentication settings from ``sshd -T`` output; ``pam_auth_stack`` left None.
|
||||
|
||||
:raises ValueError: if a setting every OpenSSH prints is missing -- not sshd -T
|
||||
output, or cut short.
|
||||
"""
|
||||
settings = _settings(text)
|
||||
missing = [
|
||||
k
|
||||
for k in ("passwordauthentication", "pubkeyauthentication", "permitrootlogin")
|
||||
if k not in settings
|
||||
]
|
||||
if missing:
|
||||
raise ValueError(f"sshd -T printed no {', '.join(missing)}")
|
||||
use_pam = settings.get("usepam")
|
||||
return {
|
||||
"password_authentication": _flag(settings, "passwordauthentication"),
|
||||
"kbd_interactive_authentication": _flag(settings, "kbdinteractiveauthentication")
|
||||
or _flag(settings, "challengeresponseauthentication"),
|
||||
"pubkey_authentication": _flag(settings, "pubkeyauthentication"),
|
||||
"hostbased_authentication": _flag(settings, "hostbasedauthentication"),
|
||||
"gssapi_authentication": _flag(settings, "gssapiauthentication"),
|
||||
"authentication_methods": _methods(settings.get("authenticationmethods", "any")),
|
||||
"permit_root_login": settings["permitrootlogin"],
|
||||
"use_pam": None if use_pam is None else use_pam == "yes",
|
||||
"pam_auth_stack": None,
|
||||
}
|
||||
|
||||
|
||||
def _pam_tokens(line: str) -> list[str]:
|
||||
"""A PAM line's tokens, a bracketed control (``[success=1 default=ignore]``) as one."""
|
||||
tokens: list[str] = []
|
||||
for part in line.split():
|
||||
if tokens and tokens[-1].startswith("[") and not tokens[-1].endswith("]"):
|
||||
tokens[-1] += " " + part
|
||||
else:
|
||||
tokens.append(part)
|
||||
return tokens
|
||||
|
||||
|
||||
def _pam_entry(line: str) -> PamAuthEntryDict | None:
|
||||
tokens = _pam_tokens(line.split("#", 1)[0])
|
||||
if len(tokens) >= 2 and tokens[0] == "@include":
|
||||
tokens = ["auth", "include", tokens[1]]
|
||||
if len(tokens) < 3 or tokens[0].lstrip("-") != "auth":
|
||||
return None
|
||||
control, target, args = tokens[1], tokens[2], tokens[3:]
|
||||
if control in _INCLUDE_CONTROLS:
|
||||
if not _SERVICE_RE.match(target):
|
||||
return None
|
||||
return {"module": target, "control": control, "args": []}
|
||||
return {"module": target.rsplit("/", 1)[-1], "control": control, "args": args}
|
||||
|
||||
|
||||
def parse_pam_auth(text: str) -> list[PamAuthEntryDict]:
|
||||
"""One PAM file's ``auth`` stack, in order.
|
||||
|
||||
A module by its file name (``pam_unix.so``), with its control and arguments.
|
||||
An include is an entry whose control is ``include`` or ``substack`` and whose
|
||||
module is the service it names: ``@include common-auth`` brings in a whole
|
||||
file, ``auth include`` and ``auth substack`` its ``auth`` lines; both are
|
||||
followed by :func:`pam_auth_stack`. An include that names a path rather than
|
||||
a service is left out.
|
||||
"""
|
||||
return [entry for line in text.splitlines() if (entry := _pam_entry(line.strip()))]
|
||||
|
||||
|
||||
def parse_pam_files(text: str) -> dict[str, str]:
|
||||
""":data:`PAM_FILES_COMMAND`'s output, by file name."""
|
||||
files: dict[str, list[str]] = {}
|
||||
current: list[str] | None = None
|
||||
for line in text.splitlines():
|
||||
header = _HEADER_RE.match(line.strip())
|
||||
if header:
|
||||
current = files.setdefault(header["name"], [])
|
||||
elif current is not None:
|
||||
current.append(line)
|
||||
return {name: "\n".join(lines).strip("\n") + "\n" for name, lines in files.items()}
|
||||
|
||||
|
||||
def pam_auth_stack(
|
||||
files: Mapping[str, str], service: str = PAM_SERVICE
|
||||
) -> list[PamAuthEntryDict] | None:
|
||||
"""*service*'s ``auth`` modules, includes and substacks followed, in stack order.
|
||||
|
||||
A file included twice -- or in a loop -- counts once; an include of a file
|
||||
that is not there adds nothing, as with PAM. None without a file for *service*.
|
||||
"""
|
||||
if service not in files:
|
||||
return None
|
||||
stack: list[PamAuthEntryDict] = []
|
||||
seen: set[str] = set()
|
||||
|
||||
def walk(name: str) -> None:
|
||||
if name in seen or name not in files:
|
||||
return
|
||||
seen.add(name)
|
||||
for entry in parse_pam_auth(files[name]):
|
||||
if entry["control"] in _INCLUDE_CONTROLS:
|
||||
walk(entry["module"])
|
||||
else:
|
||||
stack.append(entry)
|
||||
|
||||
walk(service)
|
||||
return stack
|
||||
|
||||
|
||||
class SshServerAuthMixin:
|
||||
"""Adds :meth:`get_ssh_server_auth` to a driver that can run commands on an OpenSSH host.
|
||||
|
||||
Mixed in by the drivers whose host runs a stock OpenSSH --
|
||||
:class:`~napalm_device_types.ssh_server.OpenSshServerMixin` carries it -- so
|
||||
``hasattr(driver, "get_ssh_server_auth")`` stays a truthful answer. Reading,
|
||||
not writing: a platform that generates sshd_config itself can carry it too.
|
||||
"""
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def run_command(
|
||||
self,
|
||||
command: str,
|
||||
*,
|
||||
privileged: bool = False,
|
||||
timeout: float = 60,
|
||||
stdin: bytes | None = None,
|
||||
) -> CommandResult: ...
|
||||
|
||||
def get_ssh_server_auth(self) -> SshServerAuthDict:
|
||||
"""
|
||||
Returns how sshd lets people log in, from ``sshd -T`` and PAM's sshd stack.
|
||||
|
||||
Example::
|
||||
|
||||
{
|
||||
"password_authentication": False,
|
||||
"kbd_interactive_authentication": True,
|
||||
"pubkey_authentication": True,
|
||||
"hostbased_authentication": False,
|
||||
"gssapi_authentication": False,
|
||||
"authentication_methods": [["publickey", "keyboard-interactive"]],
|
||||
"permit_root_login": "no",
|
||||
"use_pam": True,
|
||||
"pam_auth_stack": [
|
||||
{"module": "pam_google_authenticator.so", "control": "required", "args": []}
|
||||
],
|
||||
}
|
||||
|
||||
:raises RuntimeError: if ``sshd -T`` fails -- no OpenSSH, or no root.
|
||||
"""
|
||||
result = self.run_command(SSHD_EFFECTIVE_COMMAND, privileged=True, timeout=_TIMEOUT)
|
||||
if result.exit_code != 0:
|
||||
raise RuntimeError(f"sshd -T failed: {(result.stderr or result.stdout).strip()}")
|
||||
auth = parse_sshd_auth(result.stdout)
|
||||
if auth["use_pam"]:
|
||||
auth["pam_auth_stack"] = self._pam_auth_stack()
|
||||
return auth
|
||||
|
||||
def _pam_auth_stack(self) -> list[PamAuthEntryDict] | None:
|
||||
"""sshd's PAM auth stack; None when ``/etc/pam.d`` gave nothing for sshd.
|
||||
|
||||
``tail`` goes on past a file it cannot open and exits 1, so what it did
|
||||
print is used whatever the exit status.
|
||||
"""
|
||||
result = self.run_command(PAM_FILES_COMMAND, privileged=False, timeout=_TIMEOUT)
|
||||
return pam_auth_stack(parse_pam_files(result.stdout))
|
||||
@@ -0,0 +1,193 @@
|
||||
"""A device's time sources and time zone: read them, and set them to what the caller says.
|
||||
|
||||
An application keeps one list of NTP servers and a time zone per site, and aligns a
|
||||
device that differs. :class:`TimeSyncMixin` is the contract for that:
|
||||
|
||||
* ``get_time_sync()`` -- declared only, so ``callable(getattr(driver,
|
||||
"get_time_sync", None))`` is the truthful answer to "can this driver tell";
|
||||
* ``configure_time_sync(ntp_servers=None, timezone=None)`` -- a fix template
|
||||
(:mod:`napalm_device_types.fixes`): ``None`` leaves that value as it is, and the
|
||||
device ends up configured with exactly the servers given, in their order. A driver
|
||||
implements the steps; one whose device takes the change in a single call
|
||||
overrides the method;
|
||||
* ``time_sync_writes`` -- which of the two values the driver can write. A BMC that
|
||||
runs on UTC only writes ``{"ntp_servers"}``; the application never asks it for a
|
||||
zone.
|
||||
|
||||
A driver that inherits the mixin but cannot run it sets ``get_time_sync`` (and
|
||||
``configure_time_sync``) to ``None``.
|
||||
|
||||
**Servers from DHCP count** (NetOrk/netork#2380). A device that takes its time
|
||||
servers from a DHCP lease (option 42) gets them from whoever serves the subnet --
|
||||
usually the site's time giver. ``ntp_servers`` are only the servers configured on
|
||||
the device; ``dhcp_ntp_servers`` are those the device got by DHCP and uses. A write
|
||||
replaces the configured servers and leaves DHCP's alone: it never switches the use of
|
||||
DHCP-provided servers off, and the ``verify`` step does not count them.
|
||||
|
||||
A zone is an IANA name on the wire. A device that wants a POSIX rule or an offset
|
||||
with a summer-time rule gets it from :mod:`napalm_device_types.posix_tz`.
|
||||
"""
|
||||
|
||||
# No ``from __future__ import annotations`` here: TypedDict tells a NotRequired key
|
||||
# only from an evaluated annotation (``TimeSyncDict.__optional_keys__``).
|
||||
|
||||
import ipaddress
|
||||
import re
|
||||
from collections.abc import Sequence
|
||||
from typing import TYPE_CHECKING, ClassVar, NotRequired, TypedDict
|
||||
|
||||
from napalm_device_types.fixes import FixStep, run_fix_steps
|
||||
from napalm_device_types.models import FixResultDict, FixStepOutcomeDict
|
||||
from napalm_device_types.posix_tz import UnknownTimezone, is_iana_zone
|
||||
|
||||
#: The values a driver can write; ``time_sync_writes`` is a subset.
|
||||
TIME_SYNC_VALUES: frozenset[str] = frozenset({"ntp_servers", "timezone"})
|
||||
|
||||
_HOST_NAME = re.compile(
|
||||
r"(?=.{1,253}$)[A-Za-z0-9](?:[A-Za-z0-9-]{0,61}[A-Za-z0-9])?"
|
||||
r"(?:\.[A-Za-z0-9](?:[A-Za-z0-9-]{0,61}[A-Za-z0-9])?)*\.?"
|
||||
)
|
||||
|
||||
|
||||
class TimeSyncDict(TypedDict):
|
||||
"""How a device keeps its clock (``get_time_sync``)."""
|
||||
|
||||
ntp_servers: list[str] # configured time sources, in the device's order; [] = none configured
|
||||
timezone: str | None # IANA name ("Europe/Berlin"); None = not readable or no named zone
|
||||
synchronized: bool | None # the clock is synchronised to a source right now; None = unknown
|
||||
daemon: str | None # "chrony", "systemd-timesyncd", "ntpd", "openntpd", ...; None = own client
|
||||
# The servers the device got by DHCP (option 42) and uses, in its order; never in
|
||||
# ntp_servers. [] = none; absent or None = the driver cannot tell.
|
||||
dhcp_ntp_servers: NotRequired[list[str] | None]
|
||||
|
||||
|
||||
def _is_server(name: str) -> bool:
|
||||
try:
|
||||
ipaddress.ip_address(name)
|
||||
except ValueError:
|
||||
return bool(_HOST_NAME.fullmatch(name))
|
||||
return True
|
||||
|
||||
|
||||
def check_ntp_servers(servers: Sequence[str]) -> list[str]:
|
||||
"""*servers* as a driver writes them: stripped, in their order, each once.
|
||||
|
||||
:raises ValueError: for no server at all, or one that is not a host name or
|
||||
an IP address -- before anything reaches a configuration file.
|
||||
:raises TypeError: for a single string instead of a list of them.
|
||||
"""
|
||||
if isinstance(servers, str):
|
||||
raise TypeError("ntp_servers is a list of servers, not one string")
|
||||
kept: list[str] = []
|
||||
for server in (str(s).strip() for s in servers):
|
||||
if not server:
|
||||
continue
|
||||
if not _is_server(server):
|
||||
raise ValueError(f"{server!r} is not a host name or address of an NTP server")
|
||||
if server.lower() not in (k.lower() for k in kept):
|
||||
kept.append(server)
|
||||
if not kept:
|
||||
raise ValueError("name at least one NTP server; None leaves the servers as they are")
|
||||
return kept
|
||||
|
||||
|
||||
class TimeSyncMixin:
|
||||
"""Read and set a device's time sources and time zone; see the module."""
|
||||
|
||||
#: Which of the two values this driver can write: a subset of {"ntp_servers", "timezone"}.
|
||||
#: A BMC that only runs on UTC writes {"ntp_servers"}; an application never asks it for a
|
||||
#: zone. Declared only: a driver that names nothing writes nothing.
|
||||
time_sync_writes: ClassVar[frozenset[str]]
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def get_time_sync(self) -> TimeSyncDict:
|
||||
"""How the device keeps its clock now.
|
||||
|
||||
``ntp_servers`` are the sources its configuration names, in its order
|
||||
(a pool by its name, not the addresses it resolved to);
|
||||
``dhcp_ntp_servers`` those a DHCP lease gave it and it uses, apart from
|
||||
them -- left out where the driver cannot tell. ``timezone``
|
||||
is an IANA name, or None where the device keeps an offset and a rule
|
||||
instead of a name. ``daemon`` names the client that keeps the clock on
|
||||
a host that has a choice; None for a device's own.
|
||||
|
||||
Example::
|
||||
|
||||
{
|
||||
"ntp_servers": ["ntp1.example.org", "10.0.0.1"],
|
||||
"timezone": "Europe/Berlin",
|
||||
"synchronized": True,
|
||||
"daemon": "chrony",
|
||||
"dhcp_ntp_servers": ["192.168.1.1"],
|
||||
}
|
||||
|
||||
:raises Exception: when the device could not be read.
|
||||
"""
|
||||
...
|
||||
|
||||
# The steps a driver implements:
|
||||
|
||||
def _write_ntp_servers(self, servers: list[str]) -> FixStepOutcomeDict:
|
||||
"""Leave the device configured with exactly *servers*, in this order:
|
||||
sources it had configured elsewhere stop counting (commented out,
|
||||
replaced). Servers it gets by DHCP stay, and so does its use of them."""
|
||||
...
|
||||
|
||||
def _write_time_zone(self, zone: str) -> FixStepOutcomeDict:
|
||||
"""Set the device's zone to *zone*, an IANA name of the database."""
|
||||
...
|
||||
|
||||
def configure_time_sync(
|
||||
self, ntp_servers: Sequence[str] | None = None, timezone: str | None = None
|
||||
) -> FixResultDict:
|
||||
"""Give the device exactly *ntp_servers*, in order, and *timezone*; None leaves one alone.
|
||||
|
||||
What the device gets by DHCP is not touched: it keeps using those servers too.
|
||||
|
||||
The steps: ``ntp_servers``, ``timezone``, then ``verify``, which reads the
|
||||
device again.
|
||||
|
||||
:raises UnknownTimezone: for a *timezone* that is not an IANA name of the
|
||||
database, before anything is written.
|
||||
:raises ValueError: for a value this driver does not write
|
||||
(:attr:`time_sync_writes`), for no server or a malformed one, or for
|
||||
nothing asked -- before anything is written.
|
||||
"""
|
||||
servers = None if ntp_servers is None else check_ntp_servers(ntp_servers)
|
||||
if timezone is not None and not is_iana_zone(timezone):
|
||||
raise UnknownTimezone(timezone)
|
||||
asked = {name for name, v in (("ntp_servers", servers), ("timezone", timezone)) if v}
|
||||
if not asked:
|
||||
raise ValueError("nothing to configure: name ntp_servers, timezone or both")
|
||||
refused = asked - getattr(self, "time_sync_writes", frozenset())
|
||||
if refused:
|
||||
raise ValueError(f"this driver does not write {', '.join(sorted(refused))}")
|
||||
steps: list[FixStep] = []
|
||||
if servers is not None:
|
||||
steps.append(("ntp_servers", lambda: self._write_ntp_servers(servers)))
|
||||
if timezone is not None:
|
||||
steps.append(("timezone", lambda: self._write_time_zone(timezone)))
|
||||
steps.append(("verify", lambda: self._verify_time_sync(servers, timezone)))
|
||||
return run_fix_steps(steps)
|
||||
|
||||
def _verify_time_sync(self, servers: list[str] | None, zone: str | None) -> FixStepOutcomeDict:
|
||||
"""Read the device again: does it now have what was written?
|
||||
|
||||
Servers compare without case; the servers DHCP brought are not among
|
||||
them. A zone is compared where the device reads one back by name.
|
||||
"""
|
||||
now = self.get_time_sync()
|
||||
wrong: list[str] = []
|
||||
if servers is not None and [s.lower() for s in now["ntp_servers"]] != [
|
||||
s.lower() for s in servers
|
||||
]:
|
||||
wrong.append(f"the device has the servers {', '.join(now['ntp_servers']) or 'none'}")
|
||||
if zone is not None and now["timezone"] is not None and now["timezone"] != zone:
|
||||
wrong.append(f"the device is in the time zone {now['timezone']}")
|
||||
if wrong:
|
||||
return {"ok": False, "output": "; ".join(wrong)}
|
||||
return {"ok": True, "output": "the device reads back what was written"}
|
||||
|
||||
|
||||
__all__ = ["TIME_SYNC_VALUES", "TimeSyncDict", "TimeSyncMixin", "check_ntp_servers"]
|
||||
@@ -0,0 +1,219 @@
|
||||
"""Time sync on a POSIX host, over the command channel: chrony, systemd-timesyncd, timedatectl.
|
||||
|
||||
:class:`PosixTimeSyncMixin` implements
|
||||
:class:`~napalm_device_types.time_sync.TimeSyncMixin` for a driver that has
|
||||
:meth:`~napalm_device_types.channel.CommandChannelMixin.run_command`. The defaults
|
||||
are Debian's paths; a driver sets its host's.
|
||||
|
||||
**Reading** is one round trip as the login user
|
||||
(:mod:`napalm_device_types._time_sync_reading`): ``timedatectl show`` for the zone
|
||||
(``/etc/timezone`` or ``/etc/localtime``'s target without it), which time sync
|
||||
service is active, and that client's configuration -- chrony's files as chronyd
|
||||
reads them, ``timedatectl show-timesync`` for systemd-timesyncd, ``ntp.conf``,
|
||||
OpenNTPD's ``ntpd.conf``. Synchronised is chronyd's own word where it runs (a
|
||||
selected source in ``chronyc -n sources``), else the kernel's (``NTPSynchronized``).
|
||||
|
||||
What DHCP brought (option 42) is read apart, as ``dhcp_ntp_servers``, and never as a
|
||||
configured server: chrony's sources from the DHCP client's ``sourcedir``
|
||||
(:attr:`~PosixTimeSyncMixin.CHRONY_DHCP_DIRS`, the files as chronyd reads them),
|
||||
systemd-timesyncd's ``LinkNTPServers``. Where ntpd or OpenNTPD runs it is unknown.
|
||||
|
||||
**Writing** runs as root through ``run_command(privileged=True)``; root never reads
|
||||
stdin -- a file is staged by the login user and installed by root.
|
||||
|
||||
* The zone: ``timedatectl set-timezone``.
|
||||
* chrony: the servers go to a file of their own and every other configured source is
|
||||
commented out (:mod:`napalm_device_types._chrony`); the DHCP client's ``sourcedir``
|
||||
stays, and one 4.35.0 commented out comes back. Then ``chronyc reload sources``, or
|
||||
a restart of chronyd where its main configuration changed or it cannot reload.
|
||||
* systemd-timesyncd: :attr:`~PosixTimeSyncMixin.TIMESYNCD_DROPIN` resets ``NTP=``,
|
||||
sets the servers and empties ``FallbackNTP=``, then the service restarts. Servers a
|
||||
network link brings (DHCP through systemd-networkd) stay and count.
|
||||
* ntpd, OpenNTPD, or no client: the servers are read, not written -- the step fails
|
||||
and says so.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shlex
|
||||
from collections.abc import Sequence
|
||||
from typing import TYPE_CHECKING, ClassVar
|
||||
|
||||
from napalm_device_types._chrony import (
|
||||
COMMENT_MARK,
|
||||
comment_out_command,
|
||||
include_command,
|
||||
plan_chrony,
|
||||
restore_command,
|
||||
)
|
||||
from napalm_device_types._scheduled_reading import TIME_ZONE_SH
|
||||
from napalm_device_types._time_sync_reading import (
|
||||
CHRONY_DHCP_DIRS,
|
||||
TIME_SYNC_UNITS,
|
||||
Reading,
|
||||
ReadPaths,
|
||||
chrony_synchronized,
|
||||
parse_key_values,
|
||||
parse_reading,
|
||||
parse_time_sync,
|
||||
reading_command,
|
||||
time_sync_of,
|
||||
timesyncd_link_servers,
|
||||
timesyncd_servers,
|
||||
)
|
||||
from napalm_device_types.models import FixStepOutcomeDict
|
||||
from napalm_device_types.time_sync import TIME_SYNC_VALUES, TimeSyncDict, TimeSyncMixin
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from napalm_device_types.channel import CommandResult
|
||||
|
||||
_STAGE = 't=$(mktemp) && chmod 644 "$t" && cat > "$t" && printf %s "$t"'
|
||||
_WRITERS = "chrony and systemd-timesyncd"
|
||||
|
||||
#: The head of the systemd-timesyncd drop-in.
|
||||
TIMESYNCD_HEADER = (
|
||||
"# The time sources, exactly these: written by napalm-device-types'\n"
|
||||
"# PosixTimeSyncMixin and replaced on every change.\n"
|
||||
)
|
||||
|
||||
|
||||
def _said(result: CommandResult) -> str:
|
||||
return "\n".join(p for p in (result.stdout.strip(), result.stderr.strip()) if p)
|
||||
|
||||
|
||||
def timesyncd_dropin(servers: Sequence[str]) -> str:
|
||||
"""The drop-in that leaves systemd-timesyncd with exactly *servers*."""
|
||||
return f"{TIMESYNCD_HEADER}[Time]\nNTP=\nNTP={' '.join(servers)}\nFallbackNTP=\n"
|
||||
|
||||
|
||||
class PosixTimeSyncMixin(TimeSyncMixin):
|
||||
"""Time sync of a host reached through ``run_command``; the paths are the driver's."""
|
||||
|
||||
time_sync_writes: ClassVar[frozenset[str]] = TIME_SYNC_VALUES
|
||||
|
||||
#: ``(unit, daemon)``: the services that keep the clock; the first active one counts.
|
||||
TIME_SYNC_UNITS: ClassVar[tuple[tuple[str, str], ...]] = TIME_SYNC_UNITS
|
||||
#: chrony's main configuration: the first that exists.
|
||||
CHRONY_CONFIGS: ClassVar[tuple[str, ...]] = ("/etc/chrony/chrony.conf", "/etc/chrony.conf")
|
||||
#: The servers' file where the main configuration has a ``sourcedir`` of its directory.
|
||||
CHRONY_SOURCES_FILE: ClassVar[str] = "/etc/chrony/sources.d/netork.sources"
|
||||
#: The servers' file otherwise (its name also in a ``confdir``); included if need be.
|
||||
CHRONY_INCLUDE_FILE: ClassVar[str] = "/etc/chrony.d/netork.conf"
|
||||
#: The ``sourcedir`` a DHCP client writes chrony's sources to: read as
|
||||
#: ``dhcp_ntp_servers``, never commented out.
|
||||
CHRONY_DHCP_DIRS: ClassVar[tuple[str, ...]] = CHRONY_DHCP_DIRS
|
||||
TIMESYNCD_DROPIN: ClassVar[str] = "/etc/systemd/timesyncd.conf.d/netork.conf"
|
||||
NTPD_CONFIGS: ClassVar[tuple[str, ...]] = ("/etc/ntpsec/ntp.conf", "/etc/ntp.conf")
|
||||
OPENNTPD_CONFIGS: ClassVar[tuple[str, ...]] = ("/etc/openntpd/ntpd.conf", "/etc/ntpd.conf")
|
||||
#: ``sh`` that prints the zone without timedatectl: a name, or a path ending in it.
|
||||
TIME_ZONE_SH: ClassVar[str] = TIME_ZONE_SH
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def run_command(
|
||||
self,
|
||||
command: str,
|
||||
*,
|
||||
privileged: bool = False,
|
||||
timeout: float = 60,
|
||||
stdin: bytes | None = None,
|
||||
) -> CommandResult: ...
|
||||
|
||||
# -- reading ---------------------------------------------------------------------------------
|
||||
|
||||
def _ts_read(self) -> Reading:
|
||||
paths = ReadPaths(
|
||||
units=self.TIME_SYNC_UNITS,
|
||||
chrony_configs=self.CHRONY_CONFIGS,
|
||||
ntpd_configs=self.NTPD_CONFIGS,
|
||||
openntpd_configs=self.OPENNTPD_CONFIGS,
|
||||
time_zone_sh=self.TIME_ZONE_SH,
|
||||
)
|
||||
return parse_reading(self.run_command(reading_command(paths), timeout=60).stdout)
|
||||
|
||||
def get_time_sync(self) -> TimeSyncDict:
|
||||
"""The host's time sources, zone and client, in one round trip as the login user.
|
||||
|
||||
:raises ValueError: when the reading came back cut short.
|
||||
"""
|
||||
return time_sync_of(self._ts_read(), self.CHRONY_DHCP_DIRS)
|
||||
|
||||
# -- writing ---------------------------------------------------------------------------------
|
||||
|
||||
def _ts_root(self, command: str) -> CommandResult:
|
||||
return self.run_command(command, privileged=True, timeout=60)
|
||||
|
||||
def _ts_checked(self, command: str) -> str:
|
||||
result = self._ts_root(command)
|
||||
if result.exit_code != 0:
|
||||
raise RuntimeError(f"{command}: {_said(result) or f'exit {result.exit_code}'}")
|
||||
return _said(result)
|
||||
|
||||
def _ts_install(self, path: str, content: str) -> None:
|
||||
"""*content* to *path* as root, staged by the login user: root reads no stdin."""
|
||||
staged = self.run_command(_STAGE, timeout=30, stdin=content.encode())
|
||||
tmp = (staged.stdout.strip().splitlines() or [""])[-1]
|
||||
if staged.exit_code != 0 or not tmp.startswith("/"):
|
||||
raise RuntimeError(f"cannot stage {path}: {_said(staged)}")
|
||||
try:
|
||||
self._ts_checked(shlex.join(["install", "-D", "-m", "644", "--", tmp, path]))
|
||||
finally:
|
||||
self.run_command(shlex.join(["rm", "-f", "--", tmp]), timeout=30)
|
||||
|
||||
def _write_time_zone(self, zone: str) -> FixStepOutcomeDict:
|
||||
result = self._ts_root(shlex.join(["timedatectl", "set-timezone", zone]))
|
||||
if result.exit_code != 0:
|
||||
return {"ok": False, "output": _said(result) or f"exit {result.exit_code}"}
|
||||
return {"ok": True, "output": f"time zone set to {zone}"}
|
||||
|
||||
def _write_ntp_servers(self, servers: list[str]) -> FixStepOutcomeDict:
|
||||
reading = self._ts_read()
|
||||
if reading.daemon == "chrony":
|
||||
return self._ts_chrony(reading, servers)
|
||||
if reading.daemon == "systemd-timesyncd":
|
||||
return self._ts_timesyncd(servers)
|
||||
runs = f"this host runs {reading.daemon}" if reading.daemon else "none runs here"
|
||||
return {"ok": False, "output": f"time sources are written for {_WRITERS}; {runs}"}
|
||||
|
||||
def _ts_timesyncd(self, servers: list[str]) -> FixStepOutcomeDict:
|
||||
self._ts_install(self.TIMESYNCD_DROPIN, timesyncd_dropin(servers))
|
||||
self._ts_checked("systemctl restart systemd-timesyncd.service")
|
||||
return {"ok": True, "output": f"wrote {self.TIMESYNCD_DROPIN}; restarted systemd-timesyncd"}
|
||||
|
||||
def _ts_chrony(self, reading: Reading, servers: list[str]) -> FixStepOutcomeDict:
|
||||
plan = plan_chrony(
|
||||
reading.chrony,
|
||||
servers,
|
||||
sources_file=self.CHRONY_SOURCES_FILE,
|
||||
include_file=self.CHRONY_INCLUDE_FILE,
|
||||
dhcp_dirs=self.CHRONY_DHCP_DIRS,
|
||||
)
|
||||
said = [f"wrote {plan.target}"]
|
||||
self._ts_install(plan.target, plan.content)
|
||||
for path, numbers in plan.restore.items():
|
||||
self._ts_checked(restore_command(path, numbers))
|
||||
said.append(f"gave DHCP's sourcedir back in {path}:{','.join(map(str, numbers))}")
|
||||
for path, numbers in plan.comment_out.items():
|
||||
self._ts_checked(comment_out_command(path, numbers))
|
||||
said.append(f"commented out {path}:{','.join(map(str, numbers))}")
|
||||
if plan.include_in is not None:
|
||||
self._ts_checked(include_command(plan.include_in, plan.target))
|
||||
said.append(f"included it from {plan.include_in}")
|
||||
if not (plan.reload and self._ts_root("chronyc reload sources").exit_code == 0):
|
||||
self._ts_checked(shlex.join(["systemctl", "restart", f"{reading.unit}.service"]))
|
||||
said.append(f"restarted {reading.unit}")
|
||||
else:
|
||||
said.append("reloaded the sources")
|
||||
return {"ok": True, "output": "; ".join(said)}
|
||||
|
||||
|
||||
__all__ = [
|
||||
"COMMENT_MARK",
|
||||
"PosixTimeSyncMixin",
|
||||
"chrony_synchronized",
|
||||
"parse_key_values",
|
||||
"parse_time_sync",
|
||||
"timesyncd_dropin",
|
||||
"timesyncd_link_servers",
|
||||
"timesyncd_servers",
|
||||
]
|
||||
+2
-1
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "napalm-device-types"
|
||||
version = "4.24.0"
|
||||
version = "4.36.0"
|
||||
description = "Abstract device-type base classes for NAPALM drivers"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.12"
|
||||
@@ -40,6 +40,7 @@ dependencies = [
|
||||
"napalm>=5.1", # older releases import pkg_resources, which a 3.12 venv lacks
|
||||
"requests>=2.25.0",
|
||||
"typing-extensions>=4.0",
|
||||
"tzdata>=2024.1", # posix_tz reads the zones' TZif files from it
|
||||
]
|
||||
|
||||
[project.urls]
|
||||
|
||||
@@ -32,7 +32,7 @@
|
||||
},
|
||||
"mypy_relaxed": [],
|
||||
"python_file_lines": {
|
||||
"napalm_device_types/models.py": 1463
|
||||
"napalm_device_types/models.py": 1454
|
||||
},
|
||||
"untyped_public": []
|
||||
}
|
||||
@@ -0,0 +1,234 @@
|
||||
"""A backup server is a role of its own (NAPALM/napalm-device-types#82).
|
||||
|
||||
What makes a device a backup server -- datastores, the backup groups in them,
|
||||
the jobs that keep them (garbage collection, verify, prune, sync, tape) and the
|
||||
remotes it syncs with -- has no counterpart in ``StorageDriver``. A backup
|
||||
server that also manages its own disks lists ``StorageDriver`` as a second role.
|
||||
|
||||
The declarations sit under ``TYPE_CHECKING`` (see test_role_contracts), so the
|
||||
signatures are read from the source rather than from the class.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import ast
|
||||
import inspect
|
||||
from typing import NotRequired, get_args, get_type_hints
|
||||
|
||||
import pytest
|
||||
|
||||
import napalm_device_types.backup_server as backup_server_module
|
||||
from napalm_device_types import (
|
||||
BACKUP_JOB_KINDS,
|
||||
BackupDatastoreDict,
|
||||
BackupGroupDict,
|
||||
BackupRemoteDict,
|
||||
BackupServerDriver,
|
||||
BackupServerJobDict,
|
||||
BackupServerRefDict,
|
||||
BackupSnapshotDict,
|
||||
BackupSnapshotRefDict,
|
||||
BackupTaskDict,
|
||||
BackupTaskStatusDict,
|
||||
ClusterStorageDict,
|
||||
OSDriver,
|
||||
StorageDriver,
|
||||
)
|
||||
from napalm_device_types.backups import BackupJobKind, BackupRunState, BackupVerifyState
|
||||
from napalm_device_types.roles import primary_role_of, role_keys_of
|
||||
|
||||
|
||||
def _declared(name: str) -> ast.FunctionDef:
|
||||
tree = ast.parse(inspect.getsource(backup_server_module))
|
||||
for node in ast.walk(tree):
|
||||
if isinstance(node, ast.FunctionDef) and node.name == name:
|
||||
return node
|
||||
raise AssertionError(f"BackupServerDriver does not declare {name}()")
|
||||
|
||||
|
||||
def _signature(name: str) -> tuple[list[str], str]:
|
||||
fn = _declared(name)
|
||||
params = ast.unparse(fn.args).split(", ")[1:]
|
||||
return params, ast.unparse(fn.returns) if fn.returns else ""
|
||||
|
||||
|
||||
class TestRole:
|
||||
def test_role_key_and_label(self):
|
||||
assert vars(BackupServerDriver)["ROLE"] == "backup_server"
|
||||
assert BackupServerDriver.TYPE_LABEL == "Backup Server"
|
||||
|
||||
def test_backup_server_first_storage_second(self):
|
||||
"""A PBS manages its disks and ZFS pools too; that part is storage."""
|
||||
|
||||
class Pbs(BackupServerDriver, StorageDriver, OSDriver):
|
||||
pass
|
||||
|
||||
assert role_keys_of(Pbs) == ["backup_server", "storage", "linux"]
|
||||
assert primary_role_of(Pbs) == "backup_server"
|
||||
|
||||
|
||||
class TestDeclarations:
|
||||
@pytest.mark.parametrize(
|
||||
("name", "params", "returns"),
|
||||
[
|
||||
("get_backup_datastores", [], "list[BackupDatastoreDict]"),
|
||||
("get_backup_groups", ["datastore: str=''"], "list[BackupGroupDict]"),
|
||||
(
|
||||
"get_backup_snapshots",
|
||||
["datastore: str", "namespace: str", "group_type: str", "group_id: str"],
|
||||
"list[BackupSnapshotDict]",
|
||||
),
|
||||
("get_backup_jobs", [], "list[BackupServerJobDict]"),
|
||||
("get_backup_remotes", [], "list[BackupRemoteDict]"),
|
||||
("start_garbage_collection", ["datastore: str"], "BackupTaskDict"),
|
||||
("run_backup_job", ["kind: BackupJobKind", "job_id: str"], "BackupTaskDict"),
|
||||
("verify_backup_snapshot", ["snapshot: BackupSnapshotRefDict"], "BackupTaskDict"),
|
||||
(
|
||||
"set_backup_snapshot_protected",
|
||||
["snapshot: BackupSnapshotRefDict", "protected: bool"],
|
||||
"None",
|
||||
),
|
||||
("delete_backup_snapshot", ["snapshot: BackupSnapshotRefDict"], "None"),
|
||||
("get_backup_task", ["task: str"], "BackupTaskStatusDict"),
|
||||
],
|
||||
)
|
||||
def test_declared_with_its_signature(self, name, params, returns):
|
||||
got_params, got_returns = _signature(name)
|
||||
assert [p.replace(" ", "") for p in got_params] == [p.replace(" ", "") for p in params]
|
||||
assert got_returns == returns
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"name",
|
||||
[
|
||||
"get_backup_datastores",
|
||||
"get_backup_groups",
|
||||
"get_backup_snapshots",
|
||||
"get_backup_jobs",
|
||||
"get_backup_remotes",
|
||||
"start_garbage_collection",
|
||||
"run_backup_job",
|
||||
"verify_backup_snapshot",
|
||||
"set_backup_snapshot_protected",
|
||||
"delete_backup_snapshot",
|
||||
"get_backup_task",
|
||||
],
|
||||
)
|
||||
def test_absent_until_a_driver_implements_it(self, name):
|
||||
assert not hasattr(BackupServerDriver, name)
|
||||
|
||||
def test_every_declaration_has_a_docstring(self):
|
||||
tree = ast.parse(inspect.getsource(backup_server_module))
|
||||
missing = [
|
||||
n.name
|
||||
for n in ast.walk(tree)
|
||||
if isinstance(n, ast.FunctionDef) and not ast.get_docstring(n)
|
||||
]
|
||||
assert missing == []
|
||||
|
||||
|
||||
class TestShapes:
|
||||
"""Neutral names: nothing here is Proxmox's. A value the platform does not
|
||||
report is None; it is never guessed."""
|
||||
|
||||
def test_datastore(self):
|
||||
assert set(get_type_hints(BackupDatastoreDict)) == {
|
||||
"name",
|
||||
"path",
|
||||
"backend",
|
||||
"total",
|
||||
"used",
|
||||
"available",
|
||||
"estimated_full",
|
||||
"maintenance",
|
||||
"comment",
|
||||
}
|
||||
|
||||
def test_group_carries_its_newest_snapshot_not_all_of_them(self):
|
||||
assert set(get_type_hints(BackupGroupDict)) == {
|
||||
"datastore",
|
||||
"namespace",
|
||||
"type",
|
||||
"id",
|
||||
"owner",
|
||||
"count",
|
||||
"newest",
|
||||
"oldest_time",
|
||||
"comment",
|
||||
}
|
||||
assert get_type_hints(BackupGroupDict)["newest"] == BackupSnapshotDict | None
|
||||
|
||||
def test_snapshot(self):
|
||||
assert set(get_type_hints(BackupSnapshotDict)) == {
|
||||
"time",
|
||||
"size",
|
||||
"verify",
|
||||
"protected",
|
||||
"comment",
|
||||
}
|
||||
assert set(get_args(BackupVerifyState)) == {"ok", "failed"}
|
||||
|
||||
def test_snapshot_reference_names_one_snapshot(self):
|
||||
assert set(get_type_hints(BackupSnapshotRefDict)) == {
|
||||
"datastore",
|
||||
"namespace",
|
||||
"group_type",
|
||||
"group_id",
|
||||
"time",
|
||||
}
|
||||
|
||||
def test_job_kinds_include_garbage_collection(self):
|
||||
"""GC is a job with a schedule and a last run like the others, so it is
|
||||
listed with them rather than restated on the datastore."""
|
||||
assert set(get_args(BackupJobKind)) == {"gc", "verify", "prune", "sync", "tape"}
|
||||
assert set(BACKUP_JOB_KINDS) == set(get_args(BackupJobKind))
|
||||
|
||||
def test_job(self):
|
||||
assert set(get_type_hints(BackupServerJobDict)) == {
|
||||
"id",
|
||||
"kind",
|
||||
"datastore",
|
||||
"namespace",
|
||||
"schedule",
|
||||
"enabled",
|
||||
"last_run",
|
||||
"last_state",
|
||||
"last_error",
|
||||
"next_run",
|
||||
"remote",
|
||||
"direction",
|
||||
"comment",
|
||||
}
|
||||
assert set(get_args(BackupRunState)) == {"ok", "warning", "error", "running"}
|
||||
|
||||
def test_remote_never_carries_its_secret(self):
|
||||
fields = set(get_type_hints(BackupRemoteDict))
|
||||
assert fields == {"name", "host", "port", "fingerprint", "auth_id", "comment"}
|
||||
assert not fields & {"password", "secret", "token"}
|
||||
|
||||
def test_task(self):
|
||||
assert set(get_type_hints(BackupTaskDict)) == {"task"}
|
||||
assert set(get_type_hints(BackupTaskStatusDict)) == {
|
||||
"task",
|
||||
"running",
|
||||
"state",
|
||||
"error",
|
||||
"log",
|
||||
}
|
||||
|
||||
|
||||
class TestClusterStorageNamesItsBackupServer:
|
||||
"""A hypervisor's backup storage says which server and datastore it points
|
||||
at, so an application can tie the server's groups to the cluster's guests."""
|
||||
|
||||
def test_backup_server_reference_is_optional_on_a_storage(self):
|
||||
hints = get_type_hints(ClusterStorageDict, include_extras=True)
|
||||
assert hints["backup_server"] == NotRequired[BackupServerRefDict | None]
|
||||
|
||||
def test_reference(self):
|
||||
assert set(get_type_hints(BackupServerRefDict)) == {
|
||||
"server",
|
||||
"port",
|
||||
"datastore",
|
||||
"namespace",
|
||||
"fingerprint",
|
||||
}
|
||||
@@ -25,6 +25,7 @@ import napalm_device_types.hypervisor as hypervisor_module
|
||||
from napalm_device_types import (
|
||||
BackupDict,
|
||||
BackupJobDict,
|
||||
BackupServerRefDict,
|
||||
ClusterDict,
|
||||
ClusterNetworkDict,
|
||||
ClusterNodeDict,
|
||||
@@ -92,7 +93,8 @@ class TestClusterShape:
|
||||
assert "votes" not in ClusterNodeDict.__required_keys__
|
||||
|
||||
def test_storage(self):
|
||||
assert get_type_hints(ClusterStorageDict) == {
|
||||
hints = get_type_hints(ClusterStorageDict, include_extras=True)
|
||||
assert hints == {
|
||||
"name": str,
|
||||
"type": str,
|
||||
"shared": bool,
|
||||
@@ -100,7 +102,10 @@ class TestClusterShape:
|
||||
"nodes": list[str],
|
||||
"total": int | None,
|
||||
"used": int | None,
|
||||
"backup_server": NotRequired[BackupServerRefDict | None],
|
||||
}
|
||||
# A driver written before backup servers were part of the report stays valid.
|
||||
assert "backup_server" not in ClusterStorageDict.__required_keys__
|
||||
|
||||
def test_network(self):
|
||||
hints = get_type_hints(ClusterNetworkDict, include_extras=True)
|
||||
|
||||
@@ -0,0 +1,141 @@
|
||||
"""Cron expressions: what they mean, when they fire next, and as systemd ``OnCalendar``.
|
||||
|
||||
One module answers all three, because a job netOrk writes as a timer must fire
|
||||
exactly when the cron expression the user gave says, and the preview must show
|
||||
the same runs the host will make (NetOrk/netork#1933).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import UTC, datetime
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types.cron import is_reboot, next_runs, parse_cron, to_on_calendar
|
||||
|
||||
BERLIN = ZoneInfo("Europe/Berlin")
|
||||
|
||||
|
||||
class TestParse:
|
||||
def test_every_field_is_expanded_to_its_values(self):
|
||||
s = parse_cron("*/15 6-8 1,15 * *")
|
||||
assert s.minutes == frozenset({0, 15, 30, 45})
|
||||
assert s.hours == frozenset({6, 7, 8})
|
||||
assert s.days == frozenset({1, 15})
|
||||
assert s.months == frozenset(range(1, 13))
|
||||
assert s.weekdays == frozenset(range(7))
|
||||
|
||||
def test_a_range_takes_a_step(self):
|
||||
assert parse_cron("1-30/10 * * * *").minutes == frozenset({1, 11, 21})
|
||||
|
||||
def test_names_are_months_and_weekdays_in_any_case(self):
|
||||
s = parse_cron("0 0 * Jan,MAR mon-Fri")
|
||||
assert s.months == frozenset({1, 3})
|
||||
assert s.weekdays == frozenset({1, 2, 3, 4, 5})
|
||||
|
||||
def test_seven_is_sunday_too(self):
|
||||
assert parse_cron("0 0 * * 7").weekdays == frozenset({0})
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("alias", "expanded"),
|
||||
[
|
||||
("@hourly", "0 * * * *"),
|
||||
("@daily", "0 0 * * *"),
|
||||
("@midnight", "0 0 * * *"),
|
||||
("@weekly", "0 0 * * 0"),
|
||||
("@monthly", "0 0 1 * *"),
|
||||
("@yearly", "0 0 1 1 *"),
|
||||
("@annually", "0 0 1 1 *"),
|
||||
],
|
||||
)
|
||||
def test_an_alias_is_its_expression(self, alias, expanded):
|
||||
assert parse_cron(alias) == parse_cron(expanded)
|
||||
|
||||
def test_reboot_is_no_calendar(self):
|
||||
assert is_reboot("@reboot")
|
||||
assert not is_reboot("0 0 * * *")
|
||||
with pytest.raises(ValueError, match="reboot"):
|
||||
parse_cron("@reboot")
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"bad",
|
||||
[
|
||||
"",
|
||||
"* * * *",
|
||||
"* * * * * *",
|
||||
"60 * * * *",
|
||||
"* 24 * * *",
|
||||
"* * 0 * *",
|
||||
"* * * 13 *",
|
||||
"* * * * 8",
|
||||
"*/0 * * * *",
|
||||
"5-1 * * * *",
|
||||
"a * * * *",
|
||||
"@often",
|
||||
"1,,2 * * * *",
|
||||
],
|
||||
)
|
||||
def test_what_cron_would_not_take_is_refused(self, bad):
|
||||
with pytest.raises(ValueError):
|
||||
parse_cron(bad)
|
||||
|
||||
def test_day_of_month_and_weekday_are_or_only_when_both_are_restricted(self):
|
||||
assert parse_cron("0 0 1 * 1").either_day
|
||||
assert not parse_cron("0 0 */2 * 1").either_day
|
||||
assert not parse_cron("0 0 1 * *").either_day
|
||||
|
||||
|
||||
class TestNextRuns:
|
||||
def test_runs_after_the_instant_in_the_hosts_time(self):
|
||||
after = datetime(2026, 11, 3, 5, 50, tzinfo=UTC) # 06:50 in Berlin
|
||||
runs = next_runs("0 7 * * *", after, BERLIN, count=2)
|
||||
assert runs == [
|
||||
datetime(2026, 11, 3, 7, 0, tzinfo=BERLIN),
|
||||
datetime(2026, 11, 4, 7, 0, tzinfo=BERLIN),
|
||||
]
|
||||
|
||||
def test_the_instant_itself_is_not_a_next_run(self):
|
||||
after = datetime(2026, 11, 3, 6, 0, tzinfo=UTC)
|
||||
assert next_runs("0 6 * * *", after, ZoneInfo("UTC"), count=1)[0].day == 4
|
||||
|
||||
def test_either_day_fires_on_both(self):
|
||||
after = datetime(2026, 11, 30, 12, 0, tzinfo=UTC) # a Monday
|
||||
runs = next_runs("0 0 1 * 1", after, ZoneInfo("UTC"), count=3)
|
||||
assert [r.day for r in runs] == [1, 7, 14]
|
||||
|
||||
def test_a_day_that_never_comes_gives_no_runs(self):
|
||||
assert next_runs("0 0 31 2 *", datetime(2026, 1, 1, tzinfo=UTC), BERLIN) == []
|
||||
|
||||
def test_a_time_the_clock_skips_is_left_out(self):
|
||||
# 2026-03-29 02:30 does not exist in Berlin.
|
||||
after = datetime(2026, 3, 28, 12, 0, tzinfo=UTC)
|
||||
runs = next_runs("30 2 * * *", after, BERLIN, count=2)
|
||||
assert [r.day for r in runs] == [30, 31]
|
||||
|
||||
def test_reboot_has_no_next_run(self):
|
||||
assert next_runs("@reboot", datetime(2026, 1, 1, tzinfo=UTC), BERLIN) == []
|
||||
|
||||
|
||||
class TestOnCalendar:
|
||||
def test_every_field_as_its_values(self):
|
||||
assert to_on_calendar("*/15 6 * * *") == ["*-*-* 06:00,15,30,45:00"]
|
||||
|
||||
def test_a_restricted_weekday_leads(self):
|
||||
assert to_on_calendar("30 4 * * 1-5") == ["Mon,Tue,Wed,Thu,Fri *-*-* 04:30:00"]
|
||||
|
||||
def test_months_and_days(self):
|
||||
assert to_on_calendar("0 0 1,15 1,7 *") == ["*-01,07-01,15 00:00:00"]
|
||||
|
||||
def test_either_day_is_two_lines(self):
|
||||
assert to_on_calendar("0 0 1 * 0") == ["*-*-01 00:00:00", "Sun *-*-* 00:00:00"]
|
||||
|
||||
def test_both_days_with_a_star_are_one_line(self):
|
||||
assert to_on_calendar("0 0 */10 * 6") == ["Sat *-*-01,11,21,31 00:00:00"]
|
||||
|
||||
def test_an_alias(self):
|
||||
assert to_on_calendar("@daily") == ["*-*-* 00:00:00"]
|
||||
|
||||
def test_reboot_is_refused(self):
|
||||
with pytest.raises(ValueError, match="reboot"):
|
||||
to_on_calendar("@reboot")
|
||||
@@ -0,0 +1,145 @@
|
||||
"""Crontabs as text: the jobs in them, and changing one line (NetOrk/netork#1933)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types._crontab import (
|
||||
DISABLED_MARKER,
|
||||
marked_lines,
|
||||
parse_anacrontab,
|
||||
parse_crontab,
|
||||
run_script,
|
||||
switched,
|
||||
with_job,
|
||||
without_job,
|
||||
)
|
||||
|
||||
SYSTEM = """\
|
||||
# /etc/crontab: system-wide crontab
|
||||
SHELL=/bin/sh
|
||||
PATH="/usr/local/sbin:/usr/bin:/bin"
|
||||
17 * * * * root cd / && run-parts --report /etc/cron.hourly
|
||||
@reboot root /usr/local/bin/warm-cache # keeps the # in the command
|
||||
#napalm-disabled: 0 4 * * * backup /usr/local/bin/backup
|
||||
garbage line
|
||||
0 4 * * * backup /usr/local/bin/backup
|
||||
"""
|
||||
|
||||
USER = """\
|
||||
MAILTO=ops@example.org
|
||||
*/5 * * * * /usr/bin/php /var/www/cron.php
|
||||
# napalm-scheduled-job: netork-cleanup tidy the cache
|
||||
0 3 * * * find /tmp -mtime +7 -delete
|
||||
"""
|
||||
|
||||
|
||||
class TestParse:
|
||||
def test_a_system_crontab_names_the_user(self):
|
||||
jobs = parse_crontab("/etc/crontab", SYSTEM, system=True)
|
||||
first = jobs[0]
|
||||
assert (first.schedule, first.run_as) == ("17 * * * *", "root")
|
||||
assert first.command == "cd / && run-parts --report /etc/cron.hourly"
|
||||
assert first.env == {"SHELL": "/bin/sh", "PATH": "/usr/local/sbin:/usr/bin:/bin"}
|
||||
assert first.key.startswith("cron:/etc/crontab:")
|
||||
|
||||
def test_an_alias_and_a_hash_inside_the_command(self):
|
||||
reboot = parse_crontab("/etc/crontab", SYSTEM, system=True)[1]
|
||||
assert reboot.schedule == "@reboot"
|
||||
assert reboot.command == "/usr/local/bin/warm-cache # keeps the # in the command"
|
||||
|
||||
def test_a_line_switched_off_is_a_job_that_is_off(self):
|
||||
jobs = parse_crontab("/etc/crontab", SYSTEM, system=True)
|
||||
off = jobs[2]
|
||||
assert not off.enabled
|
||||
assert off.run_as == "backup"
|
||||
|
||||
def test_equal_lines_get_their_own_keys_and_switching_off_keeps_the_key(self):
|
||||
jobs = parse_crontab("/etc/crontab", SYSTEM, system=True)
|
||||
assert jobs[3].key == jobs[2].key + ":2"
|
||||
assert len(jobs) == 4 # the garbage line is left out
|
||||
|
||||
def test_a_user_crontab_runs_as_its_user(self):
|
||||
jobs = parse_crontab("/var/spool/cron/crontabs/www", USER, system=False, user="www-data")
|
||||
assert jobs[0].run_as == "www-data"
|
||||
assert jobs[0].env == {"MAILTO": "ops@example.org"}
|
||||
|
||||
def test_an_applications_job_is_keyed_by_its_name(self):
|
||||
job = parse_crontab("/x", USER, system=False, user="root")[1]
|
||||
assert (job.key, job.name, job.description) == (
|
||||
"job:netork-cleanup",
|
||||
"netork-cleanup",
|
||||
"tidy the cache",
|
||||
)
|
||||
|
||||
def test_an_anacrontab(self):
|
||||
text = "SHELL=/bin/sh\n1\t5\tcron.daily\trun-parts --report /etc/cron.daily\n@monthly 15 cron.monthly run-parts /etc/cron.monthly\nbad line here\n"
|
||||
jobs = parse_anacrontab(text)
|
||||
assert [(j.key, j.schedule, j.run_as) for j in jobs] == [
|
||||
("anacron:cron.daily", "1", "root"),
|
||||
("anacron:cron.monthly", "@monthly", "root"),
|
||||
]
|
||||
assert jobs[0].command == "run-parts --report /etc/cron.daily"
|
||||
|
||||
|
||||
class TestEdit:
|
||||
def test_switching_off_and_on_again_gives_the_text_back(self):
|
||||
jobs = parse_crontab("/etc/crontab", SYSTEM, system=True)
|
||||
off = switched(SYSTEM, jobs[0], enabled=False)
|
||||
assert DISABLED_MARKER + "17 *\t* * *\troot" in off
|
||||
again = parse_crontab("/etc/crontab", off, system=True)
|
||||
assert again[0].key == jobs[0].key and not again[0].enabled
|
||||
assert switched(off, again[0], enabled=True) == SYSTEM.replace(
|
||||
"17 *\t* * *\troot\tcd", "17 *\t* * *\troot\tcd"
|
||||
)
|
||||
|
||||
def test_an_applications_job_is_added_replaced_and_removed(self):
|
||||
lines = marked_lines("netork-x", "a job", "0 1 * * * /bin/true", enabled=True)
|
||||
added = with_job("A=1\n", [], "netork-x", lines)
|
||||
assert added == "A=1\n# napalm-scheduled-job: netork-x a job\n0 1 * * * /bin/true\n"
|
||||
jobs = parse_crontab("/x", added, system=False, user="root")
|
||||
changed = with_job(
|
||||
added,
|
||||
jobs,
|
||||
"netork-x",
|
||||
marked_lines("netork-x", "", "0 2 * * * /bin/true", enabled=False),
|
||||
)
|
||||
assert (
|
||||
changed
|
||||
== "A=1\n# napalm-scheduled-job: netork-x\n#napalm-disabled: 0 2 * * * /bin/true\n"
|
||||
)
|
||||
jobs = parse_crontab("/x", changed, system=False, user="root")
|
||||
assert without_job(changed, jobs, "netork-x") == "A=1\n"
|
||||
|
||||
def test_adding_to_a_text_without_a_last_newline(self):
|
||||
assert with_job("A=1", [], "n", ["# m", "x"]) == "A=1\n# m\nx\n"
|
||||
|
||||
def test_removing_the_only_job_leaves_nothing(self):
|
||||
text = "# napalm-scheduled-job: n\n0 1 * * * x\n"
|
||||
jobs = parse_crontab("/x", text, system=False, user="root")
|
||||
assert without_job(text, jobs, "n") == ""
|
||||
|
||||
|
||||
class TestRunScript:
|
||||
def _run(self, script: str) -> str:
|
||||
return subprocess.run(
|
||||
["sh", "-c", script], capture_output=True, text=True, check=True
|
||||
).stdout
|
||||
|
||||
def test_the_crontabs_variables_are_set(self):
|
||||
assert (
|
||||
self._run(run_script('echo "$GREETING"', {"GREETING": "hi there", "MAILTO": "x"}))
|
||||
== "hi there\n"
|
||||
)
|
||||
|
||||
def test_a_percent_starts_stdin_and_further_ones_are_newlines(self):
|
||||
assert self._run(run_script("cat%one%two", {})) == "one\ntwo\n"
|
||||
|
||||
def test_an_escaped_percent_is_a_percent(self):
|
||||
assert self._run(run_script("date -d @0 -u +\\%Y", {})) == "1970\n"
|
||||
|
||||
@pytest.mark.parametrize("command", ["echo plain", "echo 'quoted'"])
|
||||
def test_without_percent_the_command_is_as_it_is(self, command):
|
||||
assert run_script(command, {}) == command
|
||||
@@ -14,7 +14,7 @@ from typing import Any
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import DhcpServerMixin, FirewallDriver, ResidentialGatewayDriver
|
||||
from napalm_device_types.dhcp import normalize_mac
|
||||
from napalm_device_types.dhcp import DhcpApplyError, normalize_mac
|
||||
from napalm_device_types.models import DhcpReservationDict
|
||||
|
||||
|
||||
@@ -209,6 +209,95 @@ class TestApplyReservationSet:
|
||||
assert lines == ["[commit] no changes"]
|
||||
|
||||
|
||||
class _RefusingDhcpServer(_FakeDhcpServer):
|
||||
"""A server that refuses the reservations for the IPs in *refused*, as Kea does
|
||||
for one whose subnet it does not manage."""
|
||||
|
||||
def __init__(self, live: list[DhcpReservationDict] | None = None, *, refused: set[str]) -> None:
|
||||
super().__init__(live)
|
||||
self.refused = refused
|
||||
|
||||
def apply_dhcp_reservation(
|
||||
self, reservation: DhcpReservationDict, *, uuid: str | None = None
|
||||
) -> dict[str, Any]:
|
||||
if reservation["ip"] in self.refused:
|
||||
raise ValueError(f"No Kea-managed subnet contains {reservation['ip']}")
|
||||
return super().apply_dhcp_reservation(reservation, uuid=uuid)
|
||||
|
||||
|
||||
def _three() -> list[DhcpReservationDict]:
|
||||
return [
|
||||
_reservation(mac="aa:bb:cc:dd:ee:01", ip="10.10.20.50"),
|
||||
_reservation(mac="aa:bb:cc:dd:ee:02", ip="10.10.99.5", subnet=""),
|
||||
_reservation(mac="aa:bb:cc:dd:ee:03", ip="10.10.20.52"),
|
||||
]
|
||||
|
||||
|
||||
class TestApplyWithARefusedReservation:
|
||||
"""One reservation the server refuses must not hold back the others, and what was
|
||||
written must take effect: a server that saved a write and was never committed shows
|
||||
it as configured, so the next apply finds nothing to do and never commits it."""
|
||||
|
||||
def test_the_others_are_still_written(self):
|
||||
driver = _RefusingDhcpServer(refused={"10.10.99.5"})
|
||||
|
||||
with pytest.raises(DhcpApplyError):
|
||||
list(driver.apply_dhcp_reservationset(_three()))
|
||||
|
||||
assert [entry["reservation"]["ip"] for entry in driver.applied] == [
|
||||
"10.10.20.50",
|
||||
"10.10.20.52",
|
||||
]
|
||||
|
||||
def test_what_was_written_is_committed(self):
|
||||
driver = _RefusingDhcpServer(refused={"10.10.99.5"})
|
||||
|
||||
with pytest.raises(DhcpApplyError):
|
||||
list(driver.apply_dhcp_reservationset(_three()))
|
||||
|
||||
assert driver.commits == 1
|
||||
|
||||
def test_a_refused_set_still_commits(self):
|
||||
# An earlier apply that stopped before its commit left saved writes behind; a
|
||||
# run whose every write is refused commits too, so those take effect.
|
||||
driver = _RefusingDhcpServer(refused={"10.10.99.5"})
|
||||
|
||||
with pytest.raises(DhcpApplyError):
|
||||
list(driver.apply_dhcp_reservationset([_three()[1]]))
|
||||
|
||||
assert driver.commits == 1
|
||||
|
||||
def test_progress_names_the_refused_one_and_why(self):
|
||||
driver = _RefusingDhcpServer(refused={"10.10.99.5"})
|
||||
lines: list[str] = []
|
||||
|
||||
with pytest.raises(DhcpApplyError) as raised:
|
||||
for line in driver.apply_dhcp_reservationset(_three()):
|
||||
lines.append(line)
|
||||
|
||||
failed = [line for line in lines if line.startswith("[failed]")]
|
||||
assert failed == [
|
||||
"[failed] aa:bb:cc:dd:ee:02 -> 10.10.99.5: No Kea-managed subnet contains 10.10.99.5"
|
||||
]
|
||||
assert lines[-1] == "[commit] applied 2 add(s), 0 update(s)"
|
||||
assert "1 of 3" in str(raised.value)
|
||||
|
||||
def test_a_refused_update_is_reported_like_an_add(self):
|
||||
driver = _RefusingDhcpServer(
|
||||
[_reservation(uuid="u1", ip="10.10.20.9")], refused={"10.10.20.50"}
|
||||
)
|
||||
|
||||
with pytest.raises(DhcpApplyError, match="1 of 1"):
|
||||
list(driver.apply_dhcp_reservationset([_reservation()]))
|
||||
|
||||
assert driver.applied == []
|
||||
assert driver.commits == 1
|
||||
|
||||
def test_the_error_is_a_runtime_error(self):
|
||||
# Callers that catch RuntimeError for a failed apply keep working.
|
||||
assert issubclass(DhcpApplyError, RuntimeError)
|
||||
|
||||
|
||||
class TestKeysThatCannotNameOneReservation:
|
||||
def test_a_live_reservation_without_mac_is_never_matched(self):
|
||||
driver = _FakeDhcpServer([_reservation(uuid="b", mac="")])
|
||||
|
||||
@@ -11,6 +11,7 @@ import pytest
|
||||
|
||||
from napalm_device_types import (
|
||||
AccessPointDriver,
|
||||
BackupServerDriver,
|
||||
EnsurePackageMixin,
|
||||
FirewallDriver,
|
||||
HypervisorDriver,
|
||||
@@ -28,7 +29,7 @@ from napalm_device_types import (
|
||||
)
|
||||
|
||||
ROLE_BASES = [
|
||||
AccessPointDriver, FirewallDriver, HypervisorDriver, ManagementControllerDriver, MediaDriver,
|
||||
AccessPointDriver, BackupServerDriver, FirewallDriver, HypervisorDriver, ManagementControllerDriver, MediaDriver,
|
||||
OSDriver, PhoneDriver, ResidentialGatewayDriver, StorageDriver, SwitchDriver,
|
||||
] # fmt: skip
|
||||
|
||||
|
||||
@@ -13,6 +13,8 @@ from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import sqlite3
|
||||
import subprocess
|
||||
|
||||
import pytest
|
||||
@@ -25,6 +27,12 @@ from napalm_device_types.host_status import (
|
||||
kernel_reboot_pending,
|
||||
parse_host_status,
|
||||
)
|
||||
from napalm_device_types.package_managers import (
|
||||
APK_INSTALLED,
|
||||
DNF_HISTORY,
|
||||
OPKG_STATUS,
|
||||
PACKAGE_MANAGERS_COMMAND,
|
||||
)
|
||||
|
||||
|
||||
def _wire(
|
||||
@@ -570,3 +578,371 @@ class TestFilesystems:
|
||||
|
||||
mountpoints = [fs["mountpoint"] for fs in parse_host_status(out.stdout)["filesystems"]]
|
||||
assert "/" in mountpoints
|
||||
|
||||
|
||||
def _other(*sections: tuple[str, tuple[str, ...]], running: tuple[str, ...] = ()) -> str:
|
||||
"""A report from a host without dpkg: the given sections before ``[timers]``."""
|
||||
lines = ["[pm-running]", *running]
|
||||
for name, body in sections:
|
||||
lines += [f"[{name}]", *body]
|
||||
return _wire().replace("[timers]", "\n".join([*lines, "[timers]"]))
|
||||
|
||||
|
||||
class TestInterruptedRpm:
|
||||
"""dnf and yum hosts: a transaction rpm was cut off in leaves the old and the
|
||||
new version of a package both installed (``dnf check --duplicates``,
|
||||
``package-cleanup --dupes``), and dnf's history records a transaction that
|
||||
never ended. netOrk only reports it: there is no one repair that is safe on
|
||||
every host (NetOrk/netork#851, decided 2026-11-07).
|
||||
|
||||
The outputs are shaped after the documented formats -- ``rpm -qa --qf``,
|
||||
libdnf's ``trans`` table (``state`` 0 = UNKNOWN until ``finish()`` sets DONE
|
||||
1 or ERROR 2), yum's ``transaction-all.*`` journal that
|
||||
``yum-complete-transaction`` reads -- and still have to be checked against a
|
||||
real Oracle Linux 9 host.
|
||||
"""
|
||||
|
||||
INSTALLONLY = ("kernel-core", "kernel-modules", "kernel-uek-core")
|
||||
#: An upgrade of glibc and openssl-libs cut off after the new versions were in.
|
||||
CUT_OFF = ("glibc x86_64", "kernel-uek-core x86_64", "openssl-libs x86_64")
|
||||
|
||||
def _rpm(self, dupes=(), history=("42 1",), journal=(), frontend="dnf", running=()):
|
||||
return _other(
|
||||
("rpm-frontend", (frontend,)),
|
||||
("rpm-installonly", self.INSTALLONLY),
|
||||
("rpm-dupes", dupes),
|
||||
("yum-journal", journal),
|
||||
("dnf-history", history),
|
||||
running=running,
|
||||
)
|
||||
|
||||
def test_a_clean_rpm_database_is_not_interrupted(self):
|
||||
status = parse_host_status(self._rpm())
|
||||
|
||||
assert status["package_manager_interrupted"] is False
|
||||
assert status["interrupted_packages"] == []
|
||||
assert status["package_manager"] == "dnf"
|
||||
|
||||
def test_two_versions_of_a_package_left_by_a_cut_off_transaction(self):
|
||||
status = parse_host_status(self._rpm(dupes=self.CUT_OFF))
|
||||
|
||||
assert status["package_manager_interrupted"] is True
|
||||
assert status["interrupted_packages"] == [
|
||||
{"name": "glibc", "state": "duplicate"},
|
||||
{"name": "openssl-libs", "state": "duplicate"},
|
||||
]
|
||||
|
||||
def test_kernels_are_installed_side_by_side_on_purpose(self):
|
||||
"""dnf's installonly packages: several versions are the rule, not a fault."""
|
||||
status = parse_host_status(self._rpm(dupes=("kernel-uek-core x86_64",)))
|
||||
|
||||
assert status["package_manager_interrupted"] is False
|
||||
|
||||
def test_a_package_duplicated_in_two_arches_is_named_once(self):
|
||||
status = parse_host_status(self._rpm(dupes=("glibc i686", "glibc x86_64")))
|
||||
|
||||
assert status["interrupted_packages"] == [{"name": "glibc", "state": "duplicate"}]
|
||||
|
||||
def test_the_last_transaction_never_finished(self):
|
||||
status = parse_host_status(self._rpm(history=("43 0",)))
|
||||
|
||||
assert status["package_manager_interrupted"] is True
|
||||
assert status["interrupted_packages"] == []
|
||||
|
||||
@pytest.mark.parametrize("history", [("44 1",), ("45 2",), ()])
|
||||
def test_a_finished_or_unreadable_history_says_nothing(self, history):
|
||||
"""A failed transaction (ERROR) did end; an unreadable history is no answer."""
|
||||
assert parse_host_status(self._rpm(history=history))["package_manager_interrupted"] is False
|
||||
|
||||
def test_yum_left_its_transaction_journal(self):
|
||||
journal = ("transaction-all.2026-11-01.10:12.33", "transaction-done.2026-11-01.10:12.33")
|
||||
status = parse_host_status(self._rpm(frontend="yum", history=(), journal=journal))
|
||||
|
||||
assert status["package_manager_interrupted"] is True
|
||||
assert status["package_manager"] == "yum"
|
||||
|
||||
def test_a_journal_yum_gave_up_on_is_none(self):
|
||||
"""yum-complete-transaction skips a journal it renamed ``.disabled``."""
|
||||
journal = ("transaction-all.2026-11-01.10:12.33.disabled",)
|
||||
status = parse_host_status(self._rpm(frontend="yum", history=(), journal=journal))
|
||||
|
||||
assert status["package_manager_interrupted"] is False
|
||||
|
||||
@pytest.mark.parametrize("name", ["rpm", "dnf", "dnf5", "yum", "dnf-automatic", "packagekitd"])
|
||||
def test_a_running_transaction_makes_it_unknown(self, name):
|
||||
status = parse_host_status(self._rpm(dupes=self.CUT_OFF, running=(name,)))
|
||||
|
||||
assert status["package_manager_interrupted"] is None
|
||||
assert status["interrupted_packages"] == []
|
||||
assert status["package_manager"] == "dnf"
|
||||
|
||||
def test_another_managers_run_does_not(self):
|
||||
status = parse_host_status(self._rpm(dupes=self.CUT_OFF, running=("opkg",)))
|
||||
|
||||
assert status["package_manager_interrupted"] is True
|
||||
|
||||
def test_the_command_asks_rpm_only_where_dnf_or_yum_exists(self):
|
||||
assert "command -v rpm" in HOST_STATUS_COMMAND
|
||||
assert "rpm -qa --qf" in HOST_STATUS_COMMAND
|
||||
assert f"file:{DNF_HISTORY}?mode=ro" in HOST_STATUS_COMMAND
|
||||
|
||||
def test_it_never_repairs(self):
|
||||
for word in ("distro-sync", "--cleandupes", "remove", "yum-complete-transaction", "rpm -e"):
|
||||
assert word not in HOST_STATUS_COMMAND
|
||||
|
||||
|
||||
class TestInterruptedApk:
|
||||
"""Alpine and OpenWrt 25.12: apk marks a package whose files or scripts failed
|
||||
in ``/lib/apk/db/installed`` (``f:`` with ``f`` broken files, ``s`` a broken
|
||||
script; ``x`` xattrs and ``S`` the checksum kind are no fault), and ``apk fix``
|
||||
with no arguments is what picks exactly those (apk-tools ``app_fix.c``).
|
||||
netOrk only reports it (NetOrk/netork#851). Shaped after apk-tools'
|
||||
``database.c``; to be checked against a real OpenWrt 25.12 host."""
|
||||
|
||||
def _apk(self, audit=(), running=()):
|
||||
return _other(("apk-audit", audit), running=running)
|
||||
|
||||
def test_a_clean_database_is_not_interrupted(self):
|
||||
status = parse_host_status(self._apk())
|
||||
|
||||
assert status["package_manager_interrupted"] is False
|
||||
assert status["package_manager"] == "apk"
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("flags", "states"),
|
||||
[
|
||||
("s", ["broken-script"]),
|
||||
("f", ["broken-files"]),
|
||||
("fsS", ["broken-files", "broken-script"]),
|
||||
],
|
||||
)
|
||||
def test_a_package_apk_marked_broken(self, flags, states):
|
||||
status = parse_host_status(self._apk(audit=(f"{flags} busybox",)))
|
||||
|
||||
assert status["package_manager_interrupted"] is True
|
||||
assert status["interrupted_packages"] == [
|
||||
{"name": "busybox", "state": state} for state in states
|
||||
]
|
||||
|
||||
def test_a_running_apk_makes_it_unknown(self):
|
||||
status = parse_host_status(self._apk(audit=("s busybox",), running=("apk",)))
|
||||
|
||||
assert status["package_manager_interrupted"] is None
|
||||
|
||||
def test_the_command_reads_the_installed_database(self):
|
||||
assert "/lib/apk/db/installed" in HOST_STATUS_COMMAND
|
||||
assert "apk fix" not in HOST_STATUS_COMMAND
|
||||
|
||||
|
||||
class TestInterruptedOpkg:
|
||||
"""OpenWrt before 25.12: opkg writes ``Status: <want> <flags> <state>`` per
|
||||
package to ``/usr/lib/opkg/status``; a package whose run was cut off or whose
|
||||
postinst failed stays ``unpacked``, and ``opkg configure`` finishes every
|
||||
unpacked package (opkg-lede ``opkg_cmd.c``, ``pkg.c``). netOrk's update run
|
||||
does that first (NetOrk/netork#851). To be checked against a real router."""
|
||||
|
||||
def _opkg(self, audit=(), running=()):
|
||||
return _other(("opkg-audit", audit), running=running)
|
||||
|
||||
def test_a_clean_status_file_is_not_interrupted(self):
|
||||
status = parse_host_status(self._opkg())
|
||||
|
||||
assert status["package_manager_interrupted"] is False
|
||||
assert status["package_manager"] == "opkg"
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("line", "state"),
|
||||
[
|
||||
("user unpacked luci-app-sqm", "unpacked"),
|
||||
("ok half-configured luci-app-sqm", "half-configured"),
|
||||
("ok half-installed luci-app-sqm", "half-installed"),
|
||||
("reinstreq,user installed luci-app-sqm", "reinstall-required"),
|
||||
],
|
||||
)
|
||||
def test_every_state_an_interrupted_run_leaves(self, line, state):
|
||||
status = parse_host_status(self._opkg(audit=(line,)))
|
||||
|
||||
assert status["package_manager_interrupted"] is True
|
||||
assert status["interrupted_packages"] == [{"name": "luci-app-sqm", "state": state}]
|
||||
|
||||
def test_a_running_opkg_makes_it_unknown(self):
|
||||
status = parse_host_status(self._opkg(audit=("ok unpacked x",), running=("opkg",)))
|
||||
|
||||
assert status["package_manager_interrupted"] is None
|
||||
|
||||
def test_the_command_reads_the_status_file_and_never_configures(self):
|
||||
assert "/usr/lib/opkg/status" in HOST_STATUS_COMMAND
|
||||
assert "opkg configure" not in HOST_STATUS_COMMAND
|
||||
|
||||
|
||||
class TestPackageManager:
|
||||
def test_dpkg_says_so(self):
|
||||
assert parse_host_status(_wire(dpkg=()))["package_manager"] == "dpkg"
|
||||
|
||||
def test_a_busy_dpkg_still_says_which(self):
|
||||
status = parse_host_status(_wire(dpkg=(), busy=True))
|
||||
|
||||
assert (status["package_manager"], status["package_manager_interrupted"]) == (
|
||||
"dpkg",
|
||||
None,
|
||||
)
|
||||
|
||||
def test_no_package_manager_read_is_none(self):
|
||||
status = parse_host_status(_wire())
|
||||
|
||||
assert status["package_manager"] is None
|
||||
assert status["package_manager_interrupted"] is None
|
||||
|
||||
|
||||
#: An rpm that answers the two queries the command asks, as on an EL host whose
|
||||
#: glibc upgrade was cut off; and a dnf that only has to exist.
|
||||
_RPM_STUB = """#!/bin/sh
|
||||
case "$*" in
|
||||
*--whatprovides*) printf '%s\\n' kernel-core kernel-core 'no package provides kernel-PAE' ;;
|
||||
-qa*) printf '%s\\n' 'glibc x86_64' 'kernel-core x86_64' 'gpg-pubkey (none)' \\
|
||||
'glibc x86_64' 'kernel-core x86_64' 'gpg-pubkey (none)' 'bash x86_64' ;;
|
||||
esac
|
||||
"""
|
||||
|
||||
#: /lib/apk/db/installed shaped after apk-tools' database.c: busybox's script failed.
|
||||
_APK_DB = """C:Q1abc=
|
||||
P:alpine-baselayout
|
||||
V:3.6.8-r1
|
||||
A:x86_64
|
||||
F:etc
|
||||
R:fstab
|
||||
|
||||
C:Q1def=
|
||||
P:busybox
|
||||
V:1.37.0-r12
|
||||
A:x86_64
|
||||
f:s
|
||||
F:bin
|
||||
R:busybox
|
||||
|
||||
C:Q1ghi=
|
||||
P:musl
|
||||
V:1.2.5-r9
|
||||
f:S
|
||||
"""
|
||||
|
||||
#: /usr/lib/opkg/status as opkg writes it: luci-app-sqm's postinst never ran.
|
||||
_OPKG_STATUS = """Package: libc
|
||||
Version: 1.2.4-4
|
||||
Depends: libgcc1
|
||||
Status: install hold installed
|
||||
Architecture: mipsel_24kc
|
||||
Installed-Time: 1717000000
|
||||
|
||||
Package: luci-app-sqm
|
||||
Version: 1.5.3-r1
|
||||
Depends: libc, sqm-scripts
|
||||
Status: install user unpacked
|
||||
Architecture: all
|
||||
Installed-Time: 1730000000
|
||||
"""
|
||||
|
||||
|
||||
def _stubs(tmp_path, *names: str, rpm: bool = False) -> dict:
|
||||
bin_dir = tmp_path / "bin"
|
||||
bin_dir.mkdir()
|
||||
for name in names:
|
||||
stub = bin_dir / name
|
||||
stub.write_text(_RPM_STUB if name == "rpm" else "#!/bin/sh\nexit 0\n")
|
||||
stub.chmod(0o755)
|
||||
return {**os.environ, "PATH": f"{bin_dir}:/usr/bin:/bin"}
|
||||
|
||||
|
||||
def _run(command: str, env: dict, shell: tuple[str, ...]) -> str:
|
||||
report = f"printf '%s%s\\n' HSTAT_ BEGIN; {command} printf '%s%s\\n' HSTAT_ END"
|
||||
program = shutil.which(shell[0]) or shell[0]
|
||||
return subprocess.run(
|
||||
[program, *shell[1:], report], capture_output=True, text=True, env=env, timeout=30
|
||||
).stdout
|
||||
|
||||
|
||||
_SHELLS = [("sh", "-c")] + ([("busybox", "sh", "-c")] if shutil.which("busybox") else [])
|
||||
|
||||
|
||||
def _skip_busybox_rpm(shell: tuple[str, ...]) -> None:
|
||||
"""A BusyBox built with its own ``rpm`` applet runs that before any stand-in."""
|
||||
if shell[0] == "busybox":
|
||||
applets = subprocess.run(["busybox", "--list"], capture_output=True, text=True).stdout
|
||||
if "rpm" in applets.split():
|
||||
pytest.skip("this busybox has an rpm applet of its own")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("shell", _SHELLS)
|
||||
class TestThePackageManagerCommandOnARealShell:
|
||||
"""The command part run for real, with stand-in programs and files."""
|
||||
|
||||
def test_rpm_duplicates_without_the_kernel_and_the_keys(self, tmp_path, shell):
|
||||
_skip_busybox_rpm(shell)
|
||||
env = _stubs(tmp_path, "rpm", "dnf")
|
||||
out = _run(PACKAGE_MANAGERS_COMMAND, env, shell)
|
||||
|
||||
status = parse_host_status(out)
|
||||
|
||||
assert status["package_manager"] == "dnf"
|
||||
assert status["interrupted_packages"] == [{"name": "glibc", "state": "duplicate"}]
|
||||
|
||||
def test_dnfs_last_transaction_that_never_ended(self, tmp_path, shell):
|
||||
_skip_busybox_rpm(shell)
|
||||
history = tmp_path / "history.sqlite"
|
||||
with sqlite3.connect(history) as db:
|
||||
db.execute("CREATE TABLE trans (id INTEGER PRIMARY KEY, state INTEGER NOT NULL)")
|
||||
db.executemany("INSERT INTO trans VALUES (?, ?)", [(41, 1), (42, 0)])
|
||||
env = _stubs(tmp_path, "rpm", "dnf")
|
||||
out = _run(PACKAGE_MANAGERS_COMMAND.replace(DNF_HISTORY, str(history)), env, shell)
|
||||
|
||||
assert "[dnf-history]\n42 0\n" in out
|
||||
|
||||
def test_apk_names_the_package_it_marked(self, tmp_path, shell):
|
||||
installed = tmp_path / "installed"
|
||||
installed.write_text(_APK_DB)
|
||||
env = _stubs(tmp_path, "apk")
|
||||
out = _run(PACKAGE_MANAGERS_COMMAND.replace(APK_INSTALLED, str(installed)), env, shell)
|
||||
|
||||
status = parse_host_status(out)
|
||||
|
||||
assert status["package_manager"] == "apk"
|
||||
assert status["interrupted_packages"] == [{"name": "busybox", "state": "broken-script"}]
|
||||
|
||||
def test_opkg_names_the_unpacked_package(self, tmp_path, shell):
|
||||
status_file = tmp_path / "status"
|
||||
status_file.write_text(_OPKG_STATUS)
|
||||
env = _stubs(tmp_path, "opkg")
|
||||
out = _run(PACKAGE_MANAGERS_COMMAND.replace(OPKG_STATUS, str(status_file)), env, shell)
|
||||
|
||||
status = parse_host_status(out)
|
||||
|
||||
assert status["package_manager"] == "opkg"
|
||||
assert status["interrupted_packages"] == [{"name": "luci-app-sqm", "state": "unpacked"}]
|
||||
|
||||
def test_a_host_with_none_of_them_reads_none(self, tmp_path, shell):
|
||||
env = _stubs(tmp_path)
|
||||
env["PATH"] = str(tmp_path / "bin")
|
||||
out = _run(PACKAGE_MANAGERS_COMMAND, env, shell)
|
||||
|
||||
assert parse_host_status(out)["package_manager"] is None
|
||||
|
||||
def test_the_running_processes_come_from_proc(self, tmp_path, shell):
|
||||
"""The test runner's own process is there under its own name; no manager is."""
|
||||
out = _run(PACKAGE_MANAGERS_COMMAND, _stubs(tmp_path), shell)
|
||||
|
||||
assert "[pm-running]\n" in out
|
||||
|
||||
|
||||
class TestTheWholeCommand:
|
||||
def test_it_runs_on_busybox(self):
|
||||
"""OpenWrt's and Alpine's shell: still one intact report."""
|
||||
if not shutil.which("busybox"):
|
||||
pytest.skip("no busybox here")
|
||||
out = subprocess.run(
|
||||
["busybox", "sh", "-c", HOST_STATUS_COMMAND], capture_output=True, text=True, timeout=30
|
||||
).stdout
|
||||
|
||||
assert parse_host_status(out)["filesystems"]
|
||||
|
||||
def test_the_package_managers_are_part_of_it(self):
|
||||
assert PACKAGE_MANAGERS_COMMAND in HOST_STATUS_COMMAND
|
||||
@@ -23,6 +23,8 @@ from napalm_device_types.local_accounts import (
|
||||
from napalm_device_types.models import FixStepOutcomeDict
|
||||
|
||||
ALICE_KEY = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAlice alice@laptop"
|
||||
#: The key Alice had before, on the laptop she lost.
|
||||
OLD_KEY = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOld alice@lost-laptop"
|
||||
OWN_KEY = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQOwn own@desktop"
|
||||
HASH = "$6$salt$" + "x" * 86
|
||||
#: Shell syntax a command for root must not hold: each is one fixed command.
|
||||
@@ -61,6 +63,10 @@ class _Host:
|
||||
self.ran: list[tuple[str, bool]] = []
|
||||
self.staged = 0
|
||||
|
||||
def ran_words(self, first: str) -> list[str]:
|
||||
"""The words of the last command run whose first word is *first*."""
|
||||
return next(w for w in (shlex.split(c) for c, _ in reversed(self.ran)) if w[0] == first)
|
||||
|
||||
def is_admin(self, name: str) -> bool:
|
||||
rule = self.files.get(sudoers_rule_path(_Driver.SUDOERS_DIR, name))
|
||||
return name in self.other_rules or (self.includes_dropins and rule is not None)
|
||||
@@ -200,6 +206,11 @@ class _Host:
|
||||
self.accounts[args[0]].groups.remove(args[1])
|
||||
return CommandResult("", "", 0)
|
||||
|
||||
def _clearpw(self, args: list[str], privileged: bool) -> CommandResult:
|
||||
assert privileged
|
||||
self.accounts[args[0]].password = None
|
||||
return CommandResult("", "", 0)
|
||||
|
||||
|
||||
class _Driver(LocalAccountsMixin):
|
||||
ACCOUNT_SHELL = "/bin/sh"
|
||||
@@ -238,6 +249,9 @@ class _Driver(LocalAccountsMixin):
|
||||
def _account_leave_group(self, name: str, group: str, groups: list[str]) -> list[str]:
|
||||
return ["leavegroup", name, group]
|
||||
|
||||
def _account_clear_password(self, name: str) -> list[list[str]]:
|
||||
return [["clearpw", name]]
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def host() -> _Host:
|
||||
@@ -362,7 +376,9 @@ def test_a_new_account_gets_the_key_and_the_sudo_rule(host: _Host, driver: _Driv
|
||||
|
||||
|
||||
def test_every_command_root_runs_is_one_fixed_command(host: _Host, driver: _Driver) -> None:
|
||||
driver.add_local_account("alice", ALICE_KEY, password_hash=HASH, admin=True)
|
||||
driver.add_local_account("alice", OLD_KEY, password_hash=HASH, admin=True)
|
||||
driver.add_local_account("alice", ALICE_KEY, replaces=[OLD_KEY], clear_password=True)
|
||||
driver.remove_local_account_keys("alice", [ALICE_KEY])
|
||||
driver.lock_local_account("alice")
|
||||
|
||||
for command in _root_commands(host):
|
||||
@@ -690,3 +706,344 @@ def test_a_key_file_the_host_adds_is_read_too(host: _Host) -> None:
|
||||
[account] = _ExtraFile(host).get_local_accounts(["alice"])
|
||||
|
||||
assert account["authorized_keys"] == [OWN_KEY, ALICE_KEY]
|
||||
|
||||
|
||||
# -- replacing a key (NetOrk/netork#2013) ----------------------------------------------
|
||||
|
||||
|
||||
def _alice_keys(host: _Host) -> str:
|
||||
return host.files.get("/home/alice/.ssh/authorized_keys", "")
|
||||
|
||||
|
||||
def test_a_replaced_key_goes_and_the_persons_own_keys_stay(host: _Host, driver: _Driver) -> None:
|
||||
_add_alice(host, keys=f"{OLD_KEY}\n{OWN_KEY}\n")
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY, replaces=[OLD_KEY])
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert [s["step"] for s in result["steps"]] == ["account", "sudo", "keys", "old keys"]
|
||||
assert _alice_keys(host) == f"{OWN_KEY}\n{ALICE_KEY}\n"
|
||||
account = result["account"]
|
||||
assert account is not None and account["authorized_keys"] == [OWN_KEY, ALICE_KEY]
|
||||
|
||||
|
||||
def test_a_replaced_key_goes_from_every_key_file_with_any_options_or_comment(
|
||||
host: _Host, driver: _Driver
|
||||
) -> None:
|
||||
# The same key with options in front and another comment is the same key.
|
||||
blob = OLD_KEY.rsplit(" ", 1)[0]
|
||||
_add_alice(host, keys=f'from="10.0.0.0/8" {blob} renamed\n# the laptop\n')
|
||||
host.files["/home/alice/.ssh/authorized_keys2"] = f"{OWN_KEY}\n{blob}\n"
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY, replaces=[OLD_KEY])
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert _alice_keys(host) == f"# the laptop\n{ALICE_KEY}\n"
|
||||
assert host.files["/home/alice/.ssh/authorized_keys2"] == f"{OWN_KEY}\n"
|
||||
assert not [p for p in host.files if p.startswith("/tmp/")]
|
||||
|
||||
|
||||
def test_a_replaced_key_that_is_not_there_writes_nothing(host: _Host, driver: _Driver) -> None:
|
||||
_add_alice(host, keys=f"{ALICE_KEY}\n")
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY, replaces=[OLD_KEY])
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert host.staged == 0
|
||||
assert "none of the keys" in result["steps"][-1]["output"]
|
||||
|
||||
|
||||
def test_the_new_key_is_never_taken_for_a_replaced_one(host: _Host, driver: _Driver) -> None:
|
||||
_add_alice(host, keys=f"{OLD_KEY}\n")
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY, replaces=[OLD_KEY, ALICE_KEY])
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert _alice_keys(host) == f"{ALICE_KEY}\n"
|
||||
|
||||
|
||||
def test_without_replaces_no_key_goes(host: _Host, driver: _Driver) -> None:
|
||||
_add_alice(host, keys=f"{OLD_KEY}\n")
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY)
|
||||
|
||||
assert [s["step"] for s in result["steps"]] == ["account", "sudo", "keys"]
|
||||
assert _alice_keys(host) == f"{OLD_KEY}\n{ALICE_KEY}\n"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("replaces", [["not a key"], [OLD_KEY, ""], OLD_KEY])
|
||||
def test_a_replaced_key_that_is_none_is_refused(
|
||||
host: _Host, driver: _Driver, replaces: list[str]
|
||||
) -> None:
|
||||
with pytest.raises(ValueError, match="key"):
|
||||
driver.add_local_account("alice", ALICE_KEY, replaces=replaces)
|
||||
assert _root_commands(host) == []
|
||||
|
||||
|
||||
def test_the_old_key_goes_even_when_the_new_one_could_not_be_added(host: _Host) -> None:
|
||||
class _AddFails(_Driver):
|
||||
def _account_add_key(self, name: str, home: str, public_key: str) -> FixStepOutcomeDict:
|
||||
return {"ok": False, "output": "disk full"}
|
||||
|
||||
_add_alice(host, keys=f"{OLD_KEY}\n{OWN_KEY}\n")
|
||||
|
||||
result = _AddFails(host).add_local_account("alice", ALICE_KEY, replaces=[OLD_KEY])
|
||||
|
||||
assert not result["success"]
|
||||
assert next(s for s in result["steps"] if s["step"] == "old keys")["ok"]
|
||||
assert _alice_keys(host) == f"{OWN_KEY}\n"
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("failing", "ignored", "said"),
|
||||
[
|
||||
({"as cat"}, set(), "cannot read"),
|
||||
({"as install"}, set(), "authorized_keys"),
|
||||
(set(), {"as install"}, "still has"),
|
||||
],
|
||||
)
|
||||
def test_a_replaced_key_that_did_not_go_says_so(
|
||||
host: _Host, driver: _Driver, failing: set[str], ignored: set[str], said: str
|
||||
) -> None:
|
||||
_add_alice(host, keys=f"{OLD_KEY}\n{ALICE_KEY}\n")
|
||||
host.failing |= failing
|
||||
host.ignored |= ignored
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY, replaces=[OLD_KEY])
|
||||
|
||||
old = next(s for s in result["steps"] if s["step"] == "old keys")
|
||||
assert not old["ok"] and said in old["output"]
|
||||
assert not result["success"]
|
||||
|
||||
|
||||
# -- removing the password (NetOrk/netork#2013) ----------------------------------------
|
||||
|
||||
|
||||
def test_clearing_the_password_leaves_a_key_only_account_without_sudo(
|
||||
host: _Host, driver: _Driver
|
||||
) -> None:
|
||||
_add_alice(host, keys=f"{ALICE_KEY}\n")
|
||||
host.files["/etc/sudoers.d/80-account-alice"] = sudoers_rule("alice")
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY, clear_password=True)
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert [s["step"] for s in result["steps"]] == ["account", "sudo", "password", "keys"]
|
||||
a = host.accounts["alice"]
|
||||
assert a.password is None and not a.locked and a.groups == ["alice"]
|
||||
assert "/etc/sudoers.d/80-account-alice" not in host.files
|
||||
assert _alice_keys(host) == f"{ALICE_KEY}\n"
|
||||
|
||||
|
||||
def test_a_new_account_without_a_password_gets_it_cleared_too(host: _Host, driver: _Driver) -> None:
|
||||
result = driver.add_local_account("alice", ALICE_KEY, clear_password=True)
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert ["clearpw", "alice"] == host.ran_words("clearpw")
|
||||
|
||||
|
||||
def test_without_clear_password_the_password_stays(host: _Host, driver: _Driver) -> None:
|
||||
_add_alice(host)
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY)
|
||||
|
||||
assert "password" not in [s["step"] for s in result["steps"]]
|
||||
assert host.accounts["alice"].password == HASH
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"kwargs",
|
||||
[{"password_hash": HASH}, {"admin": True}],
|
||||
)
|
||||
def test_clearing_the_password_refuses_a_password_or_sudo(
|
||||
host: _Host, driver: _Driver, kwargs: dict[str, object]
|
||||
) -> None:
|
||||
# The sudo rule asks for the password: an account without one cannot use it.
|
||||
with pytest.raises(ValueError, match="password"):
|
||||
driver.add_local_account("alice", ALICE_KEY, clear_password=True, **kwargs) # type: ignore[arg-type]
|
||||
assert _root_commands(host) == []
|
||||
|
||||
|
||||
def test_a_password_that_cannot_be_cleared_fails_its_step_only(
|
||||
host: _Host, driver: _Driver
|
||||
) -> None:
|
||||
_add_alice(host)
|
||||
host.failing.add("clearpw")
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY, clear_password=True)
|
||||
|
||||
assert not result["success"]
|
||||
password = next(s for s in result["steps"] if s["step"] == "password")
|
||||
assert not password["ok"] and "clearpw: failed" in password["output"]
|
||||
# sudo and the key do not wait for it.
|
||||
assert next(s for s in result["steps"] if s["step"] == "sudo")["ok"]
|
||||
assert _alice_keys(host) == f"{ALICE_KEY}\n"
|
||||
|
||||
|
||||
def test_a_driver_that_cannot_clear_a_password_says_so(host: _Host) -> None:
|
||||
class _NoClear(_Driver):
|
||||
_account_clear_password = LocalAccountsMixin._account_clear_password
|
||||
|
||||
_add_alice(host)
|
||||
|
||||
result = _NoClear(host).add_local_account("alice", ALICE_KEY, clear_password=True)
|
||||
|
||||
password = next(s for s in result["steps"] if s["step"] == "password")
|
||||
assert not password["ok"] and "cannot remove" in password["output"]
|
||||
assert host.accounts["alice"].password == HASH
|
||||
|
||||
|
||||
# -- removing keys without a new one (NetOrk/netork#2013) -------------------------------
|
||||
|
||||
|
||||
def test_removing_keys_takes_only_those_and_leaves_the_account_open(
|
||||
host: _Host, driver: _Driver
|
||||
) -> None:
|
||||
_add_alice(host, keys=f"{ALICE_KEY}\n{OWN_KEY}\n")
|
||||
host.files["/etc/sudoers.d/80-account-alice"] = sudoers_rule("alice")
|
||||
|
||||
result = driver.remove_local_account_keys("alice", [ALICE_KEY])
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert [s["step"] for s in result["steps"]] == ["keys"]
|
||||
assert _alice_keys(host) == f"{OWN_KEY}\n"
|
||||
a = host.accounts["alice"]
|
||||
assert not a.locked and a.password == HASH and a.groups == ["alice", "wheel"]
|
||||
assert "/etc/sudoers.d/80-account-alice" in host.files
|
||||
account = result["account"]
|
||||
assert account is not None and account["authorized_keys"] == [OWN_KEY]
|
||||
|
||||
|
||||
def test_removing_the_last_key_leaves_an_empty_key_file(host: _Host, driver: _Driver) -> None:
|
||||
_add_alice(host, keys=f"{ALICE_KEY}\n")
|
||||
|
||||
result = driver.remove_local_account_keys("alice", [ALICE_KEY])
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert _alice_keys(host) == ""
|
||||
assert not host.accounts["alice"].locked
|
||||
|
||||
|
||||
def test_removing_keys_changes_nothing_as_root(host: _Host, driver: _Driver) -> None:
|
||||
# Root only asks whether a key file is there, and reads the account back
|
||||
# (its lock, sudo -l); the files are written as the person.
|
||||
_add_alice(host, keys=f"{ALICE_KEY}\n")
|
||||
|
||||
driver.remove_local_account_keys("alice", [ALICE_KEY])
|
||||
|
||||
for command in _root_commands(host):
|
||||
words = shlex.split(command)
|
||||
assert words[0] in {"test", "islocked", "sudo"}, command
|
||||
if words[0] == "sudo":
|
||||
assert words[:3] in (["sudo", "-n", "-u"], ["sudo", "-l", "-U"]), command
|
||||
|
||||
|
||||
def test_removing_keys_of_an_account_that_is_not_there_fails(driver: _Driver) -> None:
|
||||
result = driver.remove_local_account_keys("bob", [ALICE_KEY])
|
||||
|
||||
assert not result["success"]
|
||||
assert "no account" in result["output"]
|
||||
assert result["account"] is None
|
||||
|
||||
|
||||
@pytest.mark.parametrize("keys", [[], ["not a key"], ALICE_KEY])
|
||||
def test_removing_keys_refuses_what_is_no_list_of_keys(
|
||||
host: _Host, driver: _Driver, keys: list[str]
|
||||
) -> None:
|
||||
_add_alice(host, keys=f"{ALICE_KEY}\n")
|
||||
|
||||
with pytest.raises(ValueError, match="key"):
|
||||
driver.remove_local_account_keys("alice", keys)
|
||||
assert _alice_keys(host) == f"{ALICE_KEY}\n"
|
||||
|
||||
|
||||
@pytest.mark.parametrize(("name", "said"), [("netadmin", "logs in"), ("daemon", "system")])
|
||||
def test_removing_keys_never_touches_the_login_user_or_a_system_account(
|
||||
host: _Host, driver: _Driver, name: str, said: str
|
||||
) -> None:
|
||||
with pytest.raises(ValueError, match=said):
|
||||
driver.remove_local_account_keys(name, [ALICE_KEY])
|
||||
assert _root_commands(host) == []
|
||||
|
||||
|
||||
def test_a_driver_with_its_own_key_store_drops_keys_there(host: _Host) -> None:
|
||||
class _Store(_OwnKeyStore):
|
||||
def _account_drop_keys(self, name: str, home: str, keys: list[str]) -> FixStepOutcomeDict:
|
||||
gone = {k.split()[1] for k in keys}
|
||||
self.store[name] = [k for k in self.store.get(name, []) if k.split()[2] not in gone]
|
||||
return {"ok": True, "output": "dropped some"}
|
||||
|
||||
driver = _Store(host)
|
||||
driver.add_local_account("alice", OLD_KEY)
|
||||
driver.add_local_account("alice", ALICE_KEY, replaces=[OLD_KEY])
|
||||
assert driver.store == {"alice": [f"/home/alice {ALICE_KEY}"]}
|
||||
|
||||
removed = driver.remove_local_account_keys("alice", [ALICE_KEY])
|
||||
assert removed["success"], removed["output"]
|
||||
assert driver.store == {"alice": []}
|
||||
|
||||
|
||||
# -- removing keys and the password at once (NetOrk/netork#2033) ------------------------
|
||||
|
||||
|
||||
def test_removing_keys_with_clear_password_takes_the_password_and_sudo_too(
|
||||
host: _Host, driver: _Driver
|
||||
) -> None:
|
||||
"""A person who deleted their key: the password netOrk no longer knows goes too, and
|
||||
sudo, which asks for it. The account stays open, its other keys stay."""
|
||||
_add_alice(host, keys=f"{ALICE_KEY}\n{OWN_KEY}\n")
|
||||
host.files["/etc/sudoers.d/80-account-alice"] = sudoers_rule("alice")
|
||||
|
||||
result = driver.remove_local_account_keys("alice", [ALICE_KEY], clear_password=True)
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert [s["step"] for s in result["steps"]] == ["keys", "password", "sudo"]
|
||||
assert _alice_keys(host) == f"{OWN_KEY}\n"
|
||||
a = host.accounts["alice"]
|
||||
assert a.password is None and not a.locked and a.groups == ["alice"]
|
||||
assert "/etc/sudoers.d/80-account-alice" not in host.files
|
||||
account = result["account"]
|
||||
assert account is not None and not account["admin"] and not account["locked"]
|
||||
|
||||
|
||||
def test_removing_keys_with_clear_password_runs_each_step_when_one_failed(
|
||||
host: _Host, driver: _Driver
|
||||
) -> None:
|
||||
_add_alice(host, keys=f"{ALICE_KEY}\n")
|
||||
host.files["/etc/sudoers.d/80-account-alice"] = sudoers_rule("alice")
|
||||
host.failing.add("clearpw")
|
||||
|
||||
result = driver.remove_local_account_keys("alice", [ALICE_KEY], clear_password=True)
|
||||
|
||||
assert not result["success"]
|
||||
steps = {s["step"]: s for s in result["steps"]}
|
||||
assert steps["keys"]["ok"] and not steps["password"]["ok"]
|
||||
# sudo does not wait for the password: without a known password it has to go anyway.
|
||||
assert steps["sudo"]["ok"]
|
||||
assert "/etc/sudoers.d/80-account-alice" not in host.files
|
||||
assert _alice_keys(host) == ""
|
||||
|
||||
|
||||
def test_removing_keys_with_a_driver_that_cannot_clear_a_password_says_so(host: _Host) -> None:
|
||||
class _NoClear(_Driver):
|
||||
_account_clear_password = LocalAccountsMixin._account_clear_password
|
||||
|
||||
_add_alice(host, keys=f"{ALICE_KEY}\n")
|
||||
|
||||
result = _NoClear(host).remove_local_account_keys("alice", [ALICE_KEY], clear_password=True)
|
||||
|
||||
password = next(s for s in result["steps"] if s["step"] == "password")
|
||||
assert not password["ok"] and "cannot remove" in password["output"]
|
||||
assert not result["success"]
|
||||
|
||||
|
||||
def test_removing_keys_of_an_account_that_is_not_there_clears_nothing(driver: _Driver) -> None:
|
||||
result = driver.remove_local_account_keys("bob", [ALICE_KEY], clear_password=True)
|
||||
|
||||
assert not result["success"] and "no account" in result["output"]
|
||||
assert [s["step"] for s in result["steps"]] == ["keys"]
|
||||
|
||||
|
||||
def test_clear_password_is_a_keyword(driver: _Driver) -> None:
|
||||
with pytest.raises(TypeError):
|
||||
driver.remove_local_account_keys("alice", [ALICE_KEY], True) # type: ignore[misc]
|
||||
@@ -0,0 +1,158 @@
|
||||
"""Where a host's logs go beyond it: rsyslog's forwarding, its file inputs, and
|
||||
whether it gets the journal (NetOrk/netork#2053)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types.log_forwarding import journal_to_syslog, parse_rsyslog
|
||||
|
||||
DEBIAN = """\
|
||||
module(load="imuxsock")
|
||||
module(load="imklog")
|
||||
$FileOwner root
|
||||
$IncludeConfig /etc/rsyslog.d/*.conf
|
||||
*.*;auth,authpriv.none\t\t-/var/log/syslog
|
||||
auth,authpriv.*\t\t\t/var/log/auth.log
|
||||
*.emerg\t\t\t\t:omusrmsg:*
|
||||
"""
|
||||
|
||||
GRAYLOG = "*.* @graylog.example.net:1514;RSYSLOG_SyslogProtocol23Format\n"
|
||||
|
||||
|
||||
def _rules(text: str) -> list[dict]:
|
||||
return parse_rsyslog(text)["rules"]
|
||||
|
||||
|
||||
class TestRules:
|
||||
def test_a_debian_host_forwards_nothing(self):
|
||||
assert _rules(DEBIAN) == []
|
||||
|
||||
def test_netorks_own_graylog_rule_forwards_everything_over_udp(self):
|
||||
assert _rules(DEBIAN + GRAYLOG) == [
|
||||
{
|
||||
"protocol": "udp",
|
||||
"target": "graylog.example.net",
|
||||
"port": 1514,
|
||||
"selector": "*.*",
|
||||
"complete": True,
|
||||
"ruleset": None,
|
||||
},
|
||||
]
|
||||
|
||||
def test_two_ats_are_tcp_and_a_missing_port_stays_unknown(self):
|
||||
(rule,) = _rules("*.* @@logs.lan\n")
|
||||
assert (rule["protocol"], rule["target"], rule["port"]) == ("tcp", "logs.lan", None)
|
||||
|
||||
def test_an_ipv6_target_keeps_its_brackets_off(self):
|
||||
(rule,) = _rules("*.* @[2001:db8::5]:514\n")
|
||||
assert (rule["target"], rule["port"]) == ("2001:db8::5", 514)
|
||||
|
||||
def test_a_narrower_selector_is_partial_and_says_which(self):
|
||||
(rule,) = _rules("authpriv.* @10.0.0.5\n")
|
||||
assert (rule["selector"], rule["complete"]) == ("authpriv.*", False)
|
||||
|
||||
def test_excluding_a_facility_is_partial_too(self):
|
||||
(rule,) = _rules("*.*;mail.none @10.0.0.5\n")
|
||||
assert rule["complete"] is False
|
||||
|
||||
def test_an_ampersand_line_forwards_what_the_line_before_selected(self):
|
||||
rules = _rules("kern.* /var/log/kern.log\n& @10.0.0.5:514\n")
|
||||
assert [(r["selector"], r["target"]) for r in rules] == [("kern.*", "10.0.0.5")]
|
||||
|
||||
def test_relp_is_its_own_protocol(self):
|
||||
(rule,) = _rules("*.* :omrelp:10.0.0.6:2514\n")
|
||||
assert (rule["protocol"], rule["target"], rule["port"]) == ("relp", "10.0.0.6", 2514)
|
||||
|
||||
def test_an_action_over_several_lines_is_read_whole(self):
|
||||
text = '*.* action(type="omfwd"\n target="10.0.0.7" port="6514"\n protocol="tcp")\n'
|
||||
(rule,) = _rules(text)
|
||||
assert (rule["protocol"], rule["target"], rule["port"], rule["complete"]) == (
|
||||
"tcp",
|
||||
"10.0.0.7",
|
||||
6514,
|
||||
True,
|
||||
)
|
||||
|
||||
def test_an_action_on_its_own_takes_every_message(self):
|
||||
(rule,) = _rules('action(type="omfwd" Target="10.0.0.7")\n')
|
||||
assert (rule["selector"], rule["complete"], rule["protocol"]) == ("*.*", True, "udp")
|
||||
|
||||
def test_an_action_under_a_condition_is_partial_and_names_it(self):
|
||||
(rule,) = _rules(
|
||||
'if $programname == \'nginx\' then action(type="omfwd" target="10.0.0.8")\n'
|
||||
)
|
||||
assert rule["complete"] is False
|
||||
assert rule["selector"] == "$programname == 'nginx'"
|
||||
|
||||
def test_a_condition_whose_action_is_on_the_next_line_still_holds(self):
|
||||
(rule,) = _rules('if $msg contains "x" then\n action(type="omfwd" target="10.0.0.14")\n')
|
||||
assert (rule["selector"], rule["complete"]) == ('$msg contains "x"', False)
|
||||
|
||||
def test_a_block_under_a_condition_makes_each_of_its_actions_partial(self):
|
||||
text = 'if $syslogseverity <= 4 then {\n action(type="omfwd" target="a")\n action(type="omrelp" target="b")\n}\n*.* @c\n'
|
||||
rules = _rules(text)
|
||||
assert [(r["target"], r["complete"]) for r in rules] == [
|
||||
("a", False),
|
||||
("b", False),
|
||||
("c", True),
|
||||
]
|
||||
|
||||
def test_a_property_filter_is_a_partial_selector(self):
|
||||
(rule,) = _rules(':programname, isequal, "sshd" @10.0.0.9\n')
|
||||
assert rule["complete"] is False
|
||||
assert rule["selector"] == ':programname, isequal, "sshd"'
|
||||
|
||||
def test_an_action_in_a_ruleset_belongs_to_that_ruleset(self):
|
||||
text = 'ruleset(name="remote") {\n action(type="omfwd" target="10.0.0.10" port="514")\n}\n'
|
||||
(rule,) = _rules(text)
|
||||
assert (rule["ruleset"], rule["complete"]) == ("remote", True)
|
||||
|
||||
def test_comments_are_no_rules(self):
|
||||
assert _rules("# *.* @10.0.0.11\n*.* /var/log/all.log # @10.0.0.12\n") == []
|
||||
|
||||
def test_a_hash_inside_quotes_is_no_comment(self):
|
||||
(rule,) = _rules('action(type="omfwd" target="10.0.0.13" template="a#b")\n')
|
||||
assert rule["target"] == "10.0.0.13"
|
||||
|
||||
def test_a_local_action_is_no_forwarding(self):
|
||||
assert _rules('action(type="omfile" file="/var/log/x.log")\n*.* /var/log/y.log\n') == []
|
||||
|
||||
|
||||
class TestInputs:
|
||||
def test_an_imfile_input_names_its_file_and_ruleset(self):
|
||||
text = 'input(type="imfile" File="/var/log/nginx/*.log" Tag="nginx" Ruleset="remote")\n'
|
||||
assert parse_rsyslog(text)["file_inputs"] == [
|
||||
{"path": "/var/log/nginx/*.log", "ruleset": "remote"}
|
||||
]
|
||||
|
||||
def test_a_legacy_imfile_input_goes_to_the_default_ruleset(self):
|
||||
text = "$ModLoad imfile\n$InputFileName /var/log/app.log\n$InputFileTag app:\n$InputRunFileMonitor\n"
|
||||
assert parse_rsyslog(text)["file_inputs"] == [{"path": "/var/log/app.log", "ruleset": None}]
|
||||
|
||||
|
||||
class TestModules:
|
||||
def test_imjournal_reads_the_journal(self):
|
||||
assert parse_rsyslog('module(load="imjournal")\n')["imjournal"] is True
|
||||
|
||||
def test_legacy_modload_counts(self):
|
||||
modules = parse_rsyslog("$ModLoad imuxsock\n")
|
||||
assert (modules["imuxsock"], modules["imjournal"]) == (True, False)
|
||||
|
||||
|
||||
class TestJournalToSyslog:
|
||||
"""``ForwardToSyslog``: the last setting wins; with none, the shipped default
|
||||
the distribution prints commented out in its ``journald.conf``."""
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("lines", "expected"),
|
||||
[
|
||||
(["#ForwardToSyslog=yes"], True),
|
||||
(["#ForwardToSyslog=no"], False),
|
||||
(["#ForwardToSyslog=yes", "ForwardToSyslog=no"], False),
|
||||
(["ForwardToSyslog=no", "ForwardToSyslog = yes"], True),
|
||||
([], None),
|
||||
],
|
||||
)
|
||||
def test_the_last_setting_wins_else_the_shipped_default(self, lines, expected):
|
||||
assert journal_to_syslog(lines) is expected
|
||||
@@ -0,0 +1,394 @@
|
||||
"""get_log_sources: which logs a host writes, and who writes each of them.
|
||||
|
||||
Before anything can watch a host's logs -- a person in a log window, CrowdSec,
|
||||
a syslog forwarder -- something has to say which logs there are. Reading that
|
||||
is the same on every Linux host: journald names the units it holds entries for,
|
||||
``/proc/<pid>/fd`` shows the log files running processes have open, and
|
||||
``/proc/<pid>/cgroup`` says which systemd service or container a writer runs in.
|
||||
So all of it lives here once, and a driver only carries the command across
|
||||
(NetOrk/netork#2049).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types.log_sources import (
|
||||
LOG_SOURCES_COMMAND,
|
||||
LogSourcesMixin,
|
||||
LogSourcesUnavailable,
|
||||
parse_log_forwarding,
|
||||
parse_log_sources,
|
||||
)
|
||||
|
||||
CID = "4f1c" + "0" * 60
|
||||
OTHER = "9a2b" + "1" * 60
|
||||
JOURNALD = """nginx.service
|
||||
ssh.service
|
||||
postgresql@16-main.service
|
||||
session-3.scope
|
||||
init.scope
|
||||
"""
|
||||
OPEN = f"""/proc/610/fd/7 /var/log/syslog
|
||||
/proc/610/fd/8 /var/log/auth.log
|
||||
/proc/620/fd/9 /var/log/journal/0123abcd/system.journal
|
||||
/proc/900/fd/5 /var/log/nginx/access.log
|
||||
/proc/900/fd/6 /var/log/nginx/error.log
|
||||
/proc/901/fd/5 /var/log/nginx/access.log
|
||||
/proc/700/fd/4 /var/log/old.log (deleted)
|
||||
/proc/1100/fd/30 /var/lib/docker/containers/{CID}/{CID}-json.log
|
||||
/proc/1100/fd/31 /var/lib/docker/containers/{OTHER}/local-logs/container.log
|
||||
/proc/1400/fd/3 /usr/src/paperless/data/log/paperless.log
|
||||
/proc/1500/fd/3 /srv/app/log with space.log
|
||||
/proc/1500/fd/4 /srv/app/Local Storage/leveldb/000004.log
|
||||
/proc/1500/fd/5 /var/lib/rocksdb/012345.log
|
||||
/proc/300/fd/11 /var/log/nginx/access.log
|
||||
/proc/300/fd/12 /mnt/docroot/data/log/paperless.log
|
||||
/proc/300/fd/13 socket:[48213]
|
||||
/proc/300/fd/14 pipe:[48214]
|
||||
/proc/1600/fd/6 /var/log/host/syslog
|
||||
"""
|
||||
#: Each descriptor's ``flags`` from ``/proc/<pid>/fdinfo/<fd>``, octal: the access
|
||||
#: mode (``& 3``) is 1 or 2 for a writer -- 0102001 is O_WRONLY|O_APPEND|O_LARGEFILE
|
||||
#: -- and 0 for a reader (0100000, O_RDONLY|O_LARGEFILE). CrowdSec (300) reads
|
||||
#: nginx's log and a log on a mount; a promtail in a container (1600) reads the
|
||||
#: host's syslog as its own /var/log/host/syslog. 1500 opens its log O_RDWR.
|
||||
FLAGS = """/proc/610/fdinfo/7:flags:\t0102001
|
||||
/proc/610/fdinfo/8:flags:\t0102001
|
||||
/proc/620/fdinfo/9:flags:\t02100002
|
||||
/proc/900/fdinfo/5:flags:\t0102001
|
||||
/proc/900/fdinfo/6:flags:\t0102001
|
||||
/proc/901/fdinfo/5:flags:\t0102001
|
||||
/proc/700/fdinfo/4:flags:\t0102001
|
||||
/proc/1100/fdinfo/30:flags:\t0102001
|
||||
/proc/1100/fdinfo/31:flags:\t0102001
|
||||
/proc/1400/fdinfo/3:flags:\t0102001
|
||||
/proc/1500/fdinfo/3:flags:\t0100002
|
||||
/proc/1500/fdinfo/4:flags:\t0100002
|
||||
/proc/1500/fdinfo/5:flags:\t0100002
|
||||
/proc/300/fdinfo/11:flags:\t0100000
|
||||
/proc/300/fdinfo/12:flags:\t0100000
|
||||
/proc/300/fdinfo/13:flags:\t02004002
|
||||
/proc/300/fdinfo/14:flags:\t02004001
|
||||
/proc/1600/fdinfo/6:flags:\t0100000
|
||||
"""
|
||||
STAT = """/proc/610/fd/7 99000 1762250000
|
||||
/proc/610/fd/8 4100 1762250100
|
||||
/proc/900/fd/5 2048 1762250200
|
||||
/proc/900/fd/6 512 1762250300
|
||||
/proc/901/fd/5 2048 1762250200
|
||||
/proc/1100/fd/30 7000 1762250400
|
||||
/proc/1100/fd/31 300 1762250500
|
||||
/proc/1400/fd/3 0 1762250600
|
||||
/proc/300/fd/11 2048 1762250200
|
||||
/proc/300/fd/12 81920 1762250700
|
||||
"""
|
||||
CGROUPS = f"""610 0::/system.slice/rsyslog.service
|
||||
900 0::/system.slice/nginx.service
|
||||
901 0::/system.slice/nginx.service
|
||||
1100 0::/system.slice/docker.service
|
||||
1400 0::/system.slice/docker-{CID}.scope
|
||||
1500 0::/user.slice/user-1000.slice/session-3.scope
|
||||
300 0::/system.slice/crowdsec.service
|
||||
1600 0::/system.slice/docker-{OTHER}.scope
|
||||
"""
|
||||
COMM = """610 rsyslogd
|
||||
900 nginx
|
||||
901 nginx
|
||||
1100 dockerd
|
||||
1400 python3
|
||||
1500 app
|
||||
300 crowdsec
|
||||
1600 promtail
|
||||
"""
|
||||
|
||||
|
||||
def _wire(
|
||||
*,
|
||||
journald: str | None = JOURNALD,
|
||||
open_: str = OPEN,
|
||||
stat: str = STAT,
|
||||
cgroups: str = CGROUPS,
|
||||
comm: str = COMM,
|
||||
flags: str | None = FLAGS,
|
||||
noise: str = "",
|
||||
forwarding: str = "",
|
||||
) -> str:
|
||||
"""The report as the command prints it, framed."""
|
||||
parts = [noise, "LOGS_BEGIN\n"]
|
||||
if journald is not None:
|
||||
parts += ["[journald]\n", journald]
|
||||
parts += ["[open]\n", open_, "[stat]\n", stat, "[cgroups]\n", cgroups, "[comm]\n", comm]
|
||||
if flags is not None:
|
||||
parts += ["[flags]\n", flags]
|
||||
parts += [forwarding, "LOGS_END\n"]
|
||||
return "".join(parts)
|
||||
|
||||
|
||||
def _sources(output: str | None = None) -> list:
|
||||
return parse_log_sources(_wire() if output is None else output)[1]
|
||||
|
||||
|
||||
def _by_key(sources: list) -> dict:
|
||||
return {(s["kind"], s["container_id"], s["path"] or s["unit"]): s for s in sources}
|
||||
|
||||
|
||||
class TestParse:
|
||||
def test_journald_names_each_service_unit_once_without_its_suffix(self) -> None:
|
||||
journald, sources = parse_log_sources(_wire())
|
||||
units = [s["unit"] for s in sources if s["kind"] == "journald"]
|
||||
assert units == ["nginx", "postgresql@16-main", "ssh"]
|
||||
assert journald is True
|
||||
|
||||
def test_a_host_without_journald_says_so(self) -> None:
|
||||
journald, sources = parse_log_sources(_wire(journald=None))
|
||||
assert journald is False
|
||||
assert not [s for s in sources if s["kind"] == "journald"]
|
||||
|
||||
def test_an_open_file_names_its_writer_size_and_last_write(self) -> None:
|
||||
sources = _by_key(_sources())
|
||||
access = sources[("file", None, "/var/log/nginx/access.log")]
|
||||
assert access == {
|
||||
"kind": "file",
|
||||
"path": "/var/log/nginx/access.log",
|
||||
"unit": "nginx",
|
||||
"container_id": None,
|
||||
"process": "nginx",
|
||||
"size": 2048,
|
||||
"modified": "2025-11-04T09:56:40+00:00",
|
||||
}
|
||||
|
||||
def test_a_file_two_processes_hold_is_one_source(self) -> None:
|
||||
paths = [s["path"] for s in _sources() if s["kind"] == "file"]
|
||||
assert paths.count("/var/log/nginx/access.log") == 1
|
||||
|
||||
def test_a_file_without_a_suffix_under_var_log_counts(self) -> None:
|
||||
sources = _by_key(_sources())
|
||||
assert sources[("file", None, "/var/log/syslog")]["unit"] == "rsyslog"
|
||||
|
||||
def test_a_file_a_container_writes_carries_the_container_and_its_own_path(self) -> None:
|
||||
sources = _by_key(_sources())
|
||||
paperless = sources[("file", CID, "/usr/src/paperless/data/log/paperless.log")]
|
||||
assert paperless["unit"] is None
|
||||
assert paperless["process"] == "python3"
|
||||
assert paperless["size"] == 0
|
||||
|
||||
def test_the_engines_own_log_file_is_the_containers_stream(self) -> None:
|
||||
sources = _by_key(_sources())
|
||||
json_file = sources[("container", CID, f"/var/lib/docker/containers/{CID}/{CID}-json.log")]
|
||||
assert json_file["process"] == "dockerd"
|
||||
assert json_file["size"] == 7000
|
||||
local = sources[
|
||||
("container", OTHER, f"/var/lib/docker/containers/{OTHER}/local-logs/container.log")
|
||||
]
|
||||
assert local["modified"] == "2025-11-04T10:01:40+00:00"
|
||||
assert not [s for s in _sources() if "docker" in (s["unit"] or "")]
|
||||
|
||||
def test_journal_files_and_deleted_files_are_not_sources(self) -> None:
|
||||
paths = [s["path"] or "" for s in _sources()]
|
||||
assert not [p for p in paths if "/var/log/journal/" in p or p.startswith("/var/log/old")]
|
||||
|
||||
def test_a_databases_write_ahead_log_is_not_a_source(self) -> None:
|
||||
# LevelDB and RocksDB name theirs 000004.log; it holds records, not lines.
|
||||
paths = [s["path"] or "" for s in _sources()]
|
||||
assert not [p for p in paths if p.endswith(("000004.log", "012345.log"))]
|
||||
|
||||
def test_a_path_with_a_space_survives(self) -> None:
|
||||
sources = _by_key(_sources())
|
||||
spaced = sources[("file", None, "/srv/app/log with space.log")]
|
||||
# The writer runs in a login session, not in a service; stat did not answer.
|
||||
assert spaced["unit"] is None
|
||||
assert spaced["process"] == "app"
|
||||
assert spaced["size"] is None
|
||||
assert spaced["modified"] is None
|
||||
|
||||
def test_sources_come_sorted_by_kind_then_container_then_name(self) -> None:
|
||||
sources = _sources()
|
||||
order = [(s["kind"], s["container_id"] or "", s["path"] or s["unit"]) for s in sources]
|
||||
assert order == sorted(order)
|
||||
|
||||
def test_noise_before_the_frame_is_ignored(self) -> None:
|
||||
assert _sources(_wire(noise="Last login: today\r\n\x1b[0m"))
|
||||
|
||||
def test_a_host_without_gnu_find_raises_rather_than_reading_as_no_files(self) -> None:
|
||||
# busybox find has neither -lname nor -printf: no files would read as none.
|
||||
with pytest.raises(LogSourcesUnavailable, match="find"):
|
||||
parse_log_sources("LOGS_BEGIN\n[journald]\nnginx.service\n[no-find]\nLOGS_END\n")
|
||||
|
||||
def test_a_report_cut_short_raises_rather_than_reading_as_fewer_logs(self) -> None:
|
||||
cut = _wire().replace("LOGS_END\n", "")
|
||||
with pytest.raises(ValueError, match="log source report"):
|
||||
parse_log_sources(cut)
|
||||
|
||||
|
||||
class TestWriters:
|
||||
"""A file is attributed to the process writing it, not to one reading it
|
||||
(NetOrk/netork#2092): CrowdSec, Promtail or a ``tail -F`` holding a log is
|
||||
no producer of it."""
|
||||
|
||||
def test_a_reader_with_the_lower_pid_does_not_take_the_writers_file(self) -> None:
|
||||
access = _by_key(_sources())[("file", None, "/var/log/nginx/access.log")]
|
||||
assert (access["unit"], access["process"]) == ("nginx", "nginx")
|
||||
|
||||
def test_a_file_only_readers_hold_is_listed_without_a_producer(self) -> None:
|
||||
paperless = _by_key(_sources())[("file", None, "/mnt/docroot/data/log/paperless.log")]
|
||||
assert paperless == {
|
||||
"kind": "file",
|
||||
"path": "/mnt/docroot/data/log/paperless.log",
|
||||
"unit": None,
|
||||
"container_id": None,
|
||||
"process": None,
|
||||
"size": 81920,
|
||||
"modified": "2025-11-04T10:05:00+00:00",
|
||||
}
|
||||
|
||||
def test_a_file_open_for_reading_and_writing_is_written(self) -> None:
|
||||
spaced = _by_key(_sources())[("file", None, "/srv/app/log with space.log")]
|
||||
assert spaced["process"] == "app"
|
||||
|
||||
def test_a_file_only_a_reader_in_a_container_holds_is_left_out(self) -> None:
|
||||
# Its path is the reader's view of a mount: it names nothing on the host,
|
||||
# and the container does not write it.
|
||||
assert not [s for s in _sources() if s["path"] == "/var/log/host/syslog"]
|
||||
|
||||
def test_sockets_and_pipes_are_no_sources(self) -> None:
|
||||
assert not [s for s in _sources() if not (s["path"] or "/").startswith("/")]
|
||||
|
||||
def test_a_deleted_file_stays_out_though_it_is_written(self) -> None:
|
||||
assert not [s for s in _sources() if (s["path"] or "").startswith("/var/log/old")]
|
||||
|
||||
def test_a_holder_whose_flags_did_not_answer_is_no_writer(self) -> None:
|
||||
# The process ended between the listing and the read of its fdinfo.
|
||||
flags = FLAGS.replace("/proc/900/fdinfo/5:flags:\t0102001\n", "")
|
||||
flags = flags.replace("/proc/901/fdinfo/5:flags:\t0102001\n", "")
|
||||
access = _by_key(_sources(_wire(flags=flags)))[("file", None, "/var/log/nginx/access.log")]
|
||||
assert (access["unit"], access["process"]) == (None, None)
|
||||
|
||||
def test_a_report_without_flags_names_the_first_holder_as_before(self) -> None:
|
||||
# An older command printed no [flags]: who writes cannot be told apart.
|
||||
access = _by_key(_sources(_wire(flags=None)))[("file", None, "/var/log/nginx/access.log")]
|
||||
assert (access["unit"], access["process"]) == ("crowdsec", "crowdsec")
|
||||
|
||||
|
||||
#: rsyslog running, forwarding to Graylog, its config prefixed so that no line reads as a section.
|
||||
FORWARDING = """[rsyslog]
|
||||
running
|
||||
[rsyslog-conf]
|
||||
| module(load="imuxsock")
|
||||
| [comment-looking line]
|
||||
|
|
||||
| *.* @graylog.lan:1514;RSYSLOG_SyslogProtocol23Format
|
||||
| input(type="imfile" File="/var/log/nginx/*.log" Tag="nginx")
|
||||
[journald-conf]
|
||||
#ForwardToSyslog=yes
|
||||
"""
|
||||
|
||||
|
||||
class TestForwarding:
|
||||
"""Where the logs go beyond the host, read in the same command (NetOrk/netork#2053)."""
|
||||
|
||||
def test_rsyslog_forwarding_and_the_journal_reaching_it(self) -> None:
|
||||
forwarding = parse_log_forwarding(_wire(forwarding=FORWARDING))
|
||||
assert forwarding is not None
|
||||
assert forwarding["rsyslog"] is True
|
||||
assert forwarding["journal"] is True
|
||||
assert [(r["target"], r["port"], r["complete"]) for r in forwarding["rules"]] == [
|
||||
("graylog.lan", 1514, True)
|
||||
]
|
||||
assert forwarding["file_inputs"] == [{"path": "/var/log/nginx/*.log", "ruleset": None}]
|
||||
|
||||
def test_a_config_line_in_brackets_is_no_section(self) -> None:
|
||||
forwarding = parse_log_forwarding(_wire(forwarding=FORWARDING))
|
||||
assert forwarding is not None and forwarding["rules"]
|
||||
|
||||
def test_an_rsyslog_that_does_not_run_forwards_nothing(self) -> None:
|
||||
stopped = FORWARDING.replace("running\n", "")
|
||||
forwarding = parse_log_forwarding(_wire(forwarding=stopped))
|
||||
assert forwarding == {"rsyslog": False, "journal": False, "rules": [], "file_inputs": []}
|
||||
|
||||
def test_a_report_without_the_sections_says_nothing(self) -> None:
|
||||
assert parse_log_forwarding(_wire()) is None
|
||||
|
||||
def test_the_reading_carries_it(self) -> None:
|
||||
reading = _Host({True: _wire(forwarding=FORWARDING)}).get_log_sources()
|
||||
assert reading["forwarding"] is not None
|
||||
assert reading["forwarding"]["rules"][0]["target"] == "graylog.lan"
|
||||
|
||||
|
||||
class _Host(LogSourcesMixin):
|
||||
"""Carries the command the way a driver does; *answers* per privilege."""
|
||||
|
||||
def __init__(self, answers: dict[bool, str]) -> None:
|
||||
self.answers = answers
|
||||
self.calls: list[tuple[str, bool]] = []
|
||||
|
||||
def _run_log_sources_command(self, command: str, *, privileged: bool) -> str:
|
||||
self.calls.append((command, privileged))
|
||||
return self.answers[privileged]
|
||||
|
||||
|
||||
class TestMixin:
|
||||
def test_it_reads_as_root_in_one_sh_call(self) -> None:
|
||||
host = _Host({True: _wire()})
|
||||
reading = host.get_log_sources()
|
||||
assert reading["attributed"] is True
|
||||
command, privileged = host.calls[0]
|
||||
assert privileged is True
|
||||
assert command.startswith("sh -c ")
|
||||
assert len(host.calls) == 1
|
||||
|
||||
def test_without_root_it_reads_again_and_says_so(self) -> None:
|
||||
host = _Host({True: "sudo: a password is required\n", False: _wire()})
|
||||
reading = host.get_log_sources()
|
||||
assert reading["attributed"] is False
|
||||
assert [privileged for _, privileged in host.calls] == [True, False]
|
||||
assert reading["sources"]
|
||||
|
||||
def test_a_host_without_gnu_find_is_not_read_again_unprivileged(self) -> None:
|
||||
host = _Host({True: "LOGS_BEGIN\n[no-find]\nLOGS_END\n"})
|
||||
with pytest.raises(LogSourcesUnavailable):
|
||||
host.get_log_sources()
|
||||
assert len(host.calls) == 1
|
||||
|
||||
def test_a_host_that_answers_neither_way_raises(self) -> None:
|
||||
host = _Host({True: "", False: "garbage"})
|
||||
with pytest.raises(ValueError):
|
||||
host.get_log_sources()
|
||||
|
||||
|
||||
class TestCommand:
|
||||
def test_it_is_valid_posix_sh(self) -> None:
|
||||
assert subprocess.run(["sh", "-n", "-c", LOG_SOURCES_COMMAND]).returncode == 0
|
||||
|
||||
@pytest.mark.skipif(shutil.which("find") is None, reason="needs find on this host")
|
||||
def test_here_a_file_held_for_writing_has_a_producer_and_one_held_for_reading_none(
|
||||
self, tmp_path
|
||||
) -> None:
|
||||
written, read = tmp_path / "written.log", tmp_path / "read.log"
|
||||
read.write_text("one line\n")
|
||||
with written.open("a") as w, read.open() as r:
|
||||
out = subprocess.run(
|
||||
["sh", "-c", LOG_SOURCES_COMMAND], capture_output=True, text=True, timeout=60
|
||||
).stdout
|
||||
assert w.writable() and r.readable()
|
||||
sources = {s["path"]: s for s in _sources(out)}
|
||||
assert sources[str(written)]["process"] is not None
|
||||
assert sources[str(read)]["process"] is None
|
||||
|
||||
@pytest.mark.skipif(shutil.which("find") is None, reason="needs find on this host")
|
||||
def test_it_runs_here_and_parses(self, tmp_path) -> None:
|
||||
log = tmp_path / "held.log"
|
||||
log.write_text("one line\n")
|
||||
with log.open() as held:
|
||||
out = subprocess.run(
|
||||
["sh", "-c", LOG_SOURCES_COMMAND], capture_output=True, text=True, timeout=60
|
||||
).stdout
|
||||
assert held.readable()
|
||||
# This test's own process holds the file, so it must be among the sources.
|
||||
mine = [s for s in _sources(out) if s["path"] == str(log)]
|
||||
assert mine and mine[0]["size"] == len("one line\n")
|
||||
# Whether rsyslog runs here or not, the report says where logs go.
|
||||
assert parse_log_forwarding(out) is not None
|
||||
@@ -0,0 +1,161 @@
|
||||
"""A time zone as a device takes it: an IANA name, a POSIX TZ rule, or its parts.
|
||||
|
||||
The wire carries IANA names (``Europe/Berlin``). OpenWrt hands ``TZ`` to libc and
|
||||
wants the rule itself (``CET-1CEST,M3.5.0,M10.5.0/3``); a switch wants a UTC
|
||||
offset and the summer-time rule. The rule is the footer of the zone's TZif file
|
||||
from the ``tzdata`` package (moved here from netOrk's ``core/posix_tz.py`` for
|
||||
NetOrk/netork#2361).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from importlib.resources import files
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import (
|
||||
PosixTzDict,
|
||||
UnknownTimezone,
|
||||
is_iana_zone,
|
||||
is_posix_tz,
|
||||
parse_posix_tz,
|
||||
posix_tz,
|
||||
)
|
||||
|
||||
BERLIN = "CET-1CEST,M3.5.0,M10.5.0/3"
|
||||
|
||||
|
||||
def _zones() -> list[str]:
|
||||
return files("tzdata").joinpath("zones").read_text().split()
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("zone", "rule"),
|
||||
[
|
||||
("Europe/Berlin", BERLIN),
|
||||
("America/New_York", "EST5EDT,M3.2.0,M11.1.0"),
|
||||
("America/Sao_Paulo", "<-03>3"),
|
||||
("Australia/Lord_Howe", "<+1030>-10:30<+11>-11,M10.1.0,M4.1.0"),
|
||||
("Asia/Kolkata", "IST-5:30"),
|
||||
("UTC", "UTC0"),
|
||||
],
|
||||
)
|
||||
def test_an_iana_name_becomes_its_posix_rule(zone, rule):
|
||||
assert posix_tz(zone) == rule
|
||||
assert is_iana_zone(zone)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("rule", [BERLIN, "<+03>-3", "EST5EDT,M3.2.0/2,M11.1.0/2", "UTC0"])
|
||||
def test_a_posix_string_is_taken_as_it_is(rule):
|
||||
assert posix_tz(rule) == rule
|
||||
assert not is_iana_zone(rule)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"value",
|
||||
["Mars/Olympus_Mons", "Europe", "../../etc/passwd", "/etc/localtime", "Europe/../UTC", ""],
|
||||
)
|
||||
def test_neither_a_zone_nor_a_rule_has_none(value):
|
||||
assert posix_tz(value) is None
|
||||
assert not is_iana_zone(value)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("value", ["UTC", "GMT0", BERLIN, "<-03>3", "IST-5:30"])
|
||||
def test_posix_strings(value):
|
||||
assert is_posix_tz(value)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("value", ["Europe/Berlin", "America/Argentina/Buenos_Aires", "", "CET-"])
|
||||
def test_not_posix_strings(value):
|
||||
assert not is_posix_tz(value)
|
||||
|
||||
|
||||
def test_every_zone_of_the_database_has_a_rule_the_check_accepts():
|
||||
# A caller tells an IANA name from a rule with is_posix_tz; it must never
|
||||
# take the database's own rule for a name.
|
||||
zones = _zones()
|
||||
missing = [z for z in zones if not is_posix_tz(posix_tz(z) or "")]
|
||||
assert len(zones) > 500
|
||||
assert missing == []
|
||||
|
||||
|
||||
def test_the_refusal_names_the_zone_in_plain_english():
|
||||
exc = UnknownTimezone("Mars/Olympus_Mons")
|
||||
|
||||
assert isinstance(exc, ValueError)
|
||||
assert exc.zone == "Mars/Olympus_Mons"
|
||||
assert "Mars/Olympus_Mons" in str(exc)
|
||||
assert "Europe/Berlin" in str(exc)
|
||||
|
||||
|
||||
class TestTheRuleInParts:
|
||||
"""What a switch takes: the standard offset and, where there is one, the summer-time rule."""
|
||||
|
||||
def test_berlin(self):
|
||||
assert parse_posix_tz(BERLIN) == PosixTzDict(
|
||||
std_name="CET",
|
||||
utc_offset=3600,
|
||||
dst_name="CEST",
|
||||
dst_offset=7200,
|
||||
dst_start={"month": 3, "week": 5, "weekday": 0, "time": 7200},
|
||||
dst_end={"month": 10, "week": 5, "weekday": 0, "time": 10800},
|
||||
)
|
||||
|
||||
def test_utc_has_no_summer_time(self):
|
||||
assert parse_posix_tz("UTC0") == {
|
||||
"std_name": "UTC",
|
||||
"utc_offset": 0,
|
||||
"dst_name": None,
|
||||
"dst_offset": None,
|
||||
"dst_start": None,
|
||||
"dst_end": None,
|
||||
}
|
||||
|
||||
@pytest.mark.parametrize("bare", ["UTC", "GMT"])
|
||||
def test_the_bare_names_openwrt_writes(self, bare):
|
||||
parts = parse_posix_tz(bare)
|
||||
assert parts is not None
|
||||
assert (parts["std_name"], parts["utc_offset"], parts["dst_name"]) == (bare, 0, None)
|
||||
|
||||
def test_sao_paulo_has_given_up_summer_time(self):
|
||||
parts = parse_posix_tz(posix_tz("America/Sao_Paulo") or "")
|
||||
assert parts is not None
|
||||
assert (parts["std_name"], parts["utc_offset"], parts["dst_offset"]) == (
|
||||
"-03",
|
||||
-10800,
|
||||
None,
|
||||
)
|
||||
|
||||
def test_lord_howe_moves_by_half_an_hour(self):
|
||||
parts = parse_posix_tz(posix_tz("Australia/Lord_Howe") or "")
|
||||
assert parts is not None
|
||||
assert (parts["utc_offset"], parts["dst_offset"]) == (37800, 39600)
|
||||
assert parts["dst_start"] == {"month": 10, "week": 1, "weekday": 0, "time": 7200}
|
||||
assert parts["dst_end"] == {"month": 4, "week": 1, "weekday": 0, "time": 7200}
|
||||
|
||||
def test_new_york_is_west_of_utc_and_gains_an_hour_by_default(self):
|
||||
parts = parse_posix_tz("EST5EDT,M3.2.0,M11.1.0")
|
||||
assert parts is not None
|
||||
assert (parts["utc_offset"], parts["dst_offset"]) == (-18000, -14400)
|
||||
|
||||
def test_a_transition_time_may_lie_outside_the_day(self):
|
||||
# Greenland's rule: a day's -1:00 and 0:00 (RFC 8536 3.3.1 allows -167..167 hours).
|
||||
parts = parse_posix_tz("<-02>2<-01>,M3.5.0/-1,M10.5.0/0")
|
||||
assert parts is not None
|
||||
assert parts["dst_start"] == {"month": 3, "week": 5, "weekday": 0, "time": -3600}
|
||||
assert parts["dst_end"] == {"month": 10, "week": 5, "weekday": 0, "time": 0}
|
||||
|
||||
def test_offsets_with_minutes_and_seconds(self):
|
||||
parts = parse_posix_tz("<+1245>-12:45<+1345>,M9.5.0/2:45,M4.1.0/3:45")
|
||||
assert parts is not None
|
||||
assert (parts["utc_offset"], parts["dst_offset"]) == (45900, 49500)
|
||||
assert parts["dst_start"] == {"month": 9, "week": 5, "weekday": 0, "time": 9900}
|
||||
|
||||
@pytest.mark.parametrize("rule", ["Europe/Berlin", "", "CET-", "CET-1CEST,J60,J300"])
|
||||
def test_what_it_cannot_take_apart_is_none(self, rule):
|
||||
# Julian-day rules are POSIX, but no zone of the database uses them.
|
||||
assert parse_posix_tz(rule) is None
|
||||
|
||||
def test_every_zone_of_the_database_comes_apart(self):
|
||||
broken = [z for z in _zones() if parse_posix_tz(posix_tz(z) or "") is None]
|
||||
assert broken == []
|
||||
@@ -18,6 +18,7 @@ import pytest
|
||||
|
||||
from napalm_device_types import (
|
||||
AccessPointDriver,
|
||||
BackupServerDriver,
|
||||
DeviceTypeDriver,
|
||||
FirewallDriver,
|
||||
HypervisorDriver,
|
||||
@@ -33,6 +34,7 @@ from napalm_device_types.roles import primary_role_of, roles_of
|
||||
|
||||
ROLE_BASES = [
|
||||
AccessPointDriver,
|
||||
BackupServerDriver,
|
||||
FirewallDriver,
|
||||
HypervisorDriver,
|
||||
ManagementControllerDriver,
|
||||
@@ -75,6 +77,8 @@ class TestDeclaredMethodsDoNotExistAtRuntime:
|
||||
@pytest.mark.parametrize(
|
||||
("base", "method"),
|
||||
[
|
||||
(BackupServerDriver, "get_backup_datastores"),
|
||||
(BackupServerDriver, "run_backup_job"),
|
||||
(StorageDriver, "get_disks"),
|
||||
(StorageDriver, "get_volumes"),
|
||||
(HypervisorDriver, "get_vms"),
|
||||
|
||||
@@ -0,0 +1,51 @@
|
||||
"""get_running_backups: the backups running on a device right now.
|
||||
|
||||
An application that restarts a device asks first, right before the restart: a
|
||||
restart aborts a backup that runs on the device -- a hypervisor's vzdump, a
|
||||
backup server's backup, sync, verify or garbage collection, a host's own restic
|
||||
or borg run -- and a guest's restart breaks the backup its hypervisor is taking
|
||||
of it (NetOrk/netork#2318). A driver mixes it in when it can tell; no role
|
||||
demands it, and a driver without it cannot tell.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import inspect
|
||||
from typing import get_type_hints
|
||||
|
||||
from napalm_device_types import (
|
||||
RUNNING_BACKUP_KINDS,
|
||||
BackupServerDriver,
|
||||
DeviceTypeDriver,
|
||||
HypervisorDriver,
|
||||
OSDriver,
|
||||
RunningBackupDict,
|
||||
RunningBackupsMixin,
|
||||
)
|
||||
|
||||
|
||||
def test_no_role_demands_it():
|
||||
for role in (BackupServerDriver, HypervisorDriver, OSDriver, DeviceTypeDriver):
|
||||
assert not issubclass(role, RunningBackupsMixin)
|
||||
|
||||
|
||||
def test_declared_not_implemented():
|
||||
"""hasattr is the caller's probe: only a driver that can tell answers True."""
|
||||
assert not hasattr(RunningBackupsMixin, "get_running_backups")
|
||||
|
||||
|
||||
def test_declaration_documents_the_contract():
|
||||
source = inspect.getsource(RunningBackupsMixin)
|
||||
assert "def get_running_backups(self) -> list[RunningBackupDict]" in source
|
||||
assert "empty list for a failed read" in source
|
||||
|
||||
|
||||
def test_the_kinds_cover_a_backup_servers_jobs_and_the_backup_itself():
|
||||
assert set(RUNNING_BACKUP_KINDS) == {"backup", "sync", "verify", "gc", "prune", "tape"}
|
||||
|
||||
|
||||
def test_the_shape():
|
||||
hints = get_type_hints(RunningBackupDict)
|
||||
assert set(hints) == {"kind", "job", "task", "guests", "since"}
|
||||
assert hints["guests"] == list[str]
|
||||
assert hints["since"] == int | None
|
||||
@@ -0,0 +1,108 @@
|
||||
"""The scheduled-jobs contract: its digest, the names and specs it takes (NetOrk/netork#1933)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import ScheduledJobsMixin
|
||||
from napalm_device_types.scheduled_jobs import (
|
||||
ACTIONS,
|
||||
ScheduledJobChanged,
|
||||
ScheduledJobRefused,
|
||||
check_job_name,
|
||||
check_spec,
|
||||
job_digest,
|
||||
)
|
||||
|
||||
JOB = {
|
||||
"kind": "cron",
|
||||
"source": "/etc/crontab",
|
||||
"run_as": "root",
|
||||
"schedule": "*/5 * * * *",
|
||||
"command": "/usr/local/bin/backup",
|
||||
"enabled": True,
|
||||
}
|
||||
|
||||
|
||||
class TestDigest:
|
||||
def test_sixteen_hex_characters(self):
|
||||
digest = job_digest(JOB)
|
||||
assert len(digest) == 16
|
||||
int(digest, 16)
|
||||
|
||||
def test_the_same_definition_has_the_same_digest(self):
|
||||
assert job_digest(JOB) == job_digest(dict(JOB))
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"field, value",
|
||||
[
|
||||
("schedule", "0 * * * *"),
|
||||
("command", "/usr/local/bin/other"),
|
||||
("run_as", "nobody"),
|
||||
("enabled", False),
|
||||
("source", "/etc/cron.d/backup"),
|
||||
("kind", "anacron"),
|
||||
],
|
||||
)
|
||||
def test_every_part_of_the_definition_changes_it(self, field, value):
|
||||
assert job_digest({**JOB, field: value}) != job_digest(JOB)
|
||||
|
||||
def test_content_the_definition_points_at_changes_it(self):
|
||||
assert job_digest(JOB, content="12345 678") != job_digest(JOB)
|
||||
|
||||
def test_what_the_host_says_of_its_runs_does_not(self):
|
||||
assert job_digest({**JOB, "last_run": "2026-11-03T06:00:00Z"}) == job_digest(JOB)
|
||||
|
||||
|
||||
class TestNames:
|
||||
@pytest.mark.parametrize("name", ["backup", "netork-backup-db", "a", "x1"])
|
||||
def test_a_good_name(self, name):
|
||||
check_job_name(name)
|
||||
|
||||
@pytest.mark.parametrize("name", ["", "-x", "Backup", "a b", "a/b", "a.timer", "a" * 64])
|
||||
def test_a_name_a_file_or_unit_could_not_carry_safely(self, name):
|
||||
with pytest.raises(ScheduledJobRefused):
|
||||
check_job_name(name)
|
||||
|
||||
|
||||
class TestSpec:
|
||||
SPEC = {
|
||||
"name": "backup",
|
||||
"schedule": "0 3 * * *",
|
||||
"run_as": "root",
|
||||
"command": "/usr/local/bin/backup",
|
||||
"enabled": True,
|
||||
}
|
||||
|
||||
def test_a_good_spec(self):
|
||||
check_spec(self.SPEC)
|
||||
check_spec({**self.SPEC, "schedule": "@reboot"})
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"field, value",
|
||||
[
|
||||
("schedule", "every day"),
|
||||
("run_as", "root; rm"),
|
||||
("run_as", ""),
|
||||
("command", ""),
|
||||
("command", " "),
|
||||
("description", "two\nlines"),
|
||||
],
|
||||
)
|
||||
def test_a_spec_that_cannot_be_written(self, field, value):
|
||||
with pytest.raises(ScheduledJobRefused):
|
||||
check_spec({**self.SPEC, field: value})
|
||||
|
||||
|
||||
class TestContract:
|
||||
def test_the_mixin_declares_nothing_at_runtime(self):
|
||||
driver = ScheduledJobsMixin()
|
||||
for method in ("get_scheduled_jobs", "manage_scheduled_job", "put_scheduled_job"):
|
||||
assert not hasattr(driver, method)
|
||||
|
||||
def test_the_actions(self):
|
||||
assert ACTIONS == ("enable", "disable", "run")
|
||||
|
||||
def test_a_refusal_is_a_value_error_and_a_change_is_not(self):
|
||||
assert issubclass(ScheduledJobRefused, ValueError)
|
||||
assert not issubclass(ScheduledJobChanged, ValueError)
|
||||
@@ -0,0 +1,540 @@
|
||||
"""Scheduled jobs on a POSIX host: the report as root, switching, running, an application's jobs.
|
||||
|
||||
The reading command is run by a real ``sh`` over a directory tree that stands in
|
||||
for the host, with ``systemctl``, ``crontab``, ``dpkg``, ``install`` and
|
||||
``journalctl`` replaced by scripts that keep a log -- so the shell itself is what
|
||||
is tested, not a hand-written copy of its output (NetOrk/netork#1933).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import PosixScheduledJobsMixin
|
||||
from napalm_device_types._scheduled_reading import parse_reading
|
||||
from napalm_device_types.channel import CommandResult
|
||||
from napalm_device_types.scheduled_jobs import ScheduledJobChanged, ScheduledJobRefused
|
||||
|
||||
TIMERS = """\
|
||||
Id=apt-daily.timer
|
||||
Unit=apt-daily.service
|
||||
Description=Daily apt download activities
|
||||
TimersCalendar={ OnCalendar=*-*-* 06,18:00:00 ; next_elapse=@1762192800 }
|
||||
NextElapseUSecRealtime=@1762192800
|
||||
LastTriggerUSec=@1762149600
|
||||
UnitFileState=enabled
|
||||
ActiveState=active
|
||||
FragmentPath=/usr/lib/systemd/system/apt-daily.timer
|
||||
|
||||
Id=fstrim.timer
|
||||
Unit=fstrim.service
|
||||
Description=Discard unused blocks once a week
|
||||
TimersMonotonic={ OnUnitActiveUSec=1w ; next_elapse=n/a }
|
||||
NextElapseUSecRealtime=n/a
|
||||
LastTriggerUSec=n/a
|
||||
UnitFileState=static
|
||||
ActiveState=inactive
|
||||
FragmentPath=/etc/systemd/system/fstrim.timer
|
||||
|
||||
Id=backup.timer
|
||||
Unit=backup.service
|
||||
Description=netork backup
|
||||
TimersCalendar={ OnCalendar=*-*-* 03:00:00 ; next_elapse=n/a }
|
||||
NextElapseUSecRealtime=Tue 2026-11-04 03:00:00 UTC
|
||||
LastTriggerUSec=n/a
|
||||
UnitFileState=enabled
|
||||
ActiveState=active
|
||||
FragmentPath=/etc/systemd/system/backup.timer
|
||||
"""
|
||||
|
||||
UNITS = """\
|
||||
Id=apt-daily.service
|
||||
ExecStart={ path=/usr/lib/apt/apt.systemd.daily ; argv[]=/usr/lib/apt/apt.systemd.daily update ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }
|
||||
User=
|
||||
Result=success
|
||||
FragmentPath=/usr/lib/systemd/system/apt-daily.service
|
||||
|
||||
Id=fstrim.service
|
||||
ExecStart={ path=/sbin/fstrim ; argv[]=/sbin/fstrim --listed-in /etc/fstab ; ignore_errors=no }
|
||||
Result=success
|
||||
FragmentPath=/usr/lib/systemd/system/fstrim.service
|
||||
|
||||
Id=backup.service
|
||||
ExecStart={ path=SCRIPTS/backup ; argv[]=SCRIPTS/backup ; ignore_errors=no }
|
||||
User=backup
|
||||
Result=exit-code
|
||||
FragmentPath=/etc/systemd/system/backup.service
|
||||
"""
|
||||
|
||||
SYSTEMCTL = """#!/bin/sh
|
||||
echo "systemctl $*" >> "$LOG"
|
||||
case "$1" in
|
||||
list-unit-files) printf 'apt-daily.timer enabled enabled\\nfstrim.timer static -\\nbackup.timer enabled enabled\\ngetty@.timer static -\\n' ;;
|
||||
show) case "$*" in *Unit,Description*) sed "s#SCRIPTS#$SCRIPTS#g" "$FIX/timers" ;; *) sed "s#SCRIPTS#$SCRIPTS#g" "$FIX/units" ;; esac ;;
|
||||
esac
|
||||
"""
|
||||
|
||||
CRONTAB = """#!/bin/sh
|
||||
echo "crontab $*" >> "$LOG"
|
||||
[ "$1" = -u ] || exit 2
|
||||
if [ "$3" = -l ]; then
|
||||
[ -f "$SPOOL/$2" ] || { echo "no crontab for $2" >&2; exit 1; }
|
||||
cat "$SPOOL/$2"
|
||||
else
|
||||
cp "$3" "$SPOOL/$2"
|
||||
fi
|
||||
"""
|
||||
|
||||
DPKG = """#!/bin/sh
|
||||
for p in "$@"; do case " $OWNED " in *" $p "*) echo "cron: $p" ;; esac; done
|
||||
"""
|
||||
|
||||
INSTALL = """#!/bin/sh
|
||||
echo "install $*" >> "$LOG"
|
||||
args=""
|
||||
while [ $# -gt 0 ]; do case "$1" in -o|-g) shift 2 ;; *) args="$args '$1'"; shift ;; esac; done
|
||||
eval "/usr/bin/install $args"
|
||||
"""
|
||||
|
||||
ECHO_LOG = """#!/bin/sh
|
||||
echo "$(basename "$0") $*" >> "$LOG"
|
||||
[ "$(basename "$0")" = journalctl ] && echo "backup ran fine"
|
||||
exit 0
|
||||
"""
|
||||
|
||||
|
||||
class Host(PosixScheduledJobsMixin):
|
||||
"""A host in a directory; every command is run by sh, root or not."""
|
||||
|
||||
def __init__(self, root: Path, *, systemd: bool = True, own_cron_d: bool = True) -> None:
|
||||
self.root = root
|
||||
etc = root / "etc"
|
||||
self.CRON_SYSTEM_FILES = (str(etc / "crontab"),) # type: ignore[misc]
|
||||
self.CRON_D_DIRS = (str(etc / "cron.d"),) # type: ignore[misc]
|
||||
self.CRON_SPOOL_DIRS = (str(root / "spool"),) # type: ignore[misc]
|
||||
self.RUN_PARTS_DIRS = (("daily", str(etc / "cron.daily")),) # type: ignore[misc]
|
||||
self.ANACRONTAB = str(etc / "anacrontab") # type: ignore[misc]
|
||||
self.JOB_SCRIPT_DIR = str(root / "scripts") # type: ignore[misc]
|
||||
self.UNIT_DIR = str(root / "units") # type: ignore[misc]
|
||||
self.OWN_CRON_D_DIR = str(etc / "cron.d") if own_cron_d else None # type: ignore[misc]
|
||||
self.ROOT_GROUP = subprocess.run(
|
||||
["id", "-gn"], capture_output=True, text=True
|
||||
).stdout.strip() # type: ignore[misc]
|
||||
self.systemd_dir = root / ("run-systemd" if systemd else "no-systemd")
|
||||
self.commands: list[str] = []
|
||||
|
||||
def _sj_paths(self): # type: ignore[no-untyped-def]
|
||||
paths = super()._sj_paths()
|
||||
from dataclasses import replace
|
||||
|
||||
return replace(paths, systemd_marker=str(self.systemd_dir))
|
||||
|
||||
def run_command(self, command, *, privileged=False, timeout=60, stdin=None): # type: ignore[no-untyped-def]
|
||||
self.commands.append(command)
|
||||
env = {
|
||||
**os.environ,
|
||||
"PATH": f"{self.root / 'bin'}:{os.environ['PATH']}",
|
||||
"LOG": str(self.root / "log"),
|
||||
"SPOOL": str(self.root / "spool"),
|
||||
"FIX": str(self.root / "fix"),
|
||||
"SCRIPTS": str(self.root / "scripts"),
|
||||
"OWNED": str(self.root / "etc" / "crontab"),
|
||||
}
|
||||
done = subprocess.run(
|
||||
["sh", "-c", command], input=stdin, capture_output=True, env=env, timeout=30
|
||||
)
|
||||
return CommandResult(done.stdout.decode(), done.stderr.decode(), done.returncode)
|
||||
|
||||
def log(self) -> str:
|
||||
log = self.root / "log"
|
||||
return log.read_text() if log.exists() else ""
|
||||
|
||||
|
||||
def _tool(root: Path, name: str, body: str) -> None:
|
||||
path = root / "bin" / name
|
||||
path.write_text(body)
|
||||
path.chmod(0o755)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def root(tmp_path: Path) -> Path:
|
||||
for d in (
|
||||
"etc/cron.d",
|
||||
"etc/cron.daily",
|
||||
"spool",
|
||||
"scripts",
|
||||
"units",
|
||||
"bin",
|
||||
"fix",
|
||||
"run-systemd",
|
||||
):
|
||||
(tmp_path / d).mkdir(parents=True)
|
||||
(tmp_path / "etc/crontab").write_text(
|
||||
"SHELL=/bin/sh\nOUT="
|
||||
+ str(tmp_path / "out")
|
||||
+ "\n17 * * * * root cd / && run-parts /etc/cron.hourly\n"
|
||||
'*/5 * * * * root echo "$OUT" > ' + str(tmp_path / "ran") + "\n"
|
||||
)
|
||||
(tmp_path / "etc/cron.d/certbot").write_text("0 */12 * * * root certbot -q renew\n")
|
||||
(tmp_path / "etc/anacrontab").write_text("1 5 cron.daily run-parts /etc/cron.daily\n")
|
||||
(tmp_path / "spool/www-data").write_text("*/5 * * * * php /var/www/cron.php\n")
|
||||
for name, mode in (("logrotate", 0o755), ("off", 0o644), ("x.dpkg-old", 0o755)):
|
||||
script = tmp_path / "etc/cron.daily" / name
|
||||
script.write_text("#!/bin/sh\necho rotated\n")
|
||||
script.chmod(mode)
|
||||
(tmp_path / "scripts/backup").write_text(
|
||||
"#!/bin/sh\n# napalm-scheduled-job: backup nightly backup\n# schedule: 0 3 * * *\n/usr/local/bin/backup --all\n"
|
||||
)
|
||||
(tmp_path / "fix/timers").write_text(TIMERS)
|
||||
(tmp_path / "fix/units").write_text(UNITS)
|
||||
for name, body in (
|
||||
("systemctl", SYSTEMCTL),
|
||||
("crontab", CRONTAB),
|
||||
("dpkg", DPKG),
|
||||
("install", INSTALL),
|
||||
("journalctl", ECHO_LOG),
|
||||
("systemd-analyze", ECHO_LOG),
|
||||
):
|
||||
_tool(tmp_path, name, body)
|
||||
return tmp_path
|
||||
|
||||
|
||||
def _jobs(host: Host) -> dict[str, dict]:
|
||||
return {j["key"]: j for j in host.get_scheduled_jobs()["jobs"]}
|
||||
|
||||
|
||||
class TestReading:
|
||||
def test_every_source_is_read(self, root: Path):
|
||||
jobs = _jobs(Host(root))
|
||||
kinds = sorted({j["kind"] for j in jobs.values()})
|
||||
assert kinds == ["anacron", "cron", "run_parts", "systemd_timer"]
|
||||
users = {j["run_as"] for j in jobs.values() if j["kind"] == "cron"}
|
||||
assert users == {"root", "www-data"}
|
||||
|
||||
def test_a_package_owns_what_it_installed(self, root: Path):
|
||||
jobs = _jobs(Host(root))
|
||||
crontab = [j for j in jobs.values() if j["source"] == str(root / "etc/crontab")]
|
||||
assert {j["managed_by"] for j in crontab} == {"package"}
|
||||
certbot = next(j for j in jobs.values() if j["source"].endswith("certbot"))
|
||||
assert certbot["managed_by"] == "admin"
|
||||
|
||||
def test_run_parts_by_execute_bit_and_the_names_run_parts_runs(self, root: Path):
|
||||
jobs = _jobs(Host(root))
|
||||
daily = root / "etc/cron.daily"
|
||||
assert jobs[f"run-parts:{daily}/logrotate"]["enabled"]
|
||||
assert not jobs[f"run-parts:{daily}/off"]["enabled"]
|
||||
assert f"run-parts:{daily}/x.dpkg-old" not in jobs
|
||||
|
||||
def test_a_changed_script_changes_its_digest(self, root: Path):
|
||||
host = Host(root)
|
||||
key = f"run-parts:{root}/etc/cron.daily/logrotate"
|
||||
before = _jobs(host)[key]["digest"]
|
||||
(root / "etc/cron.daily/logrotate").write_text("#!/bin/sh\necho something else\n")
|
||||
assert _jobs(host)[key]["digest"] != before
|
||||
|
||||
def test_timers_with_their_unit_and_runs(self, root: Path):
|
||||
apt = _jobs(Host(root))["timer:apt-daily.timer"]
|
||||
assert apt["schedule"] == "OnCalendar=*-*-* 06,18:00:00"
|
||||
assert apt["schedule_kind"] == "calendar"
|
||||
assert apt["command"] == "/usr/lib/apt/apt.systemd.daily update"
|
||||
assert (apt["run_as"], apt["managed_by"], apt["enabled"]) == ("root", "package", True)
|
||||
assert apt["last_run"] == "2025-11-03T06:00:00+00:00"
|
||||
assert apt["next_run"] == "2025-11-03T18:00:00+00:00"
|
||||
assert apt["last_result"] == "success"
|
||||
assert apt["actions"] == ["enable", "disable", "run"]
|
||||
|
||||
def test_a_static_timer_only_runs(self, root: Path):
|
||||
fstrim = _jobs(Host(root))["timer:fstrim.timer"]
|
||||
assert fstrim["schedule"] == "OnUnitActiveSec=1w"
|
||||
assert fstrim["schedule_kind"] == "monotonic"
|
||||
assert fstrim["actions"] == ["run"]
|
||||
assert fstrim.get("last_run") is None
|
||||
assert fstrim["managed_by"] == "admin"
|
||||
|
||||
def test_an_applications_timer_by_its_script(self, root: Path):
|
||||
backup = _jobs(Host(root))["job:backup"]
|
||||
assert backup["managed_by"] == "application"
|
||||
assert (backup["name"], backup["description"]) == ("backup", "nightly backup")
|
||||
assert (backup["schedule"], backup["schedule_kind"]) == ("0 3 * * *", "cron")
|
||||
assert backup["command"] == "/usr/local/bin/backup --all"
|
||||
assert backup["run_as"] == "backup"
|
||||
assert "edit" in backup["actions"] and "delete" in backup["actions"]
|
||||
assert backup["next_run"] == "2026-11-04T03:00:00+00:00"
|
||||
assert "last_result" not in backup
|
||||
|
||||
def test_a_file_without_a_last_newline_does_not_swallow_the_next(self, root: Path):
|
||||
(root / "etc/cron.d/certbot").write_text("0 */12 * * * root certbot -q renew")
|
||||
jobs = _jobs(Host(root))
|
||||
assert any(j["source"].endswith("certbot") for j in jobs.values())
|
||||
assert any(j["source"].endswith("spool/www-data") for j in jobs.values())
|
||||
|
||||
def test_without_systemd_no_timers(self, root: Path):
|
||||
jobs = _jobs(Host(root, systemd=False))
|
||||
assert not [j for j in jobs.values() if j["kind"] == "systemd_timer"]
|
||||
|
||||
def test_a_report_cut_short_raises(self, root: Path):
|
||||
host = Host(root)
|
||||
full = host.run_command
|
||||
host.run_command = lambda c, **kw: full(c, **kw)._replace(
|
||||
stdout=full(c, **kw).stdout.split("SJ_END")[0][:-3]
|
||||
) # type: ignore[method-assign]
|
||||
with pytest.raises(ValueError, match="cut short"):
|
||||
host.get_scheduled_jobs()
|
||||
|
||||
|
||||
class TestDriverSettings:
|
||||
def test_what_the_platform_generates(self, root: Path):
|
||||
host = Host(root)
|
||||
host.PLATFORM_SOURCES = ("*/cron.d/cert*",) # type: ignore[misc]
|
||||
certbot = next(j for j in _jobs(host).values() if j["source"].endswith("certbot"))
|
||||
assert certbot["managed_by"] == "platform"
|
||||
|
||||
def test_the_time_zone_and_owners_commands_are_the_drivers(self, root: Path):
|
||||
host = Host(root)
|
||||
host.TIME_ZONE_SH = "echo America/New_York" # type: ignore[misc]
|
||||
host.OWNERS_SH = "cat >/dev/null" # type: ignore[misc]
|
||||
reading = host.get_scheduled_jobs()
|
||||
assert reading["time_zone"] == "America/New_York"
|
||||
assert {j["managed_by"] for j in reading["jobs"] if j["kind"] == "cron"} == {"admin"}
|
||||
|
||||
|
||||
class TestParseReading:
|
||||
def test_the_time_zone_from_either_file(self):
|
||||
out = "SJ_BEGIN\n[tz]\n/usr/share/zoneinfo/Europe/Berlin\nSJ_END\n"
|
||||
assert parse_reading(out).time_zone == "Europe/Berlin"
|
||||
assert parse_reading("SJ_BEGIN\n[tz]\nEtc/UTC\nSJ_END").time_zone == "Etc/UTC"
|
||||
assert parse_reading("SJ_BEGIN\n[tz]\nSJ_END").time_zone is None
|
||||
|
||||
def test_owners_skip_diversions(self):
|
||||
out = "SJ_BEGIN\n[owners]\ndiversion by x from: /etc/crontab\ncron, cron-daemon: /etc/cron.d/x\nSJ_END"
|
||||
reading = parse_reading(out + "")
|
||||
assert reading.jobs == []
|
||||
|
||||
def test_a_time_not_in_utc_is_left_unknown(self):
|
||||
from napalm_device_types._scheduled_reading import _timestamp
|
||||
|
||||
assert _timestamp("Tue 2026-11-04 03:00:00 CET") is None
|
||||
assert _timestamp("@0") is None
|
||||
|
||||
|
||||
class TestSwitching:
|
||||
def _cron(self, host: Host, path: str) -> dict:
|
||||
return next(j for j in _jobs(host).values() if j["source"] == path and j["kind"] == "cron")
|
||||
|
||||
def test_a_system_line_switched_off_and_on(self, root: Path):
|
||||
host = Host(root)
|
||||
path = str(root / "etc/cron.d/certbot")
|
||||
job = self._cron(host, path)
|
||||
mode = os.stat(path).st_mode
|
||||
assert host.manage_scheduled_job(job["key"], "disable", digest=job["digest"])["success"]
|
||||
assert (
|
||||
root / "etc/cron.d/certbot"
|
||||
).read_text() == "#napalm-disabled: 0 */12 * * * root certbot -q renew\n"
|
||||
off = _jobs(host)[job["key"]]
|
||||
assert not off["enabled"]
|
||||
host.manage_scheduled_job(job["key"], "enable", digest=off["digest"])
|
||||
assert (root / "etc/cron.d/certbot").read_text() == "0 */12 * * * root certbot -q renew\n"
|
||||
assert f"install -D -m {mode & 0o777:o}" in host.log()
|
||||
assert os.stat(path).st_mode == mode
|
||||
|
||||
def test_a_linked_file_is_written_through_its_link(self, root: Path):
|
||||
real = root / "etc/vzdump.cron"
|
||||
real.write_text("0 1 * * * root vzdump --all\n")
|
||||
real.chmod(0o640)
|
||||
link = root / "etc/cron.d/vzdump"
|
||||
link.symlink_to(real)
|
||||
host = Host(root)
|
||||
job = self._cron(host, str(link))
|
||||
host.manage_scheduled_job(job["key"], "disable", digest=job["digest"])
|
||||
assert link.is_symlink()
|
||||
assert real.read_text() == "#napalm-disabled: 0 1 * * * root vzdump --all\n"
|
||||
assert real.stat().st_mode & 0o777 == 0o640
|
||||
|
||||
def test_a_users_line_goes_through_crontab(self, root: Path):
|
||||
host = Host(root)
|
||||
job = self._cron(host, str(root / "spool/www-data"))
|
||||
host.manage_scheduled_job(job["key"], "disable", digest=job["digest"])
|
||||
assert (root / "spool/www-data").read_text().startswith("#napalm-disabled: */5")
|
||||
assert "crontab -u www-data /" in host.log()
|
||||
|
||||
def test_a_change_it_did_not_see_is_refused(self, root: Path):
|
||||
host = Host(root)
|
||||
job = self._cron(host, str(root / "etc/cron.d/certbot"))
|
||||
with pytest.raises(ScheduledJobChanged):
|
||||
host.manage_scheduled_job(job["key"], "disable", digest="0" * 16)
|
||||
|
||||
def test_an_unknown_job_and_an_unknown_action(self, root: Path):
|
||||
host = Host(root)
|
||||
with pytest.raises(LookupError):
|
||||
host.manage_scheduled_job("cron:/nowhere:1", "disable", digest="x")
|
||||
with pytest.raises(ScheduledJobRefused):
|
||||
host.manage_scheduled_job("cron:/nowhere:1", "explode", digest="x")
|
||||
|
||||
def test_a_static_timer_is_not_disabled(self, root: Path):
|
||||
host = Host(root)
|
||||
fstrim = _jobs(host)["timer:fstrim.timer"]
|
||||
with pytest.raises(ScheduledJobRefused):
|
||||
host.manage_scheduled_job("timer:fstrim.timer", "disable", digest=fstrim["digest"])
|
||||
|
||||
def test_a_timer_and_a_run_parts_script(self, root: Path):
|
||||
host = Host(root)
|
||||
jobs = _jobs(host)
|
||||
apt = jobs["timer:apt-daily.timer"]
|
||||
host.manage_scheduled_job(apt["key"], "disable", digest=apt["digest"])
|
||||
assert "systemctl disable --now -- apt-daily.timer" in host.log()
|
||||
key = f"run-parts:{root}/etc/cron.daily/off"
|
||||
host.manage_scheduled_job(key, "enable", digest=jobs[key]["digest"])
|
||||
assert os.access(root / "etc/cron.daily/off", os.X_OK)
|
||||
|
||||
def test_an_applications_timer_is_switched_by_its_name(self, root: Path):
|
||||
host = Host(root)
|
||||
backup = _jobs(host)["job:backup"]
|
||||
host.manage_scheduled_job("job:backup", "disable", digest=backup["digest"])
|
||||
assert "systemctl disable --now -- backup.timer" in host.log()
|
||||
|
||||
|
||||
class TestRunning:
|
||||
def test_a_cron_line_runs_as_cron_would(self, root: Path):
|
||||
host = Host(root)
|
||||
job = next(j for j in _jobs(host).values() if "ran" in j["command"])
|
||||
result = host.manage_scheduled_job(job["key"], "run", digest=job["digest"])
|
||||
assert result["success"]
|
||||
assert (root / "ran").read_text() == f"{root / 'out'}\n"
|
||||
assert "timeout 300 sh -c" in host.commands[-1]
|
||||
|
||||
def test_a_timer_runs_its_unit_and_says_what_its_journal_said(self, root: Path):
|
||||
host = Host(root)
|
||||
backup = _jobs(host)["job:backup"]
|
||||
result = host.manage_scheduled_job("job:backup", "run", digest=backup["digest"])
|
||||
assert result == {"success": True, "output": "backup ran fine"}
|
||||
assert "systemctl start -- backup.service" in host.log()
|
||||
|
||||
def test_a_run_parts_script_runs(self, root: Path):
|
||||
host = Host(root)
|
||||
key = f"run-parts:{root}/etc/cron.daily/logrotate"
|
||||
result = host.manage_scheduled_job(key, "run", digest=_jobs(host)[key]["digest"])
|
||||
assert result == {"success": True, "output": "rotated"}
|
||||
|
||||
def test_another_user_runs_through_su(self):
|
||||
assert Host(Path("/x"))._sj_as("www-data", "true") == [
|
||||
"su",
|
||||
"-s",
|
||||
"/bin/sh",
|
||||
"www-data",
|
||||
"-c",
|
||||
"true",
|
||||
]
|
||||
|
||||
|
||||
SPEC = {
|
||||
"name": "netork-tidy",
|
||||
"schedule": "*/30 1-2 * * *",
|
||||
"run_as": "root",
|
||||
"command": "find /tmp -mtime +7 -delete",
|
||||
"enabled": True,
|
||||
"description": "tidy %tmp",
|
||||
}
|
||||
|
||||
|
||||
class TestApplicationJobs:
|
||||
def test_a_timer_on_a_host_with_systemd(self, root: Path):
|
||||
host = Host(root)
|
||||
assert host.put_scheduled_job(SPEC)["output"] == "job:netork-tidy created"
|
||||
timer = (root / "units/netork-tidy.timer").read_text()
|
||||
assert "OnCalendar=*-*-* 01,02:00,30:00" in timer
|
||||
assert "Description=tidy %%tmp" in timer
|
||||
service = (root / "units/netork-tidy.service").read_text()
|
||||
assert f"ExecStart={root}/scripts/netork-tidy" in service and "User=root" in service
|
||||
script = (root / "scripts/netork-tidy").read_text()
|
||||
assert script.splitlines()[1:] == [
|
||||
"# napalm-scheduled-job: netork-tidy tidy %tmp",
|
||||
"# schedule: */30 1-2 * * *",
|
||||
"find /tmp -mtime +7 -delete",
|
||||
]
|
||||
assert "systemctl daemon-reload" in host.log()
|
||||
assert "systemctl enable --now -- netork-tidy.timer" in host.log()
|
||||
|
||||
def test_reboot_is_a_boot_timer(self, root: Path):
|
||||
Host(root).put_scheduled_job({**SPEC, "schedule": "@reboot", "enabled": False})
|
||||
assert "OnBootSec=1min" in (root / "units/netork-tidy.timer").read_text()
|
||||
|
||||
def test_creating_one_that_is_there_and_changing_one_that_is_not(self, root: Path):
|
||||
host = Host(root)
|
||||
with pytest.raises(ScheduledJobChanged):
|
||||
host.put_scheduled_job({**SPEC, "name": "backup"})
|
||||
with pytest.raises(LookupError):
|
||||
host.put_scheduled_job(SPEC, digest="x")
|
||||
with pytest.raises(ScheduledJobChanged):
|
||||
host.put_scheduled_job({**SPEC, "name": "backup"}, digest="x")
|
||||
|
||||
def test_an_unknown_user_is_refused(self, root: Path):
|
||||
with pytest.raises(ScheduledJobRefused, match="no user"):
|
||||
Host(root).put_scheduled_job({**SPEC, "run_as": "no-such-user-here"})
|
||||
|
||||
def test_a_timer_deleted(self, root: Path):
|
||||
host = Host(root)
|
||||
backup = _jobs(host)["job:backup"]
|
||||
host.delete_scheduled_job("job:backup", digest=backup["digest"])
|
||||
assert not (root / "scripts/backup").exists()
|
||||
assert "systemctl disable --now -- backup.timer" in host.log()
|
||||
|
||||
def test_only_an_applications_job_is_deleted(self, root: Path):
|
||||
host = Host(root)
|
||||
apt = _jobs(host)["timer:apt-daily.timer"]
|
||||
with pytest.raises(ScheduledJobRefused):
|
||||
host.delete_scheduled_job(apt["key"], digest=apt["digest"])
|
||||
|
||||
def test_a_cron_d_file_without_systemd(self, root: Path):
|
||||
host = Host(root, systemd=False)
|
||||
host.put_scheduled_job(SPEC)
|
||||
assert (root / "etc/cron.d/netork-tidy").read_text() == (
|
||||
"# napalm-scheduled-job: netork-tidy tidy %tmp\n*/30 1-2 * * * root find /tmp -mtime +7 -delete\n"
|
||||
)
|
||||
job = _jobs(host)["job:netork-tidy"]
|
||||
assert job["managed_by"] == "application" and job["kind"] == "cron"
|
||||
host.delete_scheduled_job("job:netork-tidy", digest=job["digest"])
|
||||
assert not (root / "etc/cron.d/netork-tidy").exists()
|
||||
|
||||
def test_marked_lines_in_the_users_crontab_on_a_busybox_host(self, root: Path):
|
||||
host = Host(root, systemd=False, own_cron_d=False)
|
||||
host.put_scheduled_job({**SPEC, "run_as": "www-data"})
|
||||
text = (root / "spool/www-data").read_text()
|
||||
assert text.endswith(
|
||||
"# napalm-scheduled-job: netork-tidy tidy %tmp\n*/30 1-2 * * * find /tmp -mtime +7 -delete\n"
|
||||
)
|
||||
job = _jobs(host)["job:netork-tidy"]
|
||||
host.put_scheduled_job(
|
||||
{**SPEC, "run_as": "www-data", "enabled": False}, digest=job["digest"]
|
||||
)
|
||||
assert "#napalm-disabled: */30" in (root / "spool/www-data").read_text()
|
||||
job = _jobs(host)["job:netork-tidy"]
|
||||
host.put_scheduled_job({**SPEC, "run_as": "nobody"}, digest=job["digest"])
|
||||
assert "netork-tidy" not in (root / "spool/www-data").read_text()
|
||||
assert "netork-tidy" in (root / "spool/nobody").read_text()
|
||||
job = _jobs(host)["job:netork-tidy"]
|
||||
host.delete_scheduled_job("job:netork-tidy", digest=job["digest"])
|
||||
assert (root / "spool/nobody").read_text() == ""
|
||||
|
||||
def test_a_cron_d_that_is_not_there_is_the_crontab(self, root: Path):
|
||||
host = Host(root, systemd=False)
|
||||
host.OWN_CRON_D_DIR = str(root / "etc/no-cron.d") # type: ignore[misc]
|
||||
host.put_scheduled_job(SPEC)
|
||||
assert "netork-tidy" in (root / "spool/root").read_text()
|
||||
|
||||
def test_a_cron_line_holds_one_line(self, root: Path):
|
||||
with pytest.raises(ScheduledJobRefused, match="one line"):
|
||||
Host(root, systemd=False).put_scheduled_job({**SPEC, "command": "a\nb"})
|
||||
|
||||
def test_a_calendar_systemd_refuses(self, root: Path):
|
||||
_tool(root, "systemd-analyze", "#!/bin/sh\necho 'Failed to parse' >&2\nexit 1\n")
|
||||
with pytest.raises(ScheduledJobRefused, match="systemd does not take"):
|
||||
Host(root).put_scheduled_job(SPEC)
|
||||
|
||||
def test_a_failed_install_says_so(self, root: Path):
|
||||
_tool(root, "install", "#!/bin/sh\necho 'read-only file system' >&2\nexit 1\n")
|
||||
with pytest.raises(RuntimeError, match="read-only"):
|
||||
Host(root).put_scheduled_job(SPEC)
|
||||
@@ -14,6 +14,7 @@ import pytest
|
||||
|
||||
from napalm_device_types.channel import CommandResult
|
||||
from napalm_device_types.ssh_server import (
|
||||
DROPIN_DIR,
|
||||
DROPIN_PATH,
|
||||
MAIN_CONFIG,
|
||||
OpenSshServerMixin,
|
||||
@@ -401,6 +402,32 @@ def test_mixin_falls_back_to_sshd_config_without_an_include():
|
||||
assert host._effective()["mac"] == ["hmac-sha2-256"]
|
||||
|
||||
|
||||
class GeneratedConfigHost(FakeHost):
|
||||
"""A platform that writes sshd_config from its own database (OpenMediaVault)."""
|
||||
|
||||
SSHD_CONFIG_GENERATED = True
|
||||
|
||||
|
||||
def test_generated_sshd_config_still_gets_the_dropin():
|
||||
host = GeneratedConfigHost()
|
||||
result = host.harden_ssh_server({"mac": ["hmac-sha1"]})
|
||||
assert result.applied
|
||||
assert result.path == DROPIN_PATH
|
||||
assert host.files[MAIN_CONFIG] == "Include /etc/ssh/sshd_config.d/*.conf\nPort 22\n"
|
||||
|
||||
|
||||
def test_generated_sshd_config_without_an_include_is_left_alone():
|
||||
"""The platform would write its own file over the change on its next apply."""
|
||||
host = GeneratedConfigHost(main="Port 22\nMACs hmac-sha1,hmac-sha2-256\n")
|
||||
result = host.harden_ssh_server({"mac": ["hmac-sha1"]})
|
||||
assert not result.applied and not result.rolled_back
|
||||
assert result.path is None
|
||||
assert host.files[MAIN_CONFIG] == "Port 22\nMACs hmac-sha1,hmac-sha2-256\n"
|
||||
assert list(host.files) == [MAIN_CONFIG]
|
||||
assert host.reloads == 0
|
||||
assert DROPIN_DIR in result.message
|
||||
|
||||
|
||||
def test_mixin_restores_when_sshd_rejects_the_config():
|
||||
host = FakeHost()
|
||||
host.syntax_ok = False
|
||||
|
||||
@@ -0,0 +1,353 @@
|
||||
"""How an OpenSSH server lets people in: the effective settings and sshd's PAM auth stack.
|
||||
|
||||
The parsers are pure. The getter runs against a fake host that answers ``sshd -T``
|
||||
and the dump of ``/etc/pam.d`` from a dict, so the root read, the PAM includes
|
||||
and a failed read are covered without a machine.
|
||||
|
||||
The ``sshd -T`` texts are abridged from what OpenSSH 9.2p1 on Debian 12 prints
|
||||
(the keywords and their order as sshd writes them; the lines that say nothing
|
||||
about authentication are left out). OpenSSH before 8.7 also prints
|
||||
``challengeresponseauthentication``; OpenBSD's sshd has no ``usepam``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types.channel import CommandResult
|
||||
from napalm_device_types.ssh_server import OpenSshServerMixin
|
||||
from napalm_device_types.ssh_server_auth import (
|
||||
PAM_FILES_COMMAND,
|
||||
SSHD_EFFECTIVE_COMMAND,
|
||||
SshServerAuthMixin,
|
||||
pam_auth_stack,
|
||||
parse_pam_auth,
|
||||
parse_pam_files,
|
||||
parse_sshd_auth,
|
||||
)
|
||||
|
||||
#: Debian 12 as installed: password login on, PAM on, no AuthenticationMethods.
|
||||
SSHD_T_DEBIAN = """\
|
||||
port 22
|
||||
addressfamily any
|
||||
listenaddress [::]:22
|
||||
listenaddress 0.0.0.0:22
|
||||
usepam yes
|
||||
logingracetime 120
|
||||
x11displayoffset 10
|
||||
maxauthtries 6
|
||||
maxsessions 10
|
||||
clientaliveinterval 0
|
||||
clientalivecountmax 3
|
||||
permitrootlogin without-password
|
||||
ignorerhosts yes
|
||||
ignoreuserknownhosts no
|
||||
hostbasedauthentication no
|
||||
hostbasedusesnamefrompacketonly no
|
||||
pubkeyauthentication yes
|
||||
kerberosauthentication no
|
||||
kerberosorlocalpasswd yes
|
||||
kerberosticketcleanup yes
|
||||
gssapiauthentication no
|
||||
gssapicleanupcredentials yes
|
||||
gssapikeyexchange no
|
||||
gssapistrictacceptorcheck yes
|
||||
gssapistorecredentialsonrotate no
|
||||
passwordauthentication yes
|
||||
kbdinteractiveauthentication no
|
||||
printmotd no
|
||||
x11forwarding yes
|
||||
authenticationmethods any
|
||||
kexalgorithms sntrup761x25519-sha512@openssh.com,curve25519-sha256
|
||||
"""
|
||||
|
||||
#: Key only: password and keyboard-interactive off.
|
||||
SSHD_T_KEY_ONLY = SSHD_T_DEBIAN.replace(
|
||||
"passwordauthentication yes", "passwordauthentication no"
|
||||
).replace("permitrootlogin without-password", "permitrootlogin no")
|
||||
|
||||
#: A key and then an OTP through PAM, for everyone.
|
||||
SSHD_T_KEY_AND_OTP = SSHD_T_KEY_ONLY.replace(
|
||||
"kbdinteractiveauthentication no", "kbdinteractiveauthentication yes"
|
||||
).replace("authenticationmethods any", "authenticationmethods publickey,keyboard-interactive")
|
||||
|
||||
#: OpenSSH 8.4p1 (Debian 11): the old keyword beside the new one.
|
||||
SSHD_T_OLD = SSHD_T_KEY_ONLY.replace(
|
||||
"kbdinteractiveauthentication no",
|
||||
"kbdinteractiveauthentication yes\nchallengeresponseauthentication no",
|
||||
)
|
||||
|
||||
#: OpenBSD 7.6: no usepam line.
|
||||
SSHD_T_OPENBSD = SSHD_T_KEY_ONLY.replace("usepam yes\n", "")
|
||||
|
||||
#: Debian 12's /etc/pam.d/sshd, abridged: the auth stack comes from common-auth.
|
||||
PAM_SSHD_DEBIAN = """\
|
||||
# PAM configuration for the Secure Shell service
|
||||
|
||||
# Standard Un*x authentication.
|
||||
@include common-auth
|
||||
|
||||
# Disallow non-root logins when /etc/nologin exists.
|
||||
account required pam_nologin.so
|
||||
|
||||
# Standard Un*x authorization.
|
||||
@include common-account
|
||||
session [success=ok ignore=ignore module_unknown=ignore default=bad] pam_selinux.so close
|
||||
@include common-session
|
||||
@include common-password
|
||||
"""
|
||||
|
||||
PAM_COMMON_AUTH = """\
|
||||
auth\t[success=1 default=ignore]\tpam_unix.so nullok
|
||||
auth\trequisite\t\t\tpam_deny.so
|
||||
auth\trequired\t\t\tpam_permit.so
|
||||
"""
|
||||
|
||||
PAM_COMMON_ACCOUNT = "account [success=1 new_authtok_reqd=done default=ignore] pam_unix.so\n"
|
||||
|
||||
#: The usual way to add a TOTP: a line of its own in sshd's stack.
|
||||
PAM_SSHD_OTP = PAM_SSHD_DEBIAN.replace(
|
||||
"@include common-auth",
|
||||
"#@include common-auth\nauth required pam_google_authenticator.so nullok",
|
||||
)
|
||||
|
||||
#: Red Hat's form: ``auth substack password-auth``, a module by its full path.
|
||||
PAM_SSHD_RHEL = """\
|
||||
#%PAM-1.0
|
||||
auth substack password-auth
|
||||
auth include postlogin
|
||||
-auth optional /usr/lib64/security/pam_reauthorize.so prepare
|
||||
account required pam_sepermit.so
|
||||
"""
|
||||
PAM_PASSWORD_AUTH = "auth required pam_env.so\nauth sufficient pam_unix.so try_first_pass nullok\n"
|
||||
PAM_POSTLOGIN = "session optional pam_umask.so silent\n"
|
||||
|
||||
|
||||
# ── parse_sshd_auth ──────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_parse_reads_the_settings_that_decide_how_people_log_in():
|
||||
auth = parse_sshd_auth(SSHD_T_DEBIAN)
|
||||
assert auth == {
|
||||
"password_authentication": True,
|
||||
"kbd_interactive_authentication": False,
|
||||
"pubkey_authentication": True,
|
||||
"hostbased_authentication": False,
|
||||
"gssapi_authentication": False,
|
||||
"authentication_methods": [],
|
||||
"permit_root_login": "without-password",
|
||||
"use_pam": True,
|
||||
"pam_auth_stack": None,
|
||||
}
|
||||
|
||||
|
||||
def test_parse_splits_authentication_methods_into_lists_of_methods():
|
||||
auth = parse_sshd_auth(
|
||||
SSHD_T_KEY_ONLY.replace(
|
||||
"authenticationmethods any",
|
||||
"authenticationmethods publickey,keyboard-interactive:pam publickey,password",
|
||||
)
|
||||
)
|
||||
assert auth["authentication_methods"] == [
|
||||
["publickey", "keyboard-interactive:pam"],
|
||||
["publickey", "password"],
|
||||
]
|
||||
|
||||
|
||||
def test_parse_counts_the_old_challenge_response_keyword_as_keyboard_interactive():
|
||||
"""Before 8.7 either keyword could switch PAM's prompts on: either one is enough."""
|
||||
assert parse_sshd_auth(SSHD_T_OLD)["kbd_interactive_authentication"] is True
|
||||
old_only = SSHD_T_KEY_ONLY.replace(
|
||||
"kbdinteractiveauthentication no",
|
||||
"kbdinteractiveauthentication no\nchallengeresponseauthentication yes",
|
||||
)
|
||||
assert parse_sshd_auth(old_only)["kbd_interactive_authentication"] is True
|
||||
|
||||
|
||||
def test_parse_says_none_for_usepam_where_sshd_has_no_pam():
|
||||
assert parse_sshd_auth(SSHD_T_OPENBSD)["use_pam"] is None
|
||||
|
||||
|
||||
def test_parse_takes_a_method_sshd_was_built_without_as_off():
|
||||
"""sshd built without GSSAPI prints no gssapiauthentication line: that method cannot be used."""
|
||||
text = SSHD_T_DEBIAN.replace("gssapiauthentication no\n", "")
|
||||
assert parse_sshd_auth(text)["gssapi_authentication"] is False
|
||||
|
||||
|
||||
def test_parse_refuses_output_that_is_not_sshd_t():
|
||||
with pytest.raises(ValueError, match="passwordauthentication"):
|
||||
parse_sshd_auth("sshd: no hostkeys available -- exiting.\n")
|
||||
|
||||
|
||||
# ── PAM ──────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _pairs(entries) -> list[tuple[str, str]]:
|
||||
return [(e["control"], e["module"]) for e in entries]
|
||||
|
||||
|
||||
def _modules(stack) -> list[str]:
|
||||
return [e["module"] for e in stack]
|
||||
|
||||
|
||||
def test_pam_auth_lists_modules_and_includes_in_stack_order():
|
||||
# Every @include: which of them hold auth lines is for the included file to say.
|
||||
assert _pairs(parse_pam_auth(PAM_SSHD_DEBIAN)) == [
|
||||
("include", "common-auth"),
|
||||
("include", "common-account"),
|
||||
("include", "common-session"),
|
||||
("include", "common-password"),
|
||||
]
|
||||
|
||||
|
||||
def test_pam_auth_reads_bracketed_controls_paths_and_optional_lines():
|
||||
assert _pairs(parse_pam_auth(PAM_SSHD_RHEL)) == [
|
||||
("substack", "password-auth"),
|
||||
("include", "postlogin"),
|
||||
("optional", "pam_reauthorize.so"),
|
||||
]
|
||||
assert parse_pam_auth(PAM_COMMON_AUTH) == [
|
||||
{"module": "pam_unix.so", "control": "[success=1 default=ignore]", "args": ["nullok"]},
|
||||
{"module": "pam_deny.so", "control": "requisite", "args": []},
|
||||
{"module": "pam_permit.so", "control": "required", "args": []},
|
||||
]
|
||||
|
||||
|
||||
def test_pam_auth_keeps_a_modules_arguments():
|
||||
"""``nullok`` lets a user without an OTP secret through: the consumer has to see it."""
|
||||
assert parse_pam_auth(PAM_SSHD_OTP)[0] == {
|
||||
"module": "pam_google_authenticator.so",
|
||||
"control": "required",
|
||||
"args": ["nullok"],
|
||||
}
|
||||
|
||||
|
||||
def test_pam_auth_skips_comments_and_other_types():
|
||||
assert _pairs(parse_pam_auth(PAM_SSHD_OTP)) == [
|
||||
("required", "pam_google_authenticator.so"),
|
||||
("include", "common-account"),
|
||||
("include", "common-session"),
|
||||
("include", "common-password"),
|
||||
]
|
||||
|
||||
|
||||
def test_pam_auth_ignores_an_include_that_is_not_a_service_name():
|
||||
assert parse_pam_auth("auth include ../../etc/shadow\n@include\nauth required\n") == []
|
||||
|
||||
|
||||
def _dump(files: dict[str, str]) -> str:
|
||||
"""What ``tail -n +1`` prints for several files: a header before each."""
|
||||
return "\n".join(f"==> /etc/pam.d/{name} <==\n{text}" for name, text in files.items())
|
||||
|
||||
|
||||
def test_pam_files_splits_the_dump_by_file():
|
||||
files = {"sshd": PAM_SSHD_DEBIAN, "common-auth": PAM_COMMON_AUTH}
|
||||
assert parse_pam_files(_dump(files)) == files
|
||||
|
||||
|
||||
def test_auth_stack_follows_includes_and_substacks_in_stack_order():
|
||||
files = {
|
||||
"sshd": PAM_SSHD_RHEL + "auth include password-auth\n",
|
||||
"password-auth": PAM_PASSWORD_AUTH,
|
||||
"postlogin": PAM_POSTLOGIN,
|
||||
}
|
||||
assert _modules(pam_auth_stack(files)) == ["pam_env.so", "pam_unix.so", "pam_reauthorize.so"]
|
||||
|
||||
|
||||
def test_auth_stack_stops_at_an_include_loop_and_skips_a_missing_file():
|
||||
files = {
|
||||
"sshd": "@include a\n@include gone\n",
|
||||
"a": "@include sshd\nauth required pam_unix.so\n",
|
||||
}
|
||||
assert _modules(pam_auth_stack(files)) == ["pam_unix.so"]
|
||||
|
||||
|
||||
def test_auth_stack_is_none_without_a_file_for_the_service():
|
||||
assert pam_auth_stack({"login": PAM_COMMON_AUTH}) is None
|
||||
|
||||
|
||||
# ── the getter ───────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
class _Host(SshServerAuthMixin):
|
||||
"""Answers ``sshd -T`` (root only) and the dump of /etc/pam.d."""
|
||||
|
||||
def __init__(self, sshd_t: str | None, pam: dict[str, str] | None = None) -> None:
|
||||
self.sshd_t = sshd_t
|
||||
self.pam = pam
|
||||
self.calls: list[tuple[str, bool]] = []
|
||||
|
||||
def run_command(
|
||||
self,
|
||||
command: str,
|
||||
*,
|
||||
privileged: bool = False,
|
||||
timeout: float = 60,
|
||||
stdin: bytes | None = None,
|
||||
) -> CommandResult:
|
||||
self.calls.append((command, privileged))
|
||||
if command == SSHD_EFFECTIVE_COMMAND:
|
||||
if self.sshd_t is None or not privileged:
|
||||
return CommandResult("", "sshd: no hostkeys available -- exiting.", 1)
|
||||
return CommandResult(self.sshd_t, "", 0)
|
||||
assert command == PAM_FILES_COMMAND, command
|
||||
if self.pam is None:
|
||||
return CommandResult("", "tail: cannot open '/etc/pam.d/*' for reading", 1)
|
||||
return CommandResult(_dump(self.pam), "", 0)
|
||||
|
||||
|
||||
def test_getter_reads_sshd_t_as_root_and_pam_without():
|
||||
host = _Host(
|
||||
SSHD_T_DEBIAN,
|
||||
{
|
||||
"sshd": PAM_SSHD_DEBIAN,
|
||||
"common-auth": PAM_COMMON_AUTH,
|
||||
"common-account": PAM_COMMON_ACCOUNT,
|
||||
},
|
||||
)
|
||||
auth = host.get_ssh_server_auth()
|
||||
assert auth["password_authentication"] is True
|
||||
assert _modules(auth["pam_auth_stack"]) == ["pam_unix.so", "pam_deny.so", "pam_permit.so"]
|
||||
# sshd -T once, as root; PAM's files once, as the login user.
|
||||
assert host.calls == [(SSHD_EFFECTIVE_COMMAND, True), (PAM_FILES_COMMAND, False)]
|
||||
|
||||
|
||||
def test_getter_names_an_otp_module_in_sshds_own_stack():
|
||||
host = _Host(SSHD_T_KEY_AND_OTP, {"sshd": PAM_SSHD_OTP})
|
||||
stack = host.get_ssh_server_auth()["pam_auth_stack"]
|
||||
assert _modules(stack) == ["pam_google_authenticator.so"]
|
||||
|
||||
|
||||
def test_getter_keeps_what_it_read_when_one_pam_file_is_unreadable():
|
||||
"""tail goes on past a file it cannot open and exits 1: the rest is still read."""
|
||||
|
||||
class _Partial(_Host):
|
||||
def run_command(self, command: str, **kwargs) -> CommandResult: # type: ignore[override]
|
||||
result = super().run_command(command, **kwargs)
|
||||
if command == PAM_FILES_COMMAND:
|
||||
return CommandResult(result.stdout, "tail: cannot open 'x'", 1)
|
||||
return result
|
||||
|
||||
host = _Partial(SSHD_T_DEBIAN, {"sshd": "auth required pam_unix.so\n"})
|
||||
assert _modules(host.get_ssh_server_auth()["pam_auth_stack"]) == ["pam_unix.so"]
|
||||
|
||||
|
||||
def test_getter_leaves_pam_unread_without_usepam():
|
||||
host = _Host(SSHD_T_OPENBSD)
|
||||
auth = host.get_ssh_server_auth()
|
||||
assert auth["pam_auth_stack"] is None
|
||||
assert host.calls == [(SSHD_EFFECTIVE_COMMAND, True)]
|
||||
|
||||
|
||||
def test_getter_says_none_when_pam_cannot_be_read():
|
||||
assert _Host(SSHD_T_DEBIAN).get_ssh_server_auth()["pam_auth_stack"] is None
|
||||
|
||||
|
||||
def test_getter_raises_when_sshd_t_fails():
|
||||
with pytest.raises(RuntimeError, match="no hostkeys"):
|
||||
_Host(None).get_ssh_server_auth()
|
||||
|
||||
|
||||
def test_the_hardening_mixin_carries_the_reader():
|
||||
"""Every driver that can harden sshd can also say how it lets people in."""
|
||||
assert issubclass(OpenSshServerMixin, SshServerAuthMixin)
|
||||
@@ -0,0 +1,245 @@
|
||||
"""TimeSyncMixin: a device's time sources and time zone, read and set (NetOrk/netork#2361).
|
||||
|
||||
The application keeps one list of NTP servers and a zone per site, and aligns a
|
||||
device that differs. ``get_time_sync`` is declared only, so ``callable(getattr(
|
||||
driver, "get_time_sync", None))`` is the truthful answer; ``configure_time_sync``
|
||||
is a template of steps like every fix, and an API driver overrides it whole.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import inspect
|
||||
from typing import ClassVar, get_type_hints
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import (
|
||||
TIME_SYNC_VALUES,
|
||||
AccessPointDriver,
|
||||
DeviceTypeDriver,
|
||||
FirewallDriver,
|
||||
OSDriver,
|
||||
SwitchDriver,
|
||||
TimeSyncDict,
|
||||
TimeSyncMixin,
|
||||
UnknownTimezone,
|
||||
check_ntp_servers,
|
||||
)
|
||||
|
||||
|
||||
def test_the_shape():
|
||||
hints = get_type_hints(TimeSyncDict)
|
||||
assert hints == {
|
||||
"ntp_servers": list[str],
|
||||
"timezone": str | None,
|
||||
"synchronized": bool | None,
|
||||
"daemon": str | None,
|
||||
"dhcp_ntp_servers": list[str] | None,
|
||||
}
|
||||
|
||||
|
||||
def test_the_dhcp_servers_are_optional():
|
||||
# A driver that cannot tell what DHCP brought leaves the key out (NetOrk/netork#2380).
|
||||
assert TimeSyncDict.__required_keys__ == {"ntp_servers", "timezone", "synchronized", "daemon"}
|
||||
assert TimeSyncDict.__optional_keys__ == {"dhcp_ntp_servers"}
|
||||
|
||||
|
||||
def test_the_two_values_a_driver_may_write():
|
||||
assert frozenset({"ntp_servers", "timezone"}) == TIME_SYNC_VALUES
|
||||
|
||||
|
||||
def test_no_role_demands_it():
|
||||
for role in (AccessPointDriver, FirewallDriver, OSDriver, SwitchDriver, DeviceTypeDriver):
|
||||
assert not issubclass(role, TimeSyncMixin)
|
||||
|
||||
|
||||
def test_the_reading_is_declared_not_implemented():
|
||||
assert not hasattr(TimeSyncMixin, "get_time_sync")
|
||||
source = inspect.getsource(TimeSyncMixin)
|
||||
assert "def get_time_sync(self) -> TimeSyncDict" in source
|
||||
assert "time_sync_writes: ClassVar[frozenset[str]]" in source
|
||||
|
||||
|
||||
def test_the_template_takes_both_values_or_none_of_them():
|
||||
signature = inspect.signature(TimeSyncMixin.configure_time_sync)
|
||||
assert list(signature.parameters) == ["self", "ntp_servers", "timezone"]
|
||||
assert signature.parameters["ntp_servers"].default is None
|
||||
assert signature.parameters["timezone"].default is None
|
||||
|
||||
|
||||
class _Device(TimeSyncMixin):
|
||||
"""A device that keeps its values in memory and reads back what it was given."""
|
||||
|
||||
time_sync_writes: ClassVar[frozenset[str]] = frozenset({"ntp_servers", "timezone"})
|
||||
|
||||
def __init__(self, *, keeps_zone_name: bool = True, fail: str = "") -> None:
|
||||
self.state: TimeSyncDict = {
|
||||
"ntp_servers": ["0.pool.ntp.org"],
|
||||
"timezone": "UTC",
|
||||
"synchronized": True,
|
||||
"daemon": None,
|
||||
}
|
||||
self.keeps_zone_name = keeps_zone_name
|
||||
self.fail = fail
|
||||
self.calls: list[tuple[str, object]] = []
|
||||
|
||||
def get_time_sync(self) -> TimeSyncDict:
|
||||
state = dict(self.state)
|
||||
if not self.keeps_zone_name:
|
||||
state["timezone"] = None
|
||||
return state # type: ignore[return-value]
|
||||
|
||||
def _write_ntp_servers(self, servers):
|
||||
self.calls.append(("ntp_servers", servers))
|
||||
if self.fail == "ntp_servers":
|
||||
return {"ok": False, "output": "refused"}
|
||||
self.state["ntp_servers"] = list(servers)
|
||||
return {"ok": True, "output": "written"}
|
||||
|
||||
def _write_time_zone(self, zone):
|
||||
self.calls.append(("timezone", zone))
|
||||
self.state["timezone"] = zone
|
||||
return {"ok": True, "output": f"zone {zone}"}
|
||||
|
||||
|
||||
class TestConfigureTimeSync:
|
||||
def test_both_values_then_a_check(self):
|
||||
device = _Device()
|
||||
result = device.configure_time_sync(["ntp1.example.org", "10.0.0.1"], "Europe/Berlin")
|
||||
|
||||
assert result["success"] is True
|
||||
assert [s["step"] for s in result["steps"]] == ["ntp_servers", "timezone", "verify"]
|
||||
assert device.calls == [
|
||||
("ntp_servers", ["ntp1.example.org", "10.0.0.1"]),
|
||||
("timezone", "Europe/Berlin"),
|
||||
]
|
||||
|
||||
def test_none_leaves_a_value_alone(self):
|
||||
device = _Device()
|
||||
result = device.configure_time_sync(timezone="Europe/Berlin")
|
||||
|
||||
assert result["success"] is True
|
||||
assert device.calls == [("timezone", "Europe/Berlin")]
|
||||
assert device.state["ntp_servers"] == ["0.pool.ntp.org"]
|
||||
|
||||
def test_servers_alone(self):
|
||||
device = _Device()
|
||||
assert device.configure_time_sync(["ntp1.example.org"])["success"] is True
|
||||
assert device.calls == [("ntp_servers", ["ntp1.example.org"])]
|
||||
|
||||
def test_an_unknown_zone_is_refused_before_anything_is_written(self):
|
||||
device = _Device()
|
||||
with pytest.raises(UnknownTimezone, match="Mars/Olympus_Mons"):
|
||||
device.configure_time_sync(["ntp1.example.org"], "Mars/Olympus_Mons")
|
||||
assert device.calls == []
|
||||
|
||||
def test_a_posix_rule_is_not_a_zone_on_the_wire(self):
|
||||
with pytest.raises(UnknownTimezone):
|
||||
_Device().configure_time_sync(timezone="CET-1CEST,M3.5.0,M10.5.0/3")
|
||||
|
||||
def test_a_value_the_driver_cannot_write_is_refused_before_anything_is_written(self):
|
||||
class Bmc(_Device):
|
||||
time_sync_writes = frozenset({"ntp_servers"})
|
||||
|
||||
device = Bmc()
|
||||
with pytest.raises(ValueError, match="timezone"):
|
||||
device.configure_time_sync(["ntp1.example.org"], "Europe/Berlin")
|
||||
assert device.calls == []
|
||||
|
||||
def test_a_driver_that_names_nothing_it_writes_writes_nothing(self):
|
||||
class Silent(TimeSyncMixin):
|
||||
"""Mixes the contract in and never says what it writes."""
|
||||
|
||||
with pytest.raises(ValueError, match="ntp_servers"):
|
||||
Silent().configure_time_sync(["ntp1.example.org"])
|
||||
|
||||
def test_nothing_asked_is_a_mistake(self):
|
||||
with pytest.raises(ValueError, match="nothing"):
|
||||
_Device().configure_time_sync()
|
||||
|
||||
def test_a_failed_step_stops_the_rest(self):
|
||||
device = _Device(fail="ntp_servers")
|
||||
result = device.configure_time_sync(["ntp1.example.org"], "Europe/Berlin")
|
||||
|
||||
assert result["success"] is False
|
||||
assert [s["step"] for s in result["steps"]] == ["ntp_servers"]
|
||||
assert device.calls == [("ntp_servers", ["ntp1.example.org"])]
|
||||
|
||||
def test_the_check_reads_back_what_was_written(self):
|
||||
class Stubborn(_Device):
|
||||
def _write_ntp_servers(self, servers):
|
||||
self.calls.append(("ntp_servers", servers))
|
||||
return {"ok": True, "output": "written"} # and nothing changed
|
||||
|
||||
result = Stubborn().configure_time_sync(["ntp1.example.org"])
|
||||
|
||||
assert result["success"] is False
|
||||
verify = result["steps"][-1]
|
||||
assert verify["step"] == "verify"
|
||||
assert "0.pool.ntp.org" in verify["output"]
|
||||
|
||||
def test_the_check_names_the_zone_the_device_kept(self):
|
||||
class Elsewhere(_Device):
|
||||
def _write_time_zone(self, zone):
|
||||
return {"ok": True, "output": "set"} # and the device stayed on UTC
|
||||
|
||||
result = Elsewhere().configure_time_sync(timezone="Europe/Berlin")
|
||||
|
||||
assert result["success"] is False
|
||||
assert result["steps"][-1]["output"] == "the device is in the time zone UTC"
|
||||
|
||||
def test_servers_compare_without_case(self):
|
||||
class Lower(_Device):
|
||||
def get_time_sync(self):
|
||||
state = super().get_time_sync()
|
||||
state["ntp_servers"] = [s.lower() for s in state["ntp_servers"]]
|
||||
return state
|
||||
|
||||
assert Lower().configure_time_sync(["NTP1.Example.org"])["success"] is True
|
||||
|
||||
def test_servers_from_dhcp_do_not_fail_the_check(self):
|
||||
# DHCP option 42 counts: what the device got by DHCP stays and is no mistake.
|
||||
device = _Device()
|
||||
device.state["dhcp_ntp_servers"] = ["192.168.1.1"]
|
||||
result = device.configure_time_sync(["ntp1.example.org"])
|
||||
assert result["success"] is True, result["output"]
|
||||
|
||||
def test_a_device_without_a_zone_name_is_not_checked_for_one(self):
|
||||
# A switch keeps an offset and a summer-time rule, not a name: it reads None.
|
||||
result = _Device(keeps_zone_name=False).configure_time_sync(timezone="Europe/Berlin")
|
||||
assert result["success"] is True
|
||||
|
||||
|
||||
class TestCheckNtpServers:
|
||||
def test_in_order_stripped_each_once(self):
|
||||
assert check_ntp_servers([" ntp1.example.org", "10.0.0.1", "NTP1.example.org", "::1"]) == [
|
||||
"ntp1.example.org",
|
||||
"10.0.0.1",
|
||||
"::1",
|
||||
]
|
||||
|
||||
def test_none_at_all_is_refused(self):
|
||||
# A device without a source drifts; leaving the value alone is None, not [].
|
||||
with pytest.raises(ValueError, match="at least one"):
|
||||
check_ntp_servers([])
|
||||
with pytest.raises(ValueError, match="at least one"):
|
||||
check_ntp_servers([" "])
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"bad",
|
||||
[
|
||||
"ntp1.example.org iburst",
|
||||
"-ntp.example.org",
|
||||
"ntp1.example.org\nserver evil",
|
||||
"ntp;reboot",
|
||||
"$(id)",
|
||||
"a" * 254,
|
||||
],
|
||||
)
|
||||
def test_only_a_host_name_or_an_address(self, bad):
|
||||
with pytest.raises(ValueError, match="not a host name or address"):
|
||||
check_ntp_servers([bad])
|
||||
|
||||
def test_a_string_is_not_a_list_of_servers(self):
|
||||
with pytest.raises(TypeError):
|
||||
check_ntp_servers("ntp1.example.org") # type: ignore[arg-type]
|
||||
@@ -0,0 +1,656 @@
|
||||
"""Time sync on a POSIX host, over the command channel: chrony, systemd-timesyncd, timedatectl.
|
||||
|
||||
The reading is run by a real ``sh`` over a directory tree that stands in for the
|
||||
host, with ``timedatectl``, ``systemctl`` and ``chronyc`` replaced by scripts that
|
||||
keep a log -- so the shell itself is tested, not a hand-written copy of its output.
|
||||
``sed`` and ``install`` are the real ones, so the files the writes leave are what a
|
||||
host would get (NetOrk/netork#2361). Servers that DHCP brings are read apart and left
|
||||
alone by a write (NetOrk/netork#2380).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import PosixTimeSyncMixin, UnknownTimezone, parse_time_sync
|
||||
from napalm_device_types.channel import CommandResult
|
||||
from napalm_device_types.time_sync_posix import (
|
||||
chrony_synchronized,
|
||||
parse_key_values,
|
||||
timesyncd_link_servers,
|
||||
timesyncd_servers,
|
||||
)
|
||||
|
||||
# What the tools print, as recorded.
|
||||
|
||||
TIMEDATECTL = """\
|
||||
Timezone=Europe/Berlin
|
||||
LocalRTC=no
|
||||
CanNTP=yes
|
||||
NTP=yes
|
||||
NTPSynchronized=yes
|
||||
TimeUSec=Wed 2026-11-11 14:23:16 CET
|
||||
RTCTimeUSec=Wed 2026-11-11 13:23:16 CET
|
||||
"""
|
||||
|
||||
#: Ubuntu 24.04 without any server configured: only the compiled-in fallback.
|
||||
TIMESYNC_FALLBACK = """\
|
||||
FallbackNTPServers=ntp.ubuntu.com
|
||||
ServerName=ntp.ubuntu.com
|
||||
ServerAddress=91.189.91.157
|
||||
RootDistanceMaxUSec=5s
|
||||
PollIntervalMinUSec=32s
|
||||
PollIntervalMaxUSec=34min 8s
|
||||
PollIntervalUSec=34min 8s
|
||||
NTPMessage={ Leap=0, Version=4, Mode=4, Stratum=2, Precision=-24, RootDelay=42.190ms, RootDispersion=1.190ms, Reference=84A36001, Ignored=no, PacketCount=119, Jitter=7.481ms }
|
||||
Frequency=455471
|
||||
"""
|
||||
|
||||
TIMESYNC_CONFIGURED = """\
|
||||
LinkNTPServers=192.168.1.1
|
||||
SystemNTPServers=ntp1.example.org ntp2.example.org
|
||||
FallbackNTPServers=0.debian.pool.ntp.org 1.debian.pool.ntp.org
|
||||
ServerName=ntp1.example.org
|
||||
"""
|
||||
|
||||
CHRONYC_SYNCED = """\
|
||||
MS Name/IP address Stratum Poll Reach LastRx Last sample
|
||||
===============================================================================
|
||||
^- 162.159.200.123 3 6 377 41 +814us[ +814us] +/- 14ms
|
||||
^* 192.168.1.1 2 6 377 40 -45us[ -63us] +/- 10ms
|
||||
"""
|
||||
|
||||
CHRONYC_UNSYNCED = """\
|
||||
MS Name/IP address Stratum Poll Reach LastRx Last sample
|
||||
===============================================================================
|
||||
^? 162.159.200.123 0 6 0 - +0ns[ +0ns] +/- 0ns
|
||||
"""
|
||||
|
||||
#: Debian 12's chrony.conf, comments shortened, its directories under the test's root.
|
||||
DEBIAN_CHRONY_CONF = """\
|
||||
# Welcome to the chrony configuration file. See chrony.conf(5) for more
|
||||
# information about usable directives.
|
||||
|
||||
# Include configuration files found in /etc/chrony/conf.d.
|
||||
confdir {root}/etc/chrony/conf.d
|
||||
|
||||
# Use Debian vendor zone.
|
||||
pool 2.debian.pool.ntp.org iburst
|
||||
|
||||
# Use time sources from DHCP.
|
||||
sourcedir {root}/run/chrony-dhcp
|
||||
|
||||
# Use NTP sources found in /etc/chrony/sources.d.
|
||||
sourcedir {root}/etc/chrony/sources.d
|
||||
|
||||
keyfile /etc/chrony/chrony.keys
|
||||
driftfile /var/lib/chrony/chrony.drift
|
||||
rtcsync
|
||||
makestep 1 3
|
||||
leapsectz right/UTC
|
||||
"""
|
||||
|
||||
TIMEDATECTL_TOOL = """#!/bin/sh
|
||||
echo "timedatectl $*" >> "$LOG"
|
||||
case "$1" in
|
||||
show) cat "$FIX/timedatectl" ;;
|
||||
show-timesync) [ -f "$FIX/timesync" ] && cat "$FIX/timesync" ;;
|
||||
set-timezone)
|
||||
[ "$2" = "$REFUSE_ZONE" ] && { echo "Failed to set time zone: Invalid or not installed time zone '$2'" >&2; exit 1; }
|
||||
sed -i "s#^Timezone=.*#Timezone=$2#" "$FIX/timedatectl" ;;
|
||||
esac
|
||||
"""
|
||||
|
||||
SYSTEMCTL_TOOL = """#!/bin/sh
|
||||
echo "systemctl $*" >> "$LOG"
|
||||
case "$1" in
|
||||
is-active)
|
||||
u=${2%.service}
|
||||
case " $ACTIVE " in *" $u "*) echo active ;; *) echo inactive; exit 3 ;; esac ;;
|
||||
restart)
|
||||
[ -n "$FAIL_RESTART" ] && { echo "Job for $2 failed." >&2; exit 1; }
|
||||
if [ "$2" = systemd-timesyncd.service ]; then
|
||||
servers=$(sed -n 's/^NTP=\\(..*\\)$/\\1/p' "$DROPIN")
|
||||
printf 'SystemNTPServers=%s\\n' "$servers" > "$FIX/timesync"
|
||||
fi ;;
|
||||
esac
|
||||
"""
|
||||
|
||||
CHRONYC_TOOL = """#!/bin/sh
|
||||
echo "chronyc $*" >> "$LOG"
|
||||
case "$*" in
|
||||
"-n sources") [ -f "$FIX/chronyc" ] && cat "$FIX/chronyc" ;;
|
||||
"reload sources") exit "${RELOAD_EXIT:-0}" ;;
|
||||
esac
|
||||
"""
|
||||
|
||||
|
||||
class Host(PosixTimeSyncMixin):
|
||||
"""A host in a directory; every command is run by sh, root or not."""
|
||||
|
||||
def __init__(self, root: Path, *, active: str = "chrony") -> None:
|
||||
self.root = root
|
||||
etc = root / "etc"
|
||||
self.CHRONY_CONFIGS = (str(etc / "chrony/chrony.conf"), str(etc / "chrony.conf")) # type: ignore[misc]
|
||||
self.CHRONY_SOURCES_FILE = str(etc / "chrony/sources.d/netork.sources") # type: ignore[misc]
|
||||
self.CHRONY_INCLUDE_FILE = str(etc / "chrony.d/netork.conf") # type: ignore[misc]
|
||||
self.CHRONY_DHCP_DIRS = (str(root / "run/chrony-dhcp"),) # type: ignore[misc]
|
||||
self.TIMESYNCD_DROPIN = str(etc / "systemd/timesyncd.conf.d/netork.conf") # type: ignore[misc]
|
||||
self.NTPD_CONFIGS = (str(etc / "ntpsec/ntp.conf"), str(etc / "ntp.conf")) # type: ignore[misc]
|
||||
self.OPENNTPD_CONFIGS = (str(etc / "openntpd/ntpd.conf"),) # type: ignore[misc]
|
||||
self.TIME_ZONE_SH = ( # type: ignore[misc]
|
||||
f"cat {etc}/timezone 2>/dev/null || readlink {etc}/localtime 2>/dev/null"
|
||||
)
|
||||
self.env = {
|
||||
"PATH": f"{root / 'bin'}:{os.environ['PATH']}",
|
||||
"LOG": str(root / "log"),
|
||||
"FIX": str(root / "fix"),
|
||||
"ACTIVE": active,
|
||||
"DROPIN": self.TIMESYNCD_DROPIN,
|
||||
}
|
||||
self.commands: list[tuple[str, bool]] = []
|
||||
|
||||
def run_command(self, command, *, privileged=False, timeout=60, stdin=None): # type: ignore[no-untyped-def]
|
||||
self.commands.append((command, privileged))
|
||||
done = subprocess.run(
|
||||
["sh", "-c", command],
|
||||
input=stdin,
|
||||
capture_output=True,
|
||||
env={**os.environ, **self.env},
|
||||
timeout=30,
|
||||
)
|
||||
return CommandResult(done.stdout.decode(), done.stderr.decode(), done.returncode)
|
||||
|
||||
def log(self) -> list[str]:
|
||||
log = self.root / "log"
|
||||
return log.read_text().splitlines() if log.exists() else []
|
||||
|
||||
def privileged(self) -> list[str]:
|
||||
return [c for c, p in self.commands if p]
|
||||
|
||||
|
||||
def _tool(root: Path, name: str, body: str) -> None:
|
||||
path = root / "bin" / name
|
||||
path.write_text(body)
|
||||
path.chmod(0o755)
|
||||
|
||||
|
||||
def _write(path: Path, text: str) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(text)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def root(tmp_path: Path) -> Path:
|
||||
for d in ("bin", "fix", "etc"):
|
||||
(tmp_path / d).mkdir()
|
||||
(tmp_path / "fix/timedatectl").write_text(TIMEDATECTL)
|
||||
for name, body in (
|
||||
("timedatectl", TIMEDATECTL_TOOL),
|
||||
("systemctl", SYSTEMCTL_TOOL),
|
||||
("chronyc", CHRONYC_TOOL),
|
||||
):
|
||||
_tool(tmp_path, name, body)
|
||||
return tmp_path
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def debian(root: Path) -> Path:
|
||||
"""chrony as Debian 12 ships it, with a source in each place it reads."""
|
||||
_write(root / "etc/chrony/chrony.conf", DEBIAN_CHRONY_CONF.format(root=root))
|
||||
_write(root / "etc/chrony/conf.d/local.conf", "server ntp-conf.example.org iburst\n")
|
||||
_write(root / "run/chrony-dhcp/eth0.sources", "server 192.168.1.1 iburst\n")
|
||||
_write(root / "etc/chrony/sources.d/other.sources", "server ntp-other.example.org\n")
|
||||
(root / "fix/chronyc").write_text(CHRONYC_SYNCED)
|
||||
return root
|
||||
|
||||
|
||||
def _no_timedatectl(root: Path) -> None:
|
||||
"""A host without systemd: the command is not there (the build host's must not answer)."""
|
||||
_tool(root, "timedatectl", '#!/bin/sh\necho "sh: timedatectl: not found" >&2\nexit 127\n')
|
||||
|
||||
|
||||
def _text(path: Path) -> str:
|
||||
return path.read_text()
|
||||
|
||||
|
||||
class TestParsing:
|
||||
def test_key_values(self):
|
||||
values = parse_key_values(TIMEDATECTL)
|
||||
assert values["Timezone"] == "Europe/Berlin"
|
||||
assert values["NTPSynchronized"] == "yes"
|
||||
|
||||
def test_timesyncd_uses_the_fallback_only_when_nothing_else_is_known(self):
|
||||
assert timesyncd_servers(parse_key_values(TIMESYNC_FALLBACK)) == ["ntp.ubuntu.com"]
|
||||
# runtime, then system: the configured ones; a link's servers are DHCP's.
|
||||
assert timesyncd_servers(parse_key_values(TIMESYNC_CONFIGURED)) == [
|
||||
"ntp1.example.org",
|
||||
"ntp2.example.org",
|
||||
]
|
||||
assert timesyncd_servers({}) == []
|
||||
|
||||
def test_timesyncd_does_not_fall_back_while_a_link_brings_servers(self):
|
||||
values = {"LinkNTPServers": "192.168.1.1", "FallbackNTPServers": "ntp.ubuntu.com"}
|
||||
assert timesyncd_servers(values) == []
|
||||
assert timesyncd_link_servers(values) == ["192.168.1.1"]
|
||||
|
||||
def test_timesyncd_link_servers(self):
|
||||
assert timesyncd_link_servers(parse_key_values(TIMESYNC_CONFIGURED)) == ["192.168.1.1"]
|
||||
assert timesyncd_link_servers({"LinkNTPServers": "10.0.0.1 10.0.0.1 10.0.0.2"}) == [
|
||||
"10.0.0.1",
|
||||
"10.0.0.2",
|
||||
]
|
||||
assert timesyncd_link_servers(parse_key_values(TIMESYNC_FALLBACK)) == []
|
||||
|
||||
def test_chrony_is_synchronised_while_it_has_selected_a_source(self):
|
||||
assert chrony_synchronized(CHRONYC_SYNCED) is True
|
||||
assert chrony_synchronized(CHRONYC_UNSYNCED) is False
|
||||
assert chrony_synchronized("") is None
|
||||
|
||||
def test_chrony_without_a_main_configuration_has_no_sources(self):
|
||||
reading = parse_time_sync("@@ts active\nchrony chrony active\n@@ts end\n")
|
||||
assert (reading["ntp_servers"], reading["dhcp_ntp_servers"]) == ([], [])
|
||||
|
||||
def test_the_dhcp_directory_is_debian_s_and_red_hat_s_by_default(self):
|
||||
reading = parse_time_sync(
|
||||
"@@ts active\nchrony chrony active\n"
|
||||
"@@ts chrony 0 main /etc/chrony/chrony.conf\nsourcedir /run/chrony-dhcp\n"
|
||||
"@@ts chrony 1 sourcedir /run/chrony-dhcp/eth0.sources\nserver 10.0.0.1 iburst\n"
|
||||
"@@ts end\n"
|
||||
)
|
||||
assert (reading["ntp_servers"], reading["dhcp_ntp_servers"]) == ([], ["10.0.0.1"])
|
||||
|
||||
def test_a_reading_cut_short_raises(self):
|
||||
with pytest.raises(ValueError, match="cut short"):
|
||||
parse_time_sync("@@ts timedatectl\nTimezone=UTC\n")
|
||||
|
||||
|
||||
class TestReading:
|
||||
def test_chrony_names_its_sources_in_the_order_it_reads_them(self, debian: Path):
|
||||
host = Host(debian)
|
||||
assert host.get_time_sync() == {
|
||||
"ntp_servers": [
|
||||
"ntp-conf.example.org",
|
||||
"2.debian.pool.ntp.org",
|
||||
"ntp-other.example.org",
|
||||
],
|
||||
"timezone": "Europe/Berlin",
|
||||
"synchronized": True,
|
||||
"daemon": "chrony",
|
||||
"dhcp_ntp_servers": ["192.168.1.1"],
|
||||
}
|
||||
|
||||
def test_dhcp_servers_in_the_order_chrony_reads_them(self, debian: Path):
|
||||
_write(debian / "run/chrony-dhcp/wlan0.sources", "server 10.9.0.1 iburst\n")
|
||||
_write(debian / "run/chrony-dhcp/eth0.sources", "server 192.168.1.1\npool p.example\n")
|
||||
assert Host(debian).get_time_sync()["dhcp_ntp_servers"] == [
|
||||
"192.168.1.1",
|
||||
"p.example",
|
||||
"10.9.0.1",
|
||||
]
|
||||
|
||||
def test_a_dhcp_directory_chrony_does_not_read_brings_nothing(self, debian: Path):
|
||||
# What 4.35.0 left behind: DHCP's sourcedir commented out.
|
||||
conf = debian / "etc/chrony/chrony.conf"
|
||||
conf.write_text(_text(conf).replace("\nsourcedir ", "\n#napalm-time-sync: sourcedir ", 1))
|
||||
reading = Host(debian).get_time_sync()
|
||||
assert reading["dhcp_ntp_servers"] == []
|
||||
assert "192.168.1.1" not in reading["ntp_servers"]
|
||||
|
||||
def test_without_a_dhcp_directory_dhcp_brings_nothing(self, root: Path):
|
||||
_write(root / "etc/chrony.conf", "pool 2.rhel.pool.ntp.org iburst\n")
|
||||
assert Host(root, active="chronyd").get_time_sync()["dhcp_ntp_servers"] == []
|
||||
|
||||
def test_the_reading_is_one_round_trip_without_root(self, debian: Path):
|
||||
host = Host(debian)
|
||||
host.get_time_sync()
|
||||
assert [p for _, p in host.commands] == [False]
|
||||
|
||||
def test_chrony_says_whether_it_is_synchronised_not_the_kernel(self, debian: Path):
|
||||
(debian / "fix/chronyc").write_text(CHRONYC_UNSYNCED)
|
||||
assert Host(debian).get_time_sync()["synchronized"] is False
|
||||
|
||||
def test_without_an_answer_from_chronyc_the_kernel_says(self, debian: Path):
|
||||
(debian / "fix/chronyc").unlink()
|
||||
assert Host(debian).get_time_sync()["synchronized"] is True
|
||||
|
||||
def test_commented_and_other_directives_are_no_sources(self, root: Path):
|
||||
_write(
|
||||
root / "etc/chrony.conf",
|
||||
"#pool 2.rhel.pool.ntp.org iburst\n! server old.example.org\n"
|
||||
" server ntp1.example.org iburst\nrefclock PPS /dev/pps0\npeer peer.example.org\n",
|
||||
)
|
||||
reading = Host(root, active="chronyd").get_time_sync()
|
||||
assert reading["ntp_servers"] == ["ntp1.example.org", "peer.example.org"]
|
||||
assert reading["daemon"] == "chrony"
|
||||
|
||||
def test_an_include_pattern_is_followed(self, root: Path):
|
||||
_write(root / "etc/chrony.conf", f"include {root}/etc/chrony.d/*.conf\nmakestep 1 3\n")
|
||||
_write(root / "etc/chrony.d/pool.conf", "pool suse.pool.ntp.org iburst\n")
|
||||
assert Host(root).get_time_sync()["ntp_servers"] == ["suse.pool.ntp.org"]
|
||||
|
||||
def test_timesyncd(self, root: Path):
|
||||
(root / "fix/timesync").write_text(TIMESYNC_CONFIGURED)
|
||||
reading = Host(root, active="systemd-timesyncd").get_time_sync()
|
||||
assert reading["daemon"] == "systemd-timesyncd"
|
||||
assert reading["ntp_servers"] == ["ntp1.example.org", "ntp2.example.org"]
|
||||
assert reading["dhcp_ntp_servers"] == ["192.168.1.1"]
|
||||
assert reading["synchronized"] is True
|
||||
|
||||
def test_ntpd(self, root: Path):
|
||||
_write(
|
||||
root / "etc/ntpsec/ntp.conf",
|
||||
"driftfile /var/lib/ntpsec/ntp.drift\npool 0.debian.pool.ntp.org iburst\n"
|
||||
"server 127.127.1.0\nserver ntp1.example.org iburst\nrestrict default kod\n",
|
||||
)
|
||||
reading = Host(root, active="ntpsec").get_time_sync()
|
||||
assert reading["daemon"] == "ntpd"
|
||||
assert reading["ntp_servers"] == ["0.debian.pool.ntp.org", "ntp1.example.org"]
|
||||
# Debian's DHCP hook gives ntpd a configuration of its own: not read, so unknown.
|
||||
assert reading["dhcp_ntp_servers"] is None
|
||||
|
||||
def test_openntpd(self, root: Path):
|
||||
_write(
|
||||
root / "etc/openntpd/ntpd.conf",
|
||||
'servers pool.ntp.org\nserver ntp1.example.org\nconstraints from "https://www.google.com"\n',
|
||||
)
|
||||
reading = Host(root, active="openntpd").get_time_sync()
|
||||
assert reading["daemon"] == "openntpd"
|
||||
assert reading["ntp_servers"] == ["pool.ntp.org", "ntp1.example.org"]
|
||||
assert reading["dhcp_ntp_servers"] is None
|
||||
|
||||
def test_no_client_runs(self, root: Path):
|
||||
(root / "fix/timedatectl").write_text(
|
||||
TIMEDATECTL.replace("NTPSynchronized=yes", "NTPSynchronized=no")
|
||||
)
|
||||
assert Host(root, active="").get_time_sync() == {
|
||||
"ntp_servers": [],
|
||||
"timezone": "Europe/Berlin",
|
||||
"synchronized": False,
|
||||
"daemon": None,
|
||||
"dhcp_ntp_servers": [],
|
||||
}
|
||||
|
||||
def test_without_timedatectl_the_zone_comes_from_the_files(self, root: Path):
|
||||
_no_timedatectl(root)
|
||||
(root / "etc/localtime").symlink_to("/usr/share/zoneinfo/America/Sao_Paulo")
|
||||
reading = Host(root, active="").get_time_sync()
|
||||
assert reading["timezone"] == "America/Sao_Paulo"
|
||||
assert reading["synchronized"] is None
|
||||
|
||||
def test_etc_timezone_wins_over_the_link(self, root: Path):
|
||||
_no_timedatectl(root)
|
||||
(root / "etc/timezone").write_text("Australia/Lord_Howe\n")
|
||||
assert Host(root, active="").get_time_sync()["timezone"] == "Australia/Lord_Howe"
|
||||
|
||||
def test_a_copied_localtime_has_no_name(self, root: Path):
|
||||
_no_timedatectl(root)
|
||||
assert Host(root, active="").get_time_sync()["timezone"] is None
|
||||
|
||||
|
||||
class TestChronyServers:
|
||||
def test_exactly_these_servers(self, debian: Path):
|
||||
host = Host(debian)
|
||||
result = host.configure_time_sync(["ntp1.example.org", "10.0.0.1", "2.de.pool.ntp.org"])
|
||||
|
||||
assert result["success"] is True, result["output"]
|
||||
assert _text(debian / "etc/chrony/sources.d/netork.sources") == (
|
||||
"server ntp1.example.org iburst\nserver 10.0.0.1 iburst\npool 2.de.pool.ntp.org iburst\n"
|
||||
)
|
||||
conf = _text(debian / "etc/chrony/chrony.conf")
|
||||
assert "\n#napalm-time-sync: pool 2.debian.pool.ntp.org iburst\n" in conf
|
||||
# DHCP option 42 counts: chrony keeps reading what DHCP brings.
|
||||
assert f"\nsourcedir {debian}/run/chrony-dhcp\n" in conf
|
||||
assert f"\nsourcedir {debian}/etc/chrony/sources.d\n" in conf
|
||||
assert f"\nconfdir {debian}/etc/chrony/conf.d\n" in conf
|
||||
assert _text(debian / "etc/chrony/conf.d/local.conf") == (
|
||||
"#napalm-time-sync: server ntp-conf.example.org iburst\n"
|
||||
)
|
||||
assert _text(debian / "etc/chrony/sources.d/other.sources") == (
|
||||
"#napalm-time-sync: server ntp-other.example.org\n"
|
||||
)
|
||||
# DHCP's own file is left alone: the DHCP client rewrites it on every lease.
|
||||
assert _text(debian / "run/chrony-dhcp/eth0.sources") == "server 192.168.1.1 iburst\n"
|
||||
# The main configuration changed: chronyd reads it only when it starts.
|
||||
assert "systemctl restart chrony.service" in host.log()
|
||||
reading = host.get_time_sync()
|
||||
assert reading["ntp_servers"] == ["ntp1.example.org", "10.0.0.1", "2.de.pool.ntp.org"]
|
||||
assert reading["dhcp_ntp_servers"] == ["192.168.1.1"]
|
||||
|
||||
def test_what_4_35_0_took_from_dhcp_is_given_back(self, debian: Path):
|
||||
conf = debian / "etc/chrony/chrony.conf"
|
||||
Host(debian).configure_time_sync(["ntp1.example.org"])
|
||||
# A host 4.35.0 aligned: DHCP's sourcedir commented out behind the mark.
|
||||
dhcp = f"sourcedir {debian}/run/chrony-dhcp"
|
||||
conf.write_text(_text(conf).replace(f"\n{dhcp}\n", f"\n#napalm-time-sync: {dhcp}\n"))
|
||||
assert Host(debian).get_time_sync()["dhcp_ntp_servers"] == []
|
||||
(debian / "log").unlink()
|
||||
|
||||
host = Host(debian)
|
||||
result = host.configure_time_sync(["ntp1.example.org"])
|
||||
|
||||
assert result["success"] is True, result["output"]
|
||||
assert f"\n{dhcp}\n" in _text(conf)
|
||||
assert "#napalm-time-sync: sourcedir" not in _text(conf)
|
||||
# The main configuration changed again: a restart, not a reload.
|
||||
assert "systemctl restart chrony.service" in host.log()
|
||||
assert host.get_time_sync()["dhcp_ntp_servers"] == ["192.168.1.1"]
|
||||
|
||||
def test_a_foreign_sourcedir_4_35_0_took_out_stays_out(self, debian: Path):
|
||||
conf = debian / "etc/chrony/chrony.conf"
|
||||
foreign = f"#napalm-time-sync: sourcedir {debian}/etc/chrony/extra.d"
|
||||
conf.write_text(_text(conf) + foreign + "\n")
|
||||
_write(debian / "etc/chrony/extra.d/x.sources", "server ntp-extra.example.org\n")
|
||||
|
||||
assert Host(debian).configure_time_sync(["ntp1.example.org"])["success"] is True
|
||||
assert f"\n{foreign}\n" in _text(conf)
|
||||
|
||||
def test_a_foreign_sourcedir_stays_and_its_sources_stop_counting(self, debian: Path):
|
||||
conf = debian / "etc/chrony/chrony.conf"
|
||||
conf.write_text(_text(conf) + f"sourcedir {debian}/etc/chrony/extra.d\n")
|
||||
_write(debian / "etc/chrony/extra.d/x.sources", "server ntp-extra.example.org\n")
|
||||
|
||||
host = Host(debian)
|
||||
result = host.configure_time_sync(["ntp1.example.org"])
|
||||
|
||||
assert result["success"] is True, result["output"]
|
||||
assert f"\nsourcedir {debian}/etc/chrony/extra.d\n" in _text(conf)
|
||||
assert _text(debian / "etc/chrony/extra.d/x.sources") == (
|
||||
"#napalm-time-sync: server ntp-extra.example.org\n"
|
||||
)
|
||||
assert host.get_time_sync()["ntp_servers"] == ["ntp1.example.org"]
|
||||
|
||||
def test_dhcp_is_kept_where_the_servers_go_to_a_confdir(self, root: Path):
|
||||
_write(
|
||||
root / "etc/chrony/chrony.conf",
|
||||
f"confdir {root}/etc/chrony/conf.d\npool a.pool.ntp.org\n"
|
||||
f"sourcedir {root}/run/chrony-dhcp\n",
|
||||
)
|
||||
_write(root / "run/chrony-dhcp/eth0.sources", "server 192.168.1.1 iburst\n")
|
||||
host = Host(root)
|
||||
assert host.configure_time_sync(["ntp1.example.org"])["success"] is True
|
||||
assert f"\nsourcedir {root}/run/chrony-dhcp\n" in _text(root / "etc/chrony/chrony.conf")
|
||||
reading = host.get_time_sync()
|
||||
assert (reading["ntp_servers"], reading["dhcp_ntp_servers"]) == (
|
||||
["ntp1.example.org"],
|
||||
["192.168.1.1"],
|
||||
)
|
||||
|
||||
def test_a_second_run_only_reloads_the_sources(self, debian: Path):
|
||||
Host(debian).configure_time_sync(["ntp1.example.org"])
|
||||
conf = _text(debian / "etc/chrony/chrony.conf")
|
||||
(debian / "log").unlink()
|
||||
|
||||
host = Host(debian)
|
||||
result = host.configure_time_sync(["ntp2.example.org", "ntp1.example.org"])
|
||||
|
||||
assert result["success"] is True, result["output"]
|
||||
assert _text(debian / "etc/chrony/chrony.conf") == conf
|
||||
assert "chronyc reload sources" in host.log()
|
||||
assert not any(line.startswith("systemctl restart") for line in host.log())
|
||||
assert _text(debian / "etc/chrony/sources.d/netork.sources") == (
|
||||
"server ntp2.example.org iburst\nserver ntp1.example.org iburst\n"
|
||||
)
|
||||
|
||||
def test_a_chronyd_that_cannot_reload_sources_is_restarted(self, debian: Path):
|
||||
Host(debian).configure_time_sync(["ntp1.example.org"])
|
||||
host = Host(debian)
|
||||
host.env["RELOAD_EXIT"] = "1"
|
||||
assert host.configure_time_sync(["ntp2.example.org"])["success"] is True
|
||||
log = host.log()
|
||||
reload = log.index("chronyc reload sources")
|
||||
assert log[reload + 1] == "systemctl restart chrony.service"
|
||||
|
||||
def test_writes_are_root_s_and_root_reads_no_stdin(self, debian: Path):
|
||||
host = Host(debian)
|
||||
host.configure_time_sync(["ntp1.example.org"])
|
||||
staged = [c for c, p in host.commands if not p and "mktemp" in c]
|
||||
assert len(staged) == 1
|
||||
assert host.privileged()
|
||||
assert all("mktemp" not in c for c in host.privileged())
|
||||
|
||||
def test_rhel_without_a_directory_gets_an_include(self, root: Path):
|
||||
_write(
|
||||
root / "etc/chrony.conf",
|
||||
"pool 2.rhel.pool.ntp.org iburst\ndriftfile /var/lib/chrony/drift\n",
|
||||
)
|
||||
host = Host(root, active="chronyd")
|
||||
result = host.configure_time_sync(["ntp1.example.org"])
|
||||
|
||||
assert result["success"] is True, result["output"]
|
||||
include = root / "etc/chrony.d/netork.conf"
|
||||
assert _text(include) == "server ntp1.example.org iburst\n"
|
||||
assert _text(root / "etc/chrony.conf") == (
|
||||
"#napalm-time-sync: pool 2.rhel.pool.ntp.org iburst\n"
|
||||
"driftfile /var/lib/chrony/drift\n"
|
||||
f"# napalm-time-sync: the time sources\ninclude {include}\n"
|
||||
)
|
||||
assert "systemctl restart chronyd.service" in host.log()
|
||||
|
||||
def test_an_include_that_already_takes_the_file_is_used(self, root: Path):
|
||||
_write(root / "etc/chrony.conf", f"include {root}/etc/chrony.d/*.conf\n")
|
||||
_write(root / "etc/chrony.d/pool.conf", "pool suse.pool.ntp.org iburst\n")
|
||||
host = Host(root)
|
||||
assert host.configure_time_sync(["ntp1.example.org"])["success"] is True
|
||||
assert _text(root / "etc/chrony.conf") == f"include {root}/etc/chrony.d/*.conf\n"
|
||||
assert (
|
||||
_text(root / "etc/chrony.d/pool.conf")
|
||||
== "#napalm-time-sync: pool suse.pool.ntp.org iburst\n"
|
||||
)
|
||||
assert _text(root / "etc/chrony.d/netork.conf") == "server ntp1.example.org iburst\n"
|
||||
|
||||
def test_a_confdir_takes_the_file_where_sources_d_is_not_read(self, root: Path):
|
||||
_write(
|
||||
root / "etc/chrony/chrony.conf",
|
||||
f"confdir {root}/etc/chrony/conf.d\npool a.pool.ntp.org\n",
|
||||
)
|
||||
host = Host(root)
|
||||
assert host.configure_time_sync(["ntp1.example.org"])["success"] is True
|
||||
assert _text(root / "etc/chrony/conf.d/netork.conf") == "server ntp1.example.org iburst\n"
|
||||
assert not (root / "etc/chrony/sources.d/netork.sources").exists()
|
||||
|
||||
def test_a_failed_restart_fails_the_step(self, debian: Path):
|
||||
host = Host(debian)
|
||||
host.env["FAIL_RESTART"] = "1"
|
||||
result = host.configure_time_sync(["ntp1.example.org"])
|
||||
|
||||
assert result["success"] is False
|
||||
assert "Job for chrony.service failed." in result["steps"][0]["output"]
|
||||
|
||||
def test_a_file_that_cannot_be_staged_is_not_installed(self, debian: Path):
|
||||
_tool(debian, "mktemp", "#!/bin/sh\necho 'mktemp: failed' >&2\nexit 1\n")
|
||||
host = Host(debian)
|
||||
result = host.configure_time_sync(["ntp1.example.org"])
|
||||
|
||||
assert result["success"] is False
|
||||
assert "cannot stage" in result["steps"][0]["output"]
|
||||
assert host.privileged() == []
|
||||
|
||||
def test_no_configuration_is_a_failed_step(self, root: Path):
|
||||
result = Host(root).configure_time_sync(["ntp1.example.org"])
|
||||
assert result["success"] is False
|
||||
assert "chrony" in result["steps"][0]["output"]
|
||||
|
||||
|
||||
class TestTimesyncdServers:
|
||||
def test_a_drop_in_with_exactly_these_servers(self, root: Path):
|
||||
(root / "fix/timesync").write_text(TIMESYNC_FALLBACK)
|
||||
host = Host(root, active="systemd-timesyncd")
|
||||
result = host.configure_time_sync(["ntp1.example.org", "10.0.0.1"])
|
||||
|
||||
assert result["success"] is True, result["output"]
|
||||
assert _text(root / "etc/systemd/timesyncd.conf.d/netork.conf").endswith(
|
||||
"[Time]\nNTP=\nNTP=ntp1.example.org 10.0.0.1\nFallbackNTP=\n"
|
||||
)
|
||||
assert "systemctl restart systemd-timesyncd.service" in host.log()
|
||||
assert host.get_time_sync()["ntp_servers"] == ["ntp1.example.org", "10.0.0.1"]
|
||||
|
||||
def test_servers_from_dhcp_stay_and_count(self, root: Path):
|
||||
(root / "fix/timesync").write_text(TIMESYNC_CONFIGURED)
|
||||
tool = (root / "bin/systemctl").read_text()
|
||||
(root / "bin/systemctl").write_text(
|
||||
tool.replace(
|
||||
"printf 'SystemNTPServers", "printf 'LinkNTPServers=192.168.1.1\\nSystemNTPServers"
|
||||
)
|
||||
)
|
||||
host = Host(root, active="systemd-timesyncd")
|
||||
result = host.configure_time_sync(["ntp1.example.org"])
|
||||
|
||||
assert result["success"] is True, result["output"]
|
||||
reading = host.get_time_sync()
|
||||
assert (reading["ntp_servers"], reading["dhcp_ntp_servers"]) == (
|
||||
["ntp1.example.org"],
|
||||
["192.168.1.1"],
|
||||
)
|
||||
|
||||
|
||||
class TestOtherClients:
|
||||
@pytest.mark.parametrize("active", ["ntpsec", "openntpd", ""])
|
||||
def test_only_chrony_and_timesyncd_are_written(self, root: Path, active: str):
|
||||
host = Host(root, active=active)
|
||||
result = host.configure_time_sync(["ntp1.example.org"])
|
||||
|
||||
assert result["success"] is False
|
||||
assert "chrony and systemd-timesyncd" in result["steps"][0]["output"]
|
||||
assert host.privileged() == []
|
||||
|
||||
|
||||
class TestTimeZone:
|
||||
def test_timedatectl_sets_it(self, root: Path):
|
||||
host = Host(root, active="")
|
||||
result = host.configure_time_sync(timezone="America/Sao_Paulo")
|
||||
|
||||
assert result["success"] is True, result["output"]
|
||||
assert host.privileged() == ["timedatectl set-timezone America/Sao_Paulo"]
|
||||
assert host.get_time_sync()["timezone"] == "America/Sao_Paulo"
|
||||
|
||||
def test_a_zone_the_host_does_not_know_fails_the_step(self, root: Path):
|
||||
host = Host(root, active="")
|
||||
host.env["REFUSE_ZONE"] = "Europe/Kyiv"
|
||||
result = host.configure_time_sync(timezone="Europe/Kyiv")
|
||||
|
||||
assert result["success"] is False
|
||||
assert "Invalid or not installed time zone" in result["steps"][0]["output"]
|
||||
|
||||
def test_an_unknown_zone_is_refused_before_anything_is_written(self, debian: Path):
|
||||
host = Host(debian)
|
||||
with pytest.raises(UnknownTimezone):
|
||||
host.configure_time_sync(["ntp1.example.org"], "Mars/Olympus_Mons")
|
||||
assert host.commands == []
|
||||
|
||||
def test_both_at_once(self, debian: Path):
|
||||
host = Host(debian)
|
||||
result = host.configure_time_sync(["ntp1.example.org"], "UTC")
|
||||
|
||||
assert result["success"] is True, result["output"]
|
||||
assert [s["step"] for s in result["steps"]] == ["ntp_servers", "timezone", "verify"]
|
||||
reading = host.get_time_sync()
|
||||
assert (reading["ntp_servers"], reading["timezone"]) == (["ntp1.example.org"], "UTC")
|
||||
|
||||
|
||||
def test_it_writes_both_values():
|
||||
assert frozenset({"ntp_servers", "timezone"}) == PosixTimeSyncMixin.time_sync_writes
|
||||
@@ -4,6 +4,7 @@ import pytest
|
||||
|
||||
from napalm_device_types import (
|
||||
AccessPointDriver,
|
||||
BackupServerDriver,
|
||||
FirewallDriver,
|
||||
HypervisorDriver,
|
||||
ManagementControllerDriver,
|
||||
@@ -15,6 +16,7 @@ from napalm_device_types import (
|
||||
|
||||
BASE_CLASSES = [
|
||||
(AccessPointDriver, "Access Point"),
|
||||
(BackupServerDriver, "Backup Server"),
|
||||
(FirewallDriver, "Firewall"),
|
||||
(HypervisorDriver, "Hypervisor"),
|
||||
(ManagementControllerDriver, "Management Controller"),
|
||||
|
||||
Reference in new issue
Block a user