Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c967635cba | ||
|
|
c470686305 | ||
|
|
1edf458feb | ||
|
|
00452d0d37 | ||
|
|
4e40f95bc1 | ||
|
|
70e1c9b218 | ||
|
|
a70501ff60 | ||
|
|
c6d119eb83 | ||
|
|
8e9d21c368 | ||
|
|
25f28297e4 | ||
|
|
847c902193 | ||
|
|
ffbb194300 | ||
|
|
edef6cf07c | ||
|
|
df476a12f2 | ||
|
|
834cb2e0ea | ||
|
|
f573fa53d2 | ||
|
|
fdf2f89115 | ||
|
|
d63abff467 | ||
|
|
9d6a1a65ed | ||
|
|
95e3aa1e26 | ||
|
|
a0901e3277 | ||
|
|
060e210130 | ||
|
|
31973d2d5f | ||
|
|
abda0973e7 | ||
|
|
4eeaaef236 | ||
|
|
3490d8b62a | ||
|
|
2542081c6f | ||
|
|
4427be35cc | ||
|
|
ad9ec0fb4d | ||
|
|
8dd7e8ab9a | ||
|
|
935fad2a2d | ||
|
|
f337ae7fb8 | ||
|
|
c7ea15dbb4 | ||
|
|
01fe680456 |
No files matched your search
@@ -7,6 +7,96 @@ recorded here; their history is the git log.
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [4.28.0] - 2026-11-04
|
||||
|
||||
### Added
|
||||
- `LocalAccountsMixin.add_local_account(..., replaces=[...])`: the keys the new one takes
|
||||
the place of go from the account, matched by their base64 whatever a line's options or
|
||||
comment. Keys the person added on the host stay. The step is `old keys`; it runs even
|
||||
when the new key could not be added (NetOrk/netork#2013).
|
||||
- `add_local_account(..., clear_password=True)`: the account's password goes, and it logs
|
||||
in with a key only. Step `password`. It cannot be combined with a `password_hash` or
|
||||
with `admin=True`, as the sudo rule asks for that password. A driver supplies
|
||||
`_account_clear_password`; one that does not fails the step and says so.
|
||||
- `LocalAccountsMixin.remove_local_account_keys(name, keys)`: takes the named keys off an
|
||||
account and leaves it open, its password and sudo as they are -- for a person who
|
||||
deleted a key and has no new one. Step `keys`.
|
||||
- A driver whose host keeps keys elsewhere overrides `_account_drop_keys` next to
|
||||
`_account_add_key` and `_account_remove_keys`.
|
||||
|
||||
## [4.27.0] - 2026-11-03
|
||||
|
||||
### Added
|
||||
- `ScheduledJobsMixin`: what a host runs on its own schedule as one shape,
|
||||
`ScheduledJobDict` — cron lines, run-parts scripts, anacron, systemd timers, `periodic`,
|
||||
a platform's own jobs, Windows tasks — with who owns each definition (`managed_by`) and
|
||||
what the driver can do to it (`actions`). `get_scheduled_jobs` reads them with the host's
|
||||
time zone; `manage_scheduled_job` enables, disables or runs one now; `put_scheduled_job`
|
||||
and `delete_scheduled_job` create, change and delete a job an application writes. Every
|
||||
write names the `digest` the caller saw (`job_digest`) and raises `ScheduledJobChanged`
|
||||
when the host's definition differs; `ScheduledJobRefused` for a job that cannot take a
|
||||
verb (NetOrk/netork#1933, #86).
|
||||
- `PosixScheduledJobsMixin`: the implementation over the command channel. One framed report
|
||||
as root reads `/etc/crontab`, `/etc/cron.d`, the user crontabs, the run-parts directories,
|
||||
`/etc/anacrontab` and every systemd timer with the unit it starts, and which package owns
|
||||
each definition. A cron line is switched off behind `#napalm-disabled: `; an
|
||||
application's job is a timer where systemd runs, else a file in `/etc/cron.d` or marked
|
||||
lines in the user's crontab.
|
||||
- `napalm_device_types.cron`: cron expressions as Vixie cron reads them, their next runs in
|
||||
a time zone, and as systemd `OnCalendar` lines that fire on exactly the same minutes.
|
||||
|
||||
### Removed
|
||||
- `OSDriver.get_cron_jobs` and `CronJobDict`. No application called them; the readers in
|
||||
napalm-linux and napalm-bsd go with them. `get_scheduled_jobs` replaces them.
|
||||
|
||||
## [4.26.0] - 2026-11-03
|
||||
|
||||
### Added
|
||||
- `OpenSshServerMixin.SSHD_CONFIG_GENERATED`: a driver whose platform writes sshd_config
|
||||
from its own database, but includes `/etc/ssh/sshd_config.d` (OpenMediaVault), sets it
|
||||
and keeps `harden_ssh_server`. The hardening then goes only into the drop-in; a
|
||||
sshd_config without that include is left alone and the result says why, since the
|
||||
platform would write over a change to it on its next apply (#88, NetOrk/netork#1950).
|
||||
|
||||
## [4.25.0] - 2026-11-02
|
||||
|
||||
### Added
|
||||
- `BackupServerDriver` (`ROLE = "backup_server"`), a role base for backup servers such as
|
||||
Proxmox Backup Server (#82). It declares `get_backup_datastores`, `get_backup_groups`
|
||||
(each group with its newest snapshot), `get_backup_snapshots`, `get_backup_jobs`
|
||||
(garbage collection, verify, prune, sync and tape in one list) and `get_backup_remotes`,
|
||||
and the actions `start_garbage_collection`, `run_backup_job`, `verify_backup_snapshot`,
|
||||
`set_backup_snapshot_protected` and `delete_backup_snapshot`, which return a task to
|
||||
follow with `get_backup_task`. The shapes live in `napalm_device_types.backups`. A
|
||||
server that manages its own disks lists `StorageDriver` as its second role.
|
||||
- `ClusterStorageDict.backup_server` (optional): which backup server, datastore and
|
||||
namespace a hypervisor's backup storage writes to, with the certificate fingerprint it
|
||||
pins, so an application can tie a server's backup groups to a cluster's guests.
|
||||
|
||||
## [4.24.0] - 2026-11-02
|
||||
|
||||
### Added
|
||||
- `parse_lldpctl_keyvalue(output)` and `LLDPCTL_KEYVALUE_COMMAND`: lldpd's neighbours
|
||||
(`lldpctl -f keyvalue`) as one `LldpNeighborRow` per neighbour, for every driver of a
|
||||
host that runs lldpd. Several neighbours on one port (a phone and the PC behind it, an
|
||||
unmanaged switch) stay apart, an interface name with dots (`eth0.1`) stays whole, the
|
||||
chassis and port ids are read whatever their type (MAC, address, interface name, local
|
||||
id), with management addresses and capabilities. A failed lldpctl (lldpd not running,
|
||||
lldpctl missing) raises instead of reading as no neighbours.
|
||||
- `lldp_neighbors(rows)` and `lldp_neighbors_detail(rows, interface="")`: NAPALM's
|
||||
`get_lldp_neighbors` and `get_lldp_neighbors_detail` from those rows, so a driver no
|
||||
longer builds the two shapes itself. A neighbour that names its chassis by a MAC
|
||||
address carries it in `get_lldp_neighbors` as `mac`, in lower case
|
||||
(`LldpNeighborDict`), so an application can match it by MAC.
|
||||
|
||||
### Fixed
|
||||
- `DNF_SECURITY_COMMAND` reads the cached metadata only (`dnf --cacheonly`), as
|
||||
`APT_UPGRADABLE_COMMAND` reads apt's local index (#76). With an expired cache dnf
|
||||
downloaded the repositories' metadata first, and on an Oracle Linux host on OCI that
|
||||
outlasted netOrk's whole poll budget. Refreshing the index stays a step of its own
|
||||
(`dnf makecache`). Without a cache dnf fails at once, which already reads as
|
||||
"security unknown".
|
||||
|
||||
## [4.23.0] - 2026-11-02
|
||||
|
||||
### Changed
|
||||
|
||||
@@ -47,6 +47,18 @@ UUID and NIC MACs -- what ties the BMC to the host it manages. Its sensors come
|
||||
NAPALM's `get_environment`, with `cpu` and `memory` empty, since a BMC does not know the
|
||||
host OS's load; it has no `HealthMetricsMixin` (SNMP) and no command channel.
|
||||
|
||||
A backup server (Proxmox Backup Server) is a role of its own too, `BackupServerDriver`
|
||||
(`backup_server`). Its datastores, the backup groups in them (each with its newest
|
||||
snapshot), its jobs (garbage collection, verify, prune, sync, tape) and its remotes have
|
||||
no counterpart in `StorageDriver`, so they are not bent into pools, volumes and
|
||||
replication jobs. A server that also manages its own disks and pools lists
|
||||
`StorageDriver` second: `class ProxmoxBackupDriver(BackupServerDriver, StorageDriver, ...)`.
|
||||
Its actions (garbage collection, running a job, verifying, protecting or deleting a
|
||||
snapshot) start a task and return its id at once; `get_backup_task` follows it. On the
|
||||
hypervisor side, a cluster storage that writes to a backup server says which one
|
||||
(`ClusterStorageDict.backup_server`: server, port, datastore, namespace, fingerprint), so
|
||||
an application can tie the server's groups to the cluster's guests.
|
||||
|
||||
### Role bases declare; they never implement
|
||||
|
||||
**A role base must not contain a single runtime method — not even a
|
||||
@@ -179,6 +191,16 @@ without `ss` is read with `netstat -lntup` (OpenWrt's busybox, old net-tools); o
|
||||
cgroup names the procd service (`/services/<name>/<instance>`). A host with neither raises
|
||||
`ListeningSocketsUnavailable`.
|
||||
|
||||
LLDP neighbours are rows: `LldpNeighborRow` is one neighbour as its local port heard it
|
||||
(chassis and port id with their type, name, description, management addresses,
|
||||
capabilities in NAPALM's names), and `lldp_neighbors(rows)` / `lldp_neighbors_detail(rows,
|
||||
interface)` are NAPALM's two getters projected from them, a port with two neighbours
|
||||
keeping both; a neighbour named by its chassis MAC carries it as `mac` (lower case). A driver for a host that runs lldpd sends `LLDPCTL_KEYVALUE_COMMAND` and
|
||||
hands its output to `parse_lldpctl_keyvalue`: one row per neighbour block, an interface
|
||||
name with dots (`eth0.1`) kept whole, ids read under the type lldpd prints them with
|
||||
(`chassis.mac`, `port.local`, ...). When lldpctl fails (lldpd not running, no `lldpctl`)
|
||||
the parse raises instead of reading as no neighbours.
|
||||
|
||||
`HostFirewallMixin` (`get_host_firewall`) reads a Linux host's own netfilter rules the same
|
||||
way: `nft -j list ruleset`, with the tables iptables-nft manages taken from `iptables-save`
|
||||
instead, because nft shows their extensions (`-m conntrack`, `-m set`) only as an opaque `xt`.
|
||||
@@ -279,6 +301,20 @@ trip (`list-unit-files` plus one `systemctl show` over every loaded unit) instea
|
||||
`is-enabled` and a `show` per unit. A host without systemd raises `SystemdUnavailable`,
|
||||
a `NotImplementedError`, so a driver can fall back to another init system.
|
||||
|
||||
`ScheduledJobsMixin` (`get_scheduled_jobs`, `manage_scheduled_job`, `put_scheduled_job`,
|
||||
`delete_scheduled_job`) is what a host runs on its own schedule — cron lines, run-parts
|
||||
scripts, anacron, systemd timers, `periodic`, a platform's own jobs, Windows tasks — as one
|
||||
shape, `ScheduledJobDict`. Every job says who owns its definition (`managed_by`: package,
|
||||
admin, platform, application) and what the driver can do to it (`actions`): every job a
|
||||
driver can switch takes `enable`, `disable` and `run`; only a job an application wrote
|
||||
through `put_scheduled_job` takes `edit` and `delete`. Every write names the `digest` the
|
||||
caller saw and raises `ScheduledJobChanged` when the host's definition is no longer that.
|
||||
`PosixScheduledJobsMixin` implements it over `run_command`: one framed report as root,
|
||||
a cron line switched off behind `#napalm-disabled: `, a timer by `systemctl`, an
|
||||
application's job as a timer where systemd runs, else in `/etc/cron.d` or the user's
|
||||
crontab. A driver sets the paths its host keeps jobs in. `napalm_device_types.cron` parses
|
||||
cron expressions, gives their next runs and writes them as `OnCalendar` lines.
|
||||
|
||||
### Access channels: why there is no container model here
|
||||
|
||||
`CommandChannelMixin` declares a driver's public channel to its host:
|
||||
@@ -451,6 +487,7 @@ Requires Python ≥ 3.12 (tested up to 3.13) and NAPALM ≥ 4.0.
|
||||
| `PhoneDriver` | IP telephony endpoints | Yealink |
|
||||
| `MediaDriver` | Speakers, streamers, media renderers | Sonos, Chromecast |
|
||||
| `ManagementControllerDriver` | Server baseboard management controllers | Supermicro BMC, Dell iDRAC, HPE iLO, Lenovo XCC |
|
||||
| `BackupServerDriver` | Backup servers: datastores, backup groups, GC/verify/prune/sync/tape jobs | Proxmox Backup Server |
|
||||
|
||||
Mixins that come with the classes above: `PingSweepMixin` (subnet sweeps, in
|
||||
every driver through `DeviceTypeDriver`) and `DhcpServerMixin` (static DHCP
|
||||
@@ -598,10 +635,6 @@ class LinuxDriver(OSDriver):
|
||||
def get_processes(self):
|
||||
# return List[ProcessDict]
|
||||
...
|
||||
|
||||
def get_cron_jobs(self):
|
||||
# return List[CronJobDict]
|
||||
...
|
||||
```
|
||||
|
||||
### Storage / NAS
|
||||
|
||||
@@ -0,0 +1,244 @@
|
||||
# Structure
|
||||
|
||||
Status: agreed plan. Nothing here is implemented yet; the migration at the end says in
|
||||
which order it will be.
|
||||
|
||||
## Principle
|
||||
|
||||
**napalm-device-types is vendor-, OS-, protocol- and product-agnostic.** It holds the
|
||||
contracts every driver implements and the generic logic that works unchanged for every
|
||||
driver of a role. It knows no shell, no paramiko, no package manager, no container CLI,
|
||||
no SNMP OID, no cloud-init and no application that uses the drivers.
|
||||
|
||||
That rules out three kinds of code that are here today:
|
||||
|
||||
- **How a kind of host is read.** Shell commands and their parsers for POSIX hosts move to
|
||||
a new package, **napalm-unix**, which napalm-linux and napalm-bsd depend on.
|
||||
- **What an application does with a device.** Polling a device over SNMP, sweeping a range
|
||||
with pings, scoring discovery evidence, choosing what a provisioned guest installs, and
|
||||
deciding what counts as a problem belong to the application, not to the drivers.
|
||||
- **Product naming.** Files, markers and messages a driver leaves on a device carry the
|
||||
driver's name or none, never the name of the application that called it.
|
||||
|
||||
```
|
||||
napalm-device-types roles, capabilities, templates, models, reconcilers <- every driver
|
||||
^
|
||||
napalm-unix POSIX host readers, SSH exec transport, privilege <- napalm-linux, napalm-bsd
|
||||
^
|
||||
napalm-linux, napalm-bsd connection, platform getters, platform steps
|
||||
|
||||
application SNMP polling, ping sweeps, discovery scoring, guest
|
||||
profiles, warnings, and the values every fix is given
|
||||
```
|
||||
|
||||
## What stays in napalm-device-types
|
||||
|
||||
| Kind | Rule | Modules |
|
||||
|---|---|---|
|
||||
| **Role** | Declares only, under `TYPE_CHECKING`. No runtime method. | `access_point`, `switch`, `firewall`, `hypervisor`, `os`, `storage`, `phone`, `media`, `residential_gateway`, `management_controller`, `backup_server` |
|
||||
| **Capability** | Declares only. | `packages`, `services`, `updates`, `nat_vpn`, `mac_acl`, `config_lifecycle`, `interface_filter`, `host_reboot` |
|
||||
| **Template** | A concrete method that runs a fixed sequence of abstract steps. A driver implements the steps, or overrides the whole method when its device does it in one go. All values come from the caller. | the fix routines below |
|
||||
| **Contract types** | Interfaces and data shapes other packages implement or fill. | `CommandChannelMixin`, `CommandResult`, `ByteStream`; the declaration of `ContainerEngineMixin`; the reader mixins' public method declarations; the SNMP profile type; `models` |
|
||||
| **Reconciler** | Concrete and vendor-neutral, without I/O of its own. | `firewall_rules` and `dhcp` diff/apply, `_matching`, `lag` |
|
||||
| **Introspection** | Concrete, without I/O. | `roles`; `base` (`FingerprintRule`, `PortSpec`, the fingerprint attributes) |
|
||||
| **Driver facts** | Class attributes that describe the device or how it is reached, not what an application does with it. | `USES_SSH`, `FULL_UPGRADE`, `platform`, `SNMP_HEALTH_PROFILE` |
|
||||
|
||||
### Fix routines are templates
|
||||
|
||||
A change the application wants made on a device, for example "run an SNMP agent that
|
||||
answers us", is one template per purpose. The template fixes the order of the steps; the
|
||||
driver knows how each step is done on its device; the caller supplies every value.
|
||||
|
||||
```python
|
||||
class SnmpAgentMixin:
|
||||
def configure_snmp_agent(self, settings: SnmpAgentSettingsDict) -> FixResultDict:
|
||||
steps = [
|
||||
self._install_snmp_agent,
|
||||
self._write_snmp_agent_config,
|
||||
self._enable_snmp_agent,
|
||||
self._verify_snmp_agent,
|
||||
]
|
||||
... # runs them in order, stops at the first failure
|
||||
|
||||
# declared only; napalm-unix implements them for net-snmp
|
||||
def _install_snmp_agent(self) -> FixStepDict: ...
|
||||
def _write_snmp_agent_config(self, settings: SnmpAgentSettingsDict) -> FixStepDict: ...
|
||||
def _enable_snmp_agent(self) -> FixStepDict: ...
|
||||
def _verify_snmp_agent(self, settings: SnmpAgentSettingsDict) -> FixStepDict: ...
|
||||
```
|
||||
|
||||
napalm-unix implements the four steps for net-snmp, for Linux and the BSDs alike. A REST
|
||||
driver such as OPNsense overrides `configure_snmp_agent` as a whole, because its device
|
||||
takes the whole configuration in one call. The community, the allowed sources, contact
|
||||
and location are the caller's.
|
||||
|
||||
The same shape replaces every action in today's `run_device_action`:
|
||||
|
||||
| Today | Template |
|
||||
|---|---|
|
||||
| `fix_snmp` | `configure_snmp_agent(settings)` |
|
||||
| `fix_apt_proxy` | `configure_package_proxy(url)` |
|
||||
| `fix_docker_permissions` | `grant_container_engine_access(user)` |
|
||||
| `install_lldpd`, `install_auc` | `ensure_package(name)` |
|
||||
|
||||
The mixin is mixed in by the driver, or by the platform package, not by a role base. A
|
||||
driver that inherits a template it cannot run on its device, as a NAS driver built on the
|
||||
Linux driver inherits the package proxy, opts out by setting the method to `None`. So a
|
||||
caller asks `callable(getattr(driver, "configure_snmp_agent", None))`, never `hasattr`.
|
||||
|
||||
The caller verifies the result from its own side where it can; the SNMP agent is asked
|
||||
by the application's SNMP client, not by a probe the driver hand-builds.
|
||||
|
||||
### SNMP: drivers declare data, the application polls
|
||||
|
||||
The drivers do not collect over SNMP. A driver whose device answers the standard UCD-MIB
|
||||
and IF-MIB declares nothing. A driver whose device publishes the same values under
|
||||
proprietary OIDs declares them as data, in a class attribute of a type defined here:
|
||||
|
||||
```python
|
||||
class ProcurveDriver(SwitchDriver):
|
||||
SNMP_HEALTH_PROFILE: ClassVar[SnmpHealthProfile] = {
|
||||
"uptime": {"oids": ["1.3.6.1.4.1.11.2.14.11.1.2.4.1.5.1"], "format": "ticks"},
|
||||
"cpu": {"oids": ["1.3.6.1.4.1.11.2.14.11.5.1.9.6.1.0", "1.3.6.1.4.1.11.2.3.7.8.6.1.0"]},
|
||||
"memory": {"total": "...", "free": "...", "unit": "bytes"},
|
||||
}
|
||||
```
|
||||
|
||||
One OID per value is not enough for the drivers there are. The profile type has to say:
|
||||
|
||||
- **Fallbacks:** an ordered list of OIDs; the first that answers wins (ProCurve's CPU).
|
||||
- **Tables:** an OID to walk and how to fold its rows: first, average or maximum (TP-Link
|
||||
reports CPU and memory per stack unit).
|
||||
- **Text values:** a pattern that takes the number out of a string (Netgear reports CPU
|
||||
as `"5 Secs ( 3.2%)"`).
|
||||
- **Memory:** total with used or free, or a percentage only, and the unit.
|
||||
- **Uptime:** which OID, standard ones included (`sysUpTime` where a device has no
|
||||
`hrSystemUptime`), and its format.
|
||||
- **Interfaces:** a pattern of names to skip, and whether the TX-error counter counts
|
||||
drops. Today the second is `True` on every firewall, because OPNsense counts drops
|
||||
there; it becomes OPNsense's profile entry.
|
||||
|
||||
Polling, the standard OIDs, the counter arithmetic (64-bit counters first, 32-bit as the
|
||||
fallback) and the defaults are the application's.
|
||||
|
||||
### Warnings are facts
|
||||
|
||||
A driver reports facts: the proxy a host is configured with, a DSL line's SNR, the
|
||||
firmware version it runs and the one it is offered. It does not compare them with a
|
||||
setting of the application or with a threshold, and it does not name a warning.
|
||||
|
||||
The same goes for lookups a device does not make itself: what firmware a vendor
|
||||
currently publishes is the application's catalogue, not a getter.
|
||||
|
||||
### Getters read, and say when they cannot
|
||||
|
||||
- A getter changes nothing on the device. It does not start a daemon, rewrite its
|
||||
configuration or install a package on the way to an answer.
|
||||
- A read that fails raises. An empty list means "there is none", never "could not ask";
|
||||
a sync that deletes and re-adds on top of an empty answer would otherwise duplicate
|
||||
everything.
|
||||
- A write reports failure from the device's answer or exit status, never `success: True`
|
||||
by default.
|
||||
|
||||
### The application uses public methods only
|
||||
|
||||
The application never calls a driver's private methods (`_get`, `_post`,
|
||||
`_send_command`, `_action_*`). What it needs from a device becomes a declared method of
|
||||
the driver.
|
||||
|
||||
## What moves to napalm-unix
|
||||
|
||||
| Module | Knows |
|
||||
|---|---|
|
||||
| `host_status` | apt, dnf, dpkg, systemd timers, `df` |
|
||||
| `kernel` | `/proc`, `/lib/modules`, gzip, base64 |
|
||||
| `listening` | `ss`, `netstat`, `/proc/<pid>/cgroup` |
|
||||
| `systemd` | `systemctl` |
|
||||
| `virtual_ips` | vip-manager, keepalived, awk |
|
||||
| `system_identity` | SMBIOS via sysfs and `dmidecode` |
|
||||
| `package_updates` | apt and dnf output |
|
||||
| `ssh_server` | OpenSSH (`sshd -T`, `sshd -t`, `sshd_config`); which algorithms to drop stays the caller's |
|
||||
| `terminal` | ANSI escape codes in shell output |
|
||||
| `channel`: `run_on_transport`, `open_stream_on_transport`, `ParamikoExecStream` | paramiko exec channels |
|
||||
| `container_engine`: `ContainerEngineConnection`, engine probing | the Docker CLI, `system dial-stdio` |
|
||||
|
||||
It also takes what napalm-linux and napalm-bsd each carry today:
|
||||
|
||||
- **Privilege:** one `sudo -S` / `sudo -n` / root layer, including the rule that a
|
||||
privileged command never gets stdin while a sudo password is set, and one way to write a
|
||||
file as root (stage with `mktemp`, then `install`). Two copies of this have already
|
||||
drifted apart.
|
||||
- **Parsers:** `/etc/passwd` with `/etc/group`, crontab lines, `ps` with `lstart`.
|
||||
- **Fix steps:** net-snmp, the package proxy, container engine group membership.
|
||||
|
||||
The FreeBSD part of `host_status` (`freebsd-version -k` against `-r`) moves to napalm-bsd.
|
||||
|
||||
In napalm-unix every reader follows one pattern:
|
||||
|
||||
- **Shape:** `FOO_COMMAND`, `parse_foo(output)`, and `FooMixin.get_foo()`.
|
||||
- **Reaching the host:** the mixin uses `run_command(command, privileged=...)` from
|
||||
`CommandChannelMixin`. The six per-reader `_run_*` hooks, with four different signatures,
|
||||
go away.
|
||||
- **Framing:** one frame helper writes and parses the `BEGIN`/`END` markers for every reader.
|
||||
- **Errors:** one exception for "cannot read" and one for "not on this host". A read that
|
||||
fails or is cut short raises; it never reads as "nothing there".
|
||||
- **Names on the host:** temporary files, markers and drop-ins carry `napalm` or the
|
||||
driver's name. A drop-in written under an older name is still recognised and replaced.
|
||||
|
||||
Layout: `napalm_unix/{posix,linux,freebsd,openssh,transport}/`.
|
||||
|
||||
## What moves to the application
|
||||
|
||||
| Today | Why it is not a driver's |
|
||||
|---|---|
|
||||
| `health_metrics`, `_ucd_metrics` | Polling over SNMP is something the application does to a device; the driver is not even connected while it runs. |
|
||||
| `ping_sweep` | Looping over addresses, capping the targets and reporting progress is a scan. A device with a faster batch keeps it as a capability of its own. |
|
||||
| `rules_max_weight` | Scoring is how one application ranks drivers. The rules themselves stay with the driver. |
|
||||
| `provisioning` (`QEMU_GUEST_AGENTS`, cloud-init network-config, decompressors) | Which agent a guest gets and how its network is configured is a guest profile. The hypervisor driver is handed the packages and commands. |
|
||||
| `TYPE_LABEL`, `REBOOT_SETTLE_SECONDS` | A display name and a polling delay. |
|
||||
| firmware catalogues (napalm-netgear's `netgear_plus`) | What a vendor publishes is not something the device reports. |
|
||||
| `run_device_action` | Replaced by the templates above. |
|
||||
|
||||
`OSDriver.ROLE` stops defaulting to `"linux"`. Each OS driver states its own role.
|
||||
|
||||
## Rules, enforced by tests in this package
|
||||
|
||||
1. Roles and capabilities own no runtime method; a template runs only declared steps.
|
||||
2. No module imports `paramiko`, `socket`, `subprocess` or `asyncio`, and none defines a
|
||||
`*_COMMAND`.
|
||||
3. No module, string or default names an application.
|
||||
4. Role methods return TypedDicts, not `Any`.
|
||||
|
||||
Each rule is added with the step that makes it pass.
|
||||
|
||||
## Migration
|
||||
|
||||
This package cannot keep re-exports pointing at napalm-unix: the contract package would
|
||||
then depend on its implementation. So a moved name is first deprecated here and then
|
||||
removed, and nothing breaks in between.
|
||||
|
||||
| Step | Change |
|
||||
|---|---|
|
||||
| 1 | The application moves to 4.9 and the drivers that need it; napalm-bsd stops passing stdin to a privileged command while a sudo password is set. |
|
||||
| 2 | This document; the application's architecture docs and this README follow it. |
|
||||
| 3 | `SnmpHealthProfile`. The application polls on its own; the four switch and phone drivers declare their OIDs as data. `HealthMetricsMixin` is deprecated. |
|
||||
| 4 | The fix templates. napalm-linux and napalm-bsd implement the steps; the application calls the templates with its values. `run_device_action` is deprecated. The other drivers follow, one issue each. |
|
||||
| 5 | Ping sweep, scoring, provisioning, `TYPE_LABEL` and `REBOOT_SETTLE_SECONDS` move to the application and are deprecated here. Drivers report facts instead of warnings. |
|
||||
| 6 | napalm-unix, with the moved code and its tests, the unified hooks and the frame helper. napalm-linux and napalm-bsd import from it; the moved names are deprecated here. |
|
||||
| 7 | 5.0 removes everything deprecated. It also splits `models.py` by area, types the remaining `Dict[str, Any]` role returns, declares what drivers already offer undeclared (`get_networks`, `search_packages`, `first_boot_pending`, the base-system update entry), and makes `install_package` declare the result drivers return. |
|
||||
| 8 | The README shrinks to the concepts; the reader details move with the readers. |
|
||||
|
||||
## Proposals this replaces
|
||||
|
||||
Several open issues proposed adding shared driver code here. Under this structure:
|
||||
|
||||
| Issue | Goes to |
|
||||
|---|---|
|
||||
| #8 netmiko session | not here: a CLI-driver helper package, or left in the drivers |
|
||||
| #9 config mode for CLI switches | not here: a CLI-driver helper package, or left in the drivers |
|
||||
| #10 LLDP rows | the projection of rows into NAPALM's two shapes stays here; the `lldpctl` parser goes to napalm-unix |
|
||||
| #11 CLI text helpers | not here; the natural port sort, which `lag` already has, may become public |
|
||||
| #12 ping | the reply builder stays here; the BusyBox and iputils parser goes to napalm-unix |
|
||||
| #13 Linux userland | napalm-unix; the snmpd part becomes the SNMP template's steps |
|
||||
| #14 `run_device_action` | replaced by the fix templates |
|
||||
| #22 hooks onto `run_command` | step 6 |
|
||||
@@ -22,6 +22,7 @@ driver do X".
|
||||
Role bases -- what a device *is*:
|
||||
|
||||
* :class:`~napalm_device_types.access_point.AccessPointDriver`
|
||||
* :class:`~napalm_device_types.backup_server.BackupServerDriver`
|
||||
* :class:`~napalm_device_types.switch.SwitchDriver`
|
||||
* :class:`~napalm_device_types.firewall.FirewallDriver`
|
||||
* :class:`~napalm_device_types.hypervisor.HypervisorDriver`
|
||||
@@ -57,6 +58,8 @@ instead of being restated on every role that happens to need it:
|
||||
* :class:`~napalm_device_types.service_clusters.ServiceClustersMixin`
|
||||
* :class:`~napalm_device_types.service_packages.ServicePackagesMixin`
|
||||
* :class:`~napalm_device_types.sudo_history.SudoHistoryMixin`
|
||||
* :class:`~napalm_device_types.scheduled_jobs.ScheduledJobsMixin`
|
||||
* :class:`~napalm_device_types.scheduled_jobs_posix.PosixScheduledJobsMixin`
|
||||
* :class:`~napalm_device_types.services.ServiceControlMixin`
|
||||
* :class:`~napalm_device_types.systemd.SystemdServicesMixin`
|
||||
* :class:`~napalm_device_types.updates.UpdateMixin`
|
||||
@@ -78,6 +81,22 @@ from napalm_device_types.apt_proxy import (
|
||||
same_proxy_url,
|
||||
)
|
||||
from napalm_device_types.arp import ARP_AGE_UNKNOWN
|
||||
from napalm_device_types.backup_server import BackupServerDriver
|
||||
from napalm_device_types.backups import (
|
||||
BACKUP_JOB_KINDS,
|
||||
BackupDatastoreDict,
|
||||
BackupGroupDict,
|
||||
BackupJobKind,
|
||||
BackupRemoteDict,
|
||||
BackupRunState,
|
||||
BackupServerJobDict,
|
||||
BackupServerRefDict,
|
||||
BackupSnapshotDict,
|
||||
BackupSnapshotRefDict,
|
||||
BackupTaskDict,
|
||||
BackupTaskStatusDict,
|
||||
BackupVerifyState,
|
||||
)
|
||||
from napalm_device_types.base import DeviceTypeDriver, FingerprintRule, PortSpec, rules_max_weight
|
||||
from napalm_device_types.channel import (
|
||||
ByteStream,
|
||||
@@ -135,6 +154,13 @@ from napalm_device_types.listening import (
|
||||
ListeningSocketsUnavailable,
|
||||
parse_listening_sockets,
|
||||
)
|
||||
from napalm_device_types.lldp import (
|
||||
LldpNeighborDict,
|
||||
LldpNeighborRow,
|
||||
lldp_neighbors,
|
||||
lldp_neighbors_detail,
|
||||
)
|
||||
from napalm_device_types.lldpctl import LLDPCTL_KEYVALUE_COMMAND, parse_lldpctl_keyvalue
|
||||
from napalm_device_types.local_accounts import (
|
||||
LocalAccountChangeDict,
|
||||
LocalAccountDict,
|
||||
@@ -181,6 +207,17 @@ from napalm_device_types.ping_sweep import PingSweepMixin, driver_supports_ping
|
||||
from napalm_device_types.provisioning import VMProvisionIncomplete
|
||||
from napalm_device_types.residential_gateway import ResidentialGatewayDriver
|
||||
from napalm_device_types.roles import primary_role_of, role_keys_of, roles_of
|
||||
from napalm_device_types.scheduled_jobs import (
|
||||
ScheduledJobChanged,
|
||||
ScheduledJobDict,
|
||||
ScheduledJobRefused,
|
||||
ScheduledJobResultDict,
|
||||
ScheduledJobsMixin,
|
||||
ScheduledJobSpecDict,
|
||||
ScheduledJobsReadingDict,
|
||||
job_digest,
|
||||
)
|
||||
from napalm_device_types.scheduled_jobs_posix import PosixScheduledJobsMixin
|
||||
from napalm_device_types.service_clusters import (
|
||||
SERVICE_CLUSTERS_COMMAND,
|
||||
ServiceClustersMixin,
|
||||
@@ -249,6 +286,19 @@ from napalm_device_types.vm_migration import MigrationPreflightDict
|
||||
from napalm_device_types.vm_stop_request import VmStopRequestMixin
|
||||
|
||||
__all__ = [
|
||||
"BACKUP_JOB_KINDS",
|
||||
"BackupDatastoreDict",
|
||||
"BackupGroupDict",
|
||||
"BackupJobKind",
|
||||
"BackupRemoteDict",
|
||||
"BackupRunState",
|
||||
"BackupServerJobDict",
|
||||
"BackupServerRefDict",
|
||||
"BackupSnapshotDict",
|
||||
"BackupSnapshotRefDict",
|
||||
"BackupTaskDict",
|
||||
"BackupTaskStatusDict",
|
||||
"BackupVerifyState",
|
||||
"BackupDict",
|
||||
"BackupJobDict",
|
||||
"ClusterDict",
|
||||
@@ -288,6 +338,7 @@ __all__ = [
|
||||
"SnmpMemorySpec",
|
||||
"SnmpValueSpec",
|
||||
"AccessPointDriver",
|
||||
"BackupServerDriver",
|
||||
"ByteStream",
|
||||
"CommandChannelMixin",
|
||||
"CommandResult",
|
||||
@@ -348,6 +399,12 @@ __all__ = [
|
||||
"ListeningSocketsMixin",
|
||||
"ListeningSocketsUnavailable",
|
||||
"parse_listening_sockets",
|
||||
"LLDPCTL_KEYVALUE_COMMAND",
|
||||
"LldpNeighborDict",
|
||||
"LldpNeighborRow",
|
||||
"lldp_neighbors",
|
||||
"lldp_neighbors_detail",
|
||||
"parse_lldpctl_keyvalue",
|
||||
"LocalAccountsMixin",
|
||||
"LocalAccountDict",
|
||||
"LocalAccountChangeDict",
|
||||
@@ -356,6 +413,15 @@ __all__ = [
|
||||
"PortSpec",
|
||||
"ResidentialGatewayDriver",
|
||||
"ServiceControlMixin",
|
||||
"PosixScheduledJobsMixin",
|
||||
"ScheduledJobChanged",
|
||||
"ScheduledJobDict",
|
||||
"ScheduledJobRefused",
|
||||
"ScheduledJobResultDict",
|
||||
"ScheduledJobSpecDict",
|
||||
"ScheduledJobsMixin",
|
||||
"ScheduledJobsReadingDict",
|
||||
"job_digest",
|
||||
"StorageDriver",
|
||||
"SwitchDriver",
|
||||
"SYSTEMD_SERVICES_COMMAND",
|
||||
|
||||
@@ -0,0 +1,222 @@
|
||||
"""Crontabs and anacrontabs as text: the jobs in them, and changing one line of them.
|
||||
|
||||
Pure: no I/O. :mod:`napalm_device_types.scheduled_jobs_posix` reads the files and
|
||||
writes them back; this module says what is in them and what the new text is.
|
||||
|
||||
**A line's key survives switching it off.** A job switched off is the same line
|
||||
behind the marker :data:`DISABLED_MARKER`; the key is built from the line without
|
||||
it, so the job keeps its key and the marker is all that changes. Two equal lines
|
||||
in one file get ``:2``, ``:3`` after the first.
|
||||
|
||||
**A job an application wrote** is the line after ``# napalm-scheduled-job: <name>``
|
||||
and is keyed ``job:<name>``, whatever its schedule.
|
||||
|
||||
**Everything else is left as it was**: comments, blank lines, variables, lines
|
||||
cron would not take.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import re
|
||||
from dataclasses import dataclass, field
|
||||
from shlex import quote
|
||||
|
||||
DISABLED_MARKER = "#napalm-disabled: "
|
||||
JOB_MARKER = "# napalm-scheduled-job: "
|
||||
|
||||
_VARIABLE = re.compile(r"^([A-Za-z_][A-Za-z0-9_]*)\s*=\s*(.*)$")
|
||||
_ANACRON_PERIOD = re.compile(r"^(\d+|@(daily|weekly|monthly|yearly|annually))$")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CronLine:
|
||||
"""One job in a crontab: where it is, what it says, and what was set before it."""
|
||||
|
||||
index: int # the line's number in the text, from 0
|
||||
key: str
|
||||
schedule: str
|
||||
run_as: str
|
||||
command: str
|
||||
enabled: bool
|
||||
env: dict[str, str] = field(default_factory=dict)
|
||||
name: str = "" # the name an application gave it
|
||||
description: str = ""
|
||||
|
||||
|
||||
def _unquote(value: str) -> str:
|
||||
value = value.strip()
|
||||
if len(value) >= 2 and value[0] == value[-1] and value[0] in "'\"":
|
||||
return value[1:-1]
|
||||
return value
|
||||
|
||||
|
||||
def _digest(text: str) -> str:
|
||||
return hashlib.sha256(" ".join(text.split()).encode()).hexdigest()[:8]
|
||||
|
||||
|
||||
def _split(content: str, *, system: bool, user: str) -> tuple[str, str, str] | None:
|
||||
"""(schedule, run_as, command) of a cron line; None for one cron would not take."""
|
||||
fields = 1 if content.startswith("@") else 5
|
||||
owner = 1 if system else 0
|
||||
parts = content.split(None, fields + owner)
|
||||
if len(parts) < fields + owner + 1:
|
||||
return None
|
||||
schedule = " ".join(parts[:fields])
|
||||
run_as = parts[fields] if system else user
|
||||
return schedule, run_as, parts[-1]
|
||||
|
||||
|
||||
def _anacron_split(content: str) -> tuple[str, str, str] | None:
|
||||
"""(period, job identifier, command) of an anacrontab line."""
|
||||
parts = content.split(None, 3)
|
||||
if len(parts) < 4 or not _ANACRON_PERIOD.match(parts[0]) or not parts[1].isdigit():
|
||||
return None
|
||||
return parts[0], parts[2], parts[3]
|
||||
|
||||
|
||||
@dataclass
|
||||
class _Scan:
|
||||
"""What is set while a text is read line by line."""
|
||||
|
||||
env: dict[str, str] = field(default_factory=dict)
|
||||
seen: dict[str, int] = field(default_factory=dict)
|
||||
name: str = ""
|
||||
description: str = ""
|
||||
|
||||
def key(self, base: str) -> str:
|
||||
n = self.seen.get(base, 0) + 1
|
||||
self.seen[base] = n
|
||||
return base if n == 1 else f"{base}:{n}"
|
||||
|
||||
def take_name(self) -> tuple[str, str]:
|
||||
named = self.name, self.description
|
||||
self.name, self.description = "", ""
|
||||
return named
|
||||
|
||||
|
||||
def _content(raw: str) -> tuple[str, bool] | None:
|
||||
"""The job text of *raw* and whether it is on; None for a comment or a blank."""
|
||||
line = raw.strip()
|
||||
if line.startswith(DISABLED_MARKER.strip()):
|
||||
return line[len(DISABLED_MARKER.strip()) :].strip(), False
|
||||
if not line or line.startswith("#"):
|
||||
return None
|
||||
return line, True
|
||||
|
||||
|
||||
def _remember(scan: _Scan, raw: str) -> bool:
|
||||
"""Take a variable or an application's marker from *raw*; whether it was one."""
|
||||
line = raw.strip()
|
||||
if line.startswith(JOB_MARKER.strip()):
|
||||
name, _, description = line[len(JOB_MARKER.strip()) :].strip().partition(" ")
|
||||
scan.name, scan.description = name, description.strip()
|
||||
return True
|
||||
variable = _VARIABLE.match(line)
|
||||
if variable:
|
||||
scan.env[variable[1]] = _unquote(variable[2])
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def parse_crontab(path: str, text: str, *, system: bool, user: str = "") -> list[CronLine]:
|
||||
"""The jobs of the crontab *text* at *path*.
|
||||
|
||||
*system*: a sixth field names the user (``/etc/crontab``, ``/etc/cron.d``);
|
||||
otherwise every job runs as *user*.
|
||||
"""
|
||||
scan, jobs = _Scan(), []
|
||||
for index, raw in enumerate(text.splitlines()):
|
||||
if _remember(scan, raw):
|
||||
continue
|
||||
found = _content(raw)
|
||||
split = _split(found[0], system=system, user=user) if found else None
|
||||
if found is None or split is None:
|
||||
continue
|
||||
content, enabled = found
|
||||
name, description = scan.take_name()
|
||||
key = f"job:{name}" if name else scan.key(f"cron:{path}:{_digest(content)}")
|
||||
schedule, run_as, command = split
|
||||
jobs.append(
|
||||
CronLine(
|
||||
index, key, schedule, run_as, command, enabled, dict(scan.env), name, description
|
||||
)
|
||||
)
|
||||
return jobs
|
||||
|
||||
|
||||
def parse_anacrontab(text: str) -> list[CronLine]:
|
||||
"""The jobs of an anacrontab: period, delay, identifier, command; all run as root."""
|
||||
scan, jobs = _Scan(), []
|
||||
for index, raw in enumerate(text.splitlines()):
|
||||
if _remember(scan, raw):
|
||||
continue
|
||||
found = _content(raw)
|
||||
split = _anacron_split(found[0]) if found else None
|
||||
if found is None or split is None:
|
||||
continue
|
||||
period, identifier, command = split
|
||||
key = scan.key(f"anacron:{identifier}")
|
||||
jobs.append(CronLine(index, key, period, "root", command, found[1], dict(scan.env)))
|
||||
return jobs
|
||||
|
||||
|
||||
def _replace(text: str, index: int, lines: list[str]) -> str:
|
||||
rows = text.splitlines()
|
||||
rows[index : index + 1] = lines
|
||||
return "\n".join(rows) + "\n"
|
||||
|
||||
|
||||
def switched(text: str, line: CronLine, *, enabled: bool) -> str:
|
||||
"""*text* with *line* switched on or off: the marker added or taken away, nothing else."""
|
||||
raw = text.splitlines()[line.index]
|
||||
found = _content(raw)
|
||||
if found is None: # pragma: no cover - a CronLine always points at a job
|
||||
raise ValueError(f"line {line.index} is no job")
|
||||
content = found[0]
|
||||
return _replace(text, line.index, [content if enabled else DISABLED_MARKER + content])
|
||||
|
||||
|
||||
def marked_lines(name: str, description: str, line: str, *, enabled: bool) -> list[str]:
|
||||
"""An application's job as crontab lines: its marker, then the job, maybe switched off."""
|
||||
marker = f"{JOB_MARKER}{name} {description}".rstrip()
|
||||
return [marker, line if enabled else DISABLED_MARKER + line]
|
||||
|
||||
|
||||
def with_job(text: str, jobs: list[CronLine], name: str, lines: list[str]) -> str:
|
||||
"""*text* with the application's job *name* as *lines*: replaced, or added at the end."""
|
||||
current = next((j for j in jobs if j.name == name), None)
|
||||
if current is None:
|
||||
body = text if not text or text.endswith("\n") else text + "\n"
|
||||
return body + "\n".join(lines) + "\n"
|
||||
rows = text.splitlines()
|
||||
rows[current.index - 1 : current.index + 1] = lines
|
||||
return "\n".join(rows) + "\n"
|
||||
|
||||
|
||||
def without_job(text: str, jobs: list[CronLine], name: str) -> str:
|
||||
"""*text* without the application's job *name* and its marker."""
|
||||
current = next(j for j in jobs if j.name == name)
|
||||
rows = text.splitlines()
|
||||
del rows[current.index - 1 : current.index + 1]
|
||||
return "\n".join(rows) + ("\n" if rows else "")
|
||||
|
||||
|
||||
def _unescape(part: str) -> str:
|
||||
return part.replace("\\%", "%")
|
||||
|
||||
|
||||
def run_script(command: str, env: dict[str, str]) -> str:
|
||||
"""*command* as ``sh -c`` takes it, the way cron would run it.
|
||||
|
||||
The crontab's variables are exported first. The first ``%`` that is not
|
||||
escaped ends the command; what follows is its stdin, with every further
|
||||
``%`` a newline, and ``\\%`` is a ``%``.
|
||||
"""
|
||||
parts = re.split(r"(?<!\\)%", command)
|
||||
exports = "".join(f"export {k}={quote(v)}; " for k, v in env.items() if k != "MAILTO")
|
||||
head = _unescape(parts[0])
|
||||
if len(parts) == 1:
|
||||
return exports + head
|
||||
stdin = "\n".join(_unescape(p) for p in parts[1:])
|
||||
return f"{exports}printf '%s\\n' {quote(stdin)} | {{ {head}\n}}"
|
||||
@@ -0,0 +1,417 @@
|
||||
"""The scheduled-jobs report of a POSIX host: the command that prints it, and its parse.
|
||||
|
||||
:mod:`napalm_device_types.scheduled_jobs_posix` sends :func:`reading_command` and
|
||||
hands what came back to :func:`parse_reading`. Pure: no I/O.
|
||||
|
||||
**One framed report, as root.** Every crontab, cron.d file, anacrontab and job
|
||||
script is printed behind a header line and with ``|`` before each of its lines,
|
||||
so no line of a file can pass for a header; run-parts scripts by their execute
|
||||
bit and checksum; systemd's timers and the units they start by ``systemctl show``;
|
||||
then the time zone and which package owns each definition file. A report whose
|
||||
end marker is missing raises: a list cut short must never read as jobs that went
|
||||
away.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import UTC, datetime
|
||||
from shlex import quote
|
||||
|
||||
from napalm_device_types._crontab import JOB_MARKER, CronLine, parse_anacrontab, parse_crontab
|
||||
from napalm_device_types.scheduled_jobs import DELETE, EDIT, job_digest
|
||||
from napalm_device_types.terminal import strip_terminal_codes
|
||||
|
||||
_BEGIN = "SJ_BEGIN"
|
||||
_END = "SJ_END"
|
||||
SCHEDULE_HEADER = "# schedule: "
|
||||
|
||||
TIMER_PROPERTIES = (
|
||||
"Id,Unit,Description,TimersCalendar,TimersMonotonic,NextElapseUSecRealtime,"
|
||||
"LastTriggerUSec,UnitFileState,ActiveState,FragmentPath"
|
||||
)
|
||||
UNIT_PROPERTIES = "Id,ExecStart,User,Result,FragmentPath"
|
||||
|
||||
#: The names run-parts runs (Debian's ``--lsbsysinit``-less default).
|
||||
_RUN_PARTS_NAME = re.compile(r"^[A-Za-z0-9_-]+$")
|
||||
_SECTION = re.compile(r"^\[([a-z-]+)(?: (.*))?\]$")
|
||||
|
||||
#: Reads definition paths on stdin, prints ``<package>: <path>`` for each a package owns.
|
||||
OWNERS_SH = (
|
||||
"if command -v dpkg >/dev/null 2>&1; then xargs dpkg -S 2>/dev/null; "
|
||||
"elif command -v rpm >/dev/null 2>&1; then while read -r p; do "
|
||||
'o=$(rpm -qf --qf \'%{NAME}\' -- "$p" 2>/dev/null) && printf \'%s: %s\\n\' "$o" "$p"; done; '
|
||||
"elif command -v apk >/dev/null 2>&1; then while read -r p; do "
|
||||
'o=$(apk info -qW -- "$p" 2>/dev/null) && [ -n "$o" ] && printf \'%s: %s\\n\' "$o" "$p"; done; '
|
||||
"else cat >/dev/null; fi"
|
||||
)
|
||||
|
||||
|
||||
#: Prints the host's time zone: an IANA name, or a path ending in ``zoneinfo/<name>``.
|
||||
TIME_ZONE_SH = "cat /etc/timezone 2>/dev/null || readlink /etc/localtime 2>/dev/null"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Paths:
|
||||
"""Where a host keeps its scheduled jobs."""
|
||||
|
||||
system_files: tuple[str, ...]
|
||||
cron_d_dirs: tuple[str, ...]
|
||||
spool_dirs: tuple[str, ...]
|
||||
run_parts: tuple[tuple[str, str], ...] # (period, directory)
|
||||
anacrontab: str
|
||||
script_dir: str
|
||||
systemd_marker: str = "/run/systemd/system"
|
||||
owners_sh: str = OWNERS_SH
|
||||
time_zone_sh: str = TIME_ZONE_SH
|
||||
|
||||
|
||||
@dataclass
|
||||
class Reading:
|
||||
"""What one report said, beyond the jobs: what writes need to know."""
|
||||
|
||||
jobs: list[dict[str, object]] = field(default_factory=list)
|
||||
time_zone: str | None = None
|
||||
systemd: bool = False
|
||||
#: per key: the unit a timer starts, the crontab line a cron job is
|
||||
units: dict[str, str] = field(default_factory=dict)
|
||||
lines: dict[str, CronLine] = field(default_factory=dict)
|
||||
|
||||
|
||||
def _q(path: str) -> str:
|
||||
return quote(path)
|
||||
|
||||
|
||||
def _files(section: str, pattern: str) -> str:
|
||||
return (
|
||||
f'for f in {pattern}; do [ -f "$f" ] || continue; printf \'[%s %s]\\n\' {section} "$f"; '
|
||||
# GNU sed keeps a missing last newline; without one of its own the next
|
||||
# header would run on into the file's last line.
|
||||
'sed \'s/^/|/\' "$f"; [ -n "$(tail -c 1 "$f")" ] && echo; P="$P\n$f"; done; '
|
||||
)
|
||||
|
||||
|
||||
def _run_parts(period: str, directory: str) -> str:
|
||||
return (
|
||||
f'for f in {_q(directory)}/*; do [ -f "$f" ] || continue; '
|
||||
'[ -x "$f" ] && x=1 || x=0; c=$(cksum < "$f" | cut -d" " -f1); '
|
||||
f'printf \'[run-parts %s %s %s %s]\\n\' {period} "$x" "$c" "$f"; P="$P\n$f"; done; '
|
||||
)
|
||||
|
||||
|
||||
def _timers(paths: Paths) -> str:
|
||||
show = f"systemctl show --no-pager -p {TIMER_PROPERTIES}"
|
||||
units = f"systemctl show --no-pager -p {UNIT_PROPERTIES}"
|
||||
return (
|
||||
f"if [ -d {_q(paths.systemd_marker)} ]; then echo '[systemd]'; "
|
||||
"t=$(systemctl list-unit-files --type=timer --no-legend --no-pager 2>/dev/null "
|
||||
"| awk '$1 !~ /@\\./ {print $1}'); "
|
||||
'if [ -n "$t" ]; then '
|
||||
f"s=$({show} --timestamp=unix $t 2>/dev/null || {show} $t 2>/dev/null); "
|
||||
"echo '[timers]'; printf '%s\\n' \"$s\" | sed 's/^/|/'; "
|
||||
"u=$(printf '%s\\n' \"$s\" | sed -n 's/^Unit=//p'); "
|
||||
f"echo '[units]'; {units} $u 2>/dev/null | sed 's/^/|/'; "
|
||||
"P=\"$P\n$(printf '%s\\n' \"$s\" | sed -n 's/^FragmentPath=//p')\"; fi; fi; "
|
||||
)
|
||||
|
||||
|
||||
def reading_command(paths: Paths) -> str:
|
||||
"""One POSIX ``sh`` script that prints every scheduled job of the host; run it as root."""
|
||||
parts = ["printf '%s%s\\n' SJ_ BEGIN; P=''; "]
|
||||
parts += [_files("system", _q(f)) for f in paths.system_files]
|
||||
parts += [_files("system", f"{_q(d)}/*") for d in paths.cron_d_dirs]
|
||||
parts += [_files("user", f"{_q(d)}/*") for d in paths.spool_dirs]
|
||||
parts.append(_files("anacron", _q(paths.anacrontab)))
|
||||
parts += [_run_parts(period, d) for period, d in paths.run_parts]
|
||||
parts.append(_files("script", f"{_q(paths.script_dir)}/*"))
|
||||
parts.append(_timers(paths))
|
||||
parts.append(
|
||||
f"echo '[tz]'; {{ {paths.time_zone_sh}; }}; "
|
||||
f"echo '[owners]'; printf '%s\\n' \"$P\" | sed '/^$/d' | {{ {paths.owners_sh}; }}; "
|
||||
"printf '%s%s\\n' SJ_ END"
|
||||
)
|
||||
return "".join(parts)
|
||||
|
||||
|
||||
# -- parse ------------------------------------------------------------------------------------
|
||||
|
||||
|
||||
@dataclass
|
||||
class _Section:
|
||||
name: str
|
||||
args: str
|
||||
lines: list[str] = field(default_factory=list)
|
||||
|
||||
|
||||
def _sections(output: str) -> list[_Section]:
|
||||
text = strip_terminal_codes(output)
|
||||
begin, end = text.rfind(_BEGIN), text.rfind(_END)
|
||||
if begin < 0 or end < begin:
|
||||
raise ValueError("the scheduled-jobs report is cut short (no end marker)")
|
||||
sections: list[_Section] = []
|
||||
for line in text[begin + len(_BEGIN) : end].splitlines():
|
||||
header = _SECTION.match(line)
|
||||
if header:
|
||||
sections.append(_Section(header[1], header[2] or ""))
|
||||
elif sections:
|
||||
sections[-1].lines.append(line[1:] if line.startswith("|") else line)
|
||||
return sections
|
||||
|
||||
|
||||
def _owned(lines: list[str]) -> set[str]:
|
||||
owned: set[str] = set()
|
||||
for line in lines:
|
||||
package, sep, path = line.partition(": ")
|
||||
if sep and path.startswith("/") and not package.startswith("diversion"):
|
||||
owned.add(path.strip())
|
||||
return owned
|
||||
|
||||
|
||||
def _time_zone(lines: list[str]) -> str | None:
|
||||
value = next((ln.strip() for ln in lines if ln.strip()), "")
|
||||
if "zoneinfo/" in value:
|
||||
value = value.split("zoneinfo/", 1)[1]
|
||||
return value or None
|
||||
|
||||
|
||||
def _blocks(lines: list[str]) -> list[dict[str, list[str]]]:
|
||||
blocks: list[dict[str, list[str]]] = [{}]
|
||||
for line in lines:
|
||||
if not line.strip():
|
||||
blocks.append({})
|
||||
continue
|
||||
key, sep, value = line.partition("=")
|
||||
if sep:
|
||||
blocks[-1].setdefault(key, []).append(value)
|
||||
return [b for b in blocks if b.get("Id")]
|
||||
|
||||
|
||||
def _one(block: dict[str, list[str]], key: str) -> str:
|
||||
return (block.get(key) or [""])[0].strip()
|
||||
|
||||
|
||||
def _timestamp(value: str) -> str | None:
|
||||
"""systemd's ``@<seconds>`` (or a UTC time) as ISO 8601; None when it never was."""
|
||||
value = value.strip()
|
||||
if value.startswith("@") and value[1:].isdigit() and int(value[1:]) > 0:
|
||||
return datetime.fromtimestamp(int(value[1:]), UTC).isoformat()
|
||||
match = re.search(r"(\d{4}-\d\d-\d\d \d\d:\d\d:\d\d) UTC$", value)
|
||||
if match:
|
||||
return datetime.strptime(match[1], "%Y-%m-%d %H:%M:%S").replace(tzinfo=UTC).isoformat()
|
||||
return None
|
||||
|
||||
|
||||
def _timer_entries(block: dict[str, list[str]], key: str) -> list[str]:
|
||||
"""``{ OnCalendar=daily ; next_elapse=… }`` → ``OnCalendar=daily``, one per entry.
|
||||
|
||||
``systemctl show`` names a monotonic setting by its property (``OnUnitActiveUSec``);
|
||||
it is given as the unit file says it (``OnUnitActiveSec``).
|
||||
"""
|
||||
entries = (v.strip(" {}").split(" ; ")[0].strip() for v in block.get(key, []) if v.strip())
|
||||
return [re.sub(r"^(On\w+)USec=", r"\1Sec=", e) for e in entries]
|
||||
|
||||
|
||||
def _argv(exec_start: list[str]) -> str:
|
||||
commands = []
|
||||
for value in exec_start:
|
||||
match = re.search(r"argv\[\]=(.*?) ; ", value)
|
||||
if match:
|
||||
commands.append(match[1].strip())
|
||||
return "; ".join(commands)
|
||||
|
||||
|
||||
@dataclass
|
||||
class _Script:
|
||||
name: str
|
||||
description: str
|
||||
schedule: str
|
||||
command: str
|
||||
|
||||
|
||||
def _script(path: str, lines: list[str]) -> _Script | None:
|
||||
"""An application's job script: its marker, schedule and command; None for any other file."""
|
||||
marker = next((ln for ln in lines if ln.startswith(JOB_MARKER)), None)
|
||||
schedule = next((ln for ln in lines if ln.startswith(SCHEDULE_HEADER)), None)
|
||||
if marker is None or schedule is None:
|
||||
return None
|
||||
name, _, description = marker[len(JOB_MARKER) :].partition(" ")
|
||||
body = [ln for ln in lines if ln not in (marker, schedule) and not ln.startswith("#!")]
|
||||
return _Script(
|
||||
name, description.strip(), schedule[len(SCHEDULE_HEADER) :], "\n".join(body).strip()
|
||||
)
|
||||
|
||||
|
||||
def _job(
|
||||
*,
|
||||
key: str,
|
||||
kind: str,
|
||||
source: str,
|
||||
run_as: str,
|
||||
schedule: str,
|
||||
schedule_kind: str,
|
||||
command: str,
|
||||
enabled: bool,
|
||||
managed_by: str,
|
||||
actions: list[str],
|
||||
content: str = "",
|
||||
**extra: object,
|
||||
) -> dict[str, object]:
|
||||
job: dict[str, object] = {
|
||||
"key": key,
|
||||
"kind": kind,
|
||||
"source": source,
|
||||
"run_as": run_as,
|
||||
"schedule": schedule,
|
||||
"schedule_kind": schedule_kind,
|
||||
"command": command,
|
||||
"enabled": enabled,
|
||||
"managed_by": managed_by,
|
||||
"actions": actions,
|
||||
}
|
||||
job["digest"] = job_digest(job, content=content)
|
||||
job.update({k: v for k, v in extra.items() if v not in (None, "")})
|
||||
return job
|
||||
|
||||
|
||||
_SWITCH = ["enable", "disable", "run"]
|
||||
_OWN = [*_SWITCH, EDIT, DELETE]
|
||||
|
||||
|
||||
def _cron_jobs(section: _Section, owned: set[str], reading: Reading) -> None:
|
||||
path = section.args
|
||||
text = "\n".join(section.lines)
|
||||
if section.name == "anacron":
|
||||
found = parse_anacrontab(text)
|
||||
else:
|
||||
user = path.rsplit("/", 1)[-1]
|
||||
found = parse_crontab(path, text, system=section.name == "system", user=user)
|
||||
kind = "anacron" if section.name == "anacron" else "cron"
|
||||
for line in found:
|
||||
own = bool(line.name)
|
||||
reading.lines[line.key] = line
|
||||
reading.jobs.append(
|
||||
_job(
|
||||
key=line.key,
|
||||
kind=kind,
|
||||
source=path,
|
||||
run_as=line.run_as,
|
||||
schedule=line.schedule,
|
||||
schedule_kind="period" if kind == "anacron" else "cron",
|
||||
command=line.command,
|
||||
enabled=line.enabled,
|
||||
managed_by="application" if own else ("package" if path in owned else "admin"),
|
||||
actions=_OWN if own else _SWITCH,
|
||||
name=line.name,
|
||||
description=line.description,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _run_parts_job(section: _Section, owned: set[str], reading: Reading) -> None:
|
||||
period, executable, checksum, path = section.args.split(" ", 3)
|
||||
if not _RUN_PARTS_NAME.match(path.rsplit("/", 1)[-1]):
|
||||
return
|
||||
reading.jobs.append(
|
||||
_job(
|
||||
key=f"run-parts:{path}",
|
||||
kind="run_parts",
|
||||
source=path,
|
||||
run_as="root",
|
||||
schedule=period,
|
||||
schedule_kind="period",
|
||||
command=path,
|
||||
enabled=executable == "1",
|
||||
managed_by="package" if path in owned else "admin",
|
||||
actions=_SWITCH,
|
||||
content=checksum,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _timer_job(
|
||||
timer: dict[str, list[str]],
|
||||
unit: dict[str, list[str]],
|
||||
scripts: dict[str, _Script],
|
||||
owned: set[str],
|
||||
) -> dict[str, object]:
|
||||
fragment = _one(timer, "FragmentPath")
|
||||
calendar = _timer_entries(timer, "TimersCalendar")
|
||||
schedule = "; ".join(calendar + _timer_entries(timer, "TimersMonotonic"))
|
||||
command = _argv(unit.get("ExecStart", []))
|
||||
script = scripts.get(command.split(" ", 1)[0])
|
||||
last_run = _timestamp(_one(timer, "LastTriggerUSec"))
|
||||
switchable = _one(timer, "UnitFileState") not in ("static", "generated", "transient")
|
||||
common = {
|
||||
"kind": "systemd_timer",
|
||||
"source": fragment,
|
||||
"run_as": _one(unit, "User") or "root",
|
||||
"enabled": _one(timer, "ActiveState") == "active",
|
||||
"last_run": last_run,
|
||||
"last_result": _one(unit, "Result") or None if last_run else None,
|
||||
"next_run": _timestamp(_one(timer, "NextElapseUSecRealtime")),
|
||||
}
|
||||
if script is not None:
|
||||
return _job(
|
||||
key=f"job:{script.name}",
|
||||
schedule=script.schedule,
|
||||
schedule_kind="cron",
|
||||
command=script.command,
|
||||
managed_by="application",
|
||||
actions=_OWN,
|
||||
name=script.name,
|
||||
description=script.description,
|
||||
**common, # type: ignore[arg-type]
|
||||
)
|
||||
packaged = fragment in owned or fragment.startswith(("/usr/lib/", "/lib/"))
|
||||
return _job(
|
||||
key=f"timer:{_one(timer, 'Id')}",
|
||||
schedule=schedule,
|
||||
schedule_kind="calendar" if calendar else "monotonic",
|
||||
command=command,
|
||||
managed_by="package" if packaged else "admin",
|
||||
actions=_SWITCH if switchable else ["run"],
|
||||
description=_one(timer, "Description"),
|
||||
**common, # type: ignore[arg-type]
|
||||
)
|
||||
|
||||
|
||||
def _timer_jobs(
|
||||
sections: dict[str, _Section], scripts: dict[str, _Script], owned: set[str], reading: Reading
|
||||
) -> None:
|
||||
timers = _blocks(sections["timers"].lines) if "timers" in sections else []
|
||||
units = (
|
||||
{_one(b, "Id"): b for b in _blocks(sections["units"].lines)} if "units" in sections else {}
|
||||
)
|
||||
for timer in timers:
|
||||
unit = units.get(_one(timer, "Unit"), {})
|
||||
job = _timer_job(timer, unit, scripts, owned)
|
||||
reading.units[str(job["key"])] = _one(timer, "Unit")
|
||||
reading.jobs.append(job)
|
||||
|
||||
|
||||
def parse_reading(output: str) -> Reading:
|
||||
"""The jobs, time zone and write context of a report from :func:`reading_command`.
|
||||
|
||||
:raises ValueError: when the report is cut short.
|
||||
"""
|
||||
sections = _sections(output)
|
||||
named = {s.name: s for s in sections if s.name in ("timers", "units", "tz", "owners")}
|
||||
owned = _owned(named["owners"].lines) if "owners" in named else set()
|
||||
reading = Reading(
|
||||
time_zone=_time_zone(named["tz"].lines) if "tz" in named else None,
|
||||
systemd=any(s.name == "systemd" for s in sections),
|
||||
)
|
||||
for section in sections:
|
||||
if section.name in ("system", "user", "anacron"):
|
||||
_cron_jobs(section, owned, reading)
|
||||
elif section.name == "run-parts":
|
||||
_run_parts_job(section, owned, reading)
|
||||
_timer_jobs(named, _scripts(sections), owned, reading)
|
||||
return reading
|
||||
|
||||
|
||||
def _scripts(sections: list[_Section]) -> dict[str, _Script]:
|
||||
"""An application's job scripts, by path."""
|
||||
scripts = ((s.args, _script(s.args, s.lines)) for s in sections if s.name == "script")
|
||||
return {path: script for path, script in scripts if script is not None}
|
||||
@@ -0,0 +1,238 @@
|
||||
"""
|
||||
Abstract base class for backup server drivers.
|
||||
|
||||
Usage::
|
||||
|
||||
from napalm_device_types import BackupServerDriver, StorageDriver
|
||||
|
||||
class ProxmoxBackupDriver(BackupServerDriver, StorageDriver):
|
||||
def get_backup_datastores(self):
|
||||
...
|
||||
"""
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from napalm_device_types.base import DeviceTypeDriver
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - for type checkers only
|
||||
from napalm_device_types.backups import (
|
||||
BackupDatastoreDict,
|
||||
BackupGroupDict,
|
||||
BackupJobKind,
|
||||
BackupRemoteDict,
|
||||
BackupServerJobDict,
|
||||
BackupSnapshotDict,
|
||||
BackupSnapshotRefDict,
|
||||
BackupTaskDict,
|
||||
BackupTaskStatusDict,
|
||||
)
|
||||
|
||||
|
||||
class BackupServerDriver(DeviceTypeDriver):
|
||||
"""
|
||||
Abstract intermediate driver for backup servers (e.g. Proxmox Backup
|
||||
Server): the device hypervisors and hosts write their backups to.
|
||||
|
||||
A backup server keeps its backups in datastores, one group per source (a
|
||||
VM, a container, a host) and one snapshot per backup run, and keeps them
|
||||
healthy with scheduled jobs: garbage collection, verify, prune, sync with
|
||||
other servers and tape. None of that is a storage pool, a volume or a
|
||||
replication job, so it is not modelled through ``StorageDriver``. A server
|
||||
that also manages its own disks and pools lists ``StorageDriver`` as a
|
||||
second role, after this one.
|
||||
|
||||
The actions start a task on the server and return at once with its id;
|
||||
:meth:`get_backup_task` follows it.
|
||||
"""
|
||||
|
||||
#: Stable key netOrk exposes as ``device_class``. The order in which a
|
||||
#: driver lists its role bases is the ranking; see
|
||||
#: :func:`napalm_device_types.roles.primary_role_of`.
|
||||
ROLE: str = "backup_server"
|
||||
TYPE_LABEL: str = "Backup Server"
|
||||
|
||||
if TYPE_CHECKING:
|
||||
# ------------------------------------------------------------------
|
||||
# Read
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def get_backup_datastores(self) -> list[BackupDatastoreDict]:
|
||||
"""
|
||||
Returns the server's datastores with their usage.
|
||||
|
||||
Example::
|
||||
|
||||
[
|
||||
{
|
||||
"name": "store1",
|
||||
"path": "/mnt/datastore/store1",
|
||||
"backend": "local",
|
||||
"total": 4000787030016,
|
||||
"used": 1529008357376,
|
||||
"available": 2471778672640,
|
||||
"estimated_full": 1798761600,
|
||||
"maintenance": None,
|
||||
"comment": None,
|
||||
},
|
||||
]
|
||||
|
||||
:raises Exception: when the server could not be read; never an
|
||||
empty list for a failed read.
|
||||
"""
|
||||
...
|
||||
|
||||
def get_backup_groups(self, datastore: str = "") -> list[BackupGroupDict]:
|
||||
"""
|
||||
Returns the backup groups of every namespace of *datastore*, or of
|
||||
every datastore when it is empty, each with its newest snapshot
|
||||
only. :meth:`get_backup_snapshots` lists all of a group's.
|
||||
|
||||
Example::
|
||||
|
||||
[
|
||||
{
|
||||
"datastore": "store1",
|
||||
"namespace": "",
|
||||
"type": "vm",
|
||||
"id": "101",
|
||||
"owner": "backup@pbs",
|
||||
"count": 14,
|
||||
"newest": {
|
||||
"time": 1793660400,
|
||||
"size": 34359738368,
|
||||
"verify": "ok",
|
||||
"protected": False,
|
||||
"comment": "web-01",
|
||||
},
|
||||
"oldest_time": 1792537200,
|
||||
"comment": None,
|
||||
},
|
||||
]
|
||||
|
||||
:raises Exception: when the server could not be read.
|
||||
"""
|
||||
...
|
||||
|
||||
def get_backup_snapshots(
|
||||
self, datastore: str, namespace: str, group_type: str, group_id: str
|
||||
) -> list[BackupSnapshotDict]:
|
||||
"""
|
||||
Returns every snapshot of one group, newest first.
|
||||
|
||||
:param namespace: ``""`` for the datastore's root.
|
||||
:raises ValueError: when the datastore or the group does not exist.
|
||||
"""
|
||||
...
|
||||
|
||||
def get_backup_jobs(self) -> list[BackupServerJobDict]:
|
||||
"""
|
||||
Returns the server's scheduled jobs of every kind, garbage
|
||||
collection included (one per datastore, ``id`` = the datastore).
|
||||
|
||||
Example::
|
||||
|
||||
[
|
||||
{
|
||||
"id": "store1",
|
||||
"kind": "gc",
|
||||
"datastore": "store1",
|
||||
"namespace": None,
|
||||
"schedule": "daily",
|
||||
"enabled": True,
|
||||
"last_run": 1793664000,
|
||||
"last_state": "ok",
|
||||
"last_error": None,
|
||||
"next_run": 1793750400,
|
||||
"remote": None,
|
||||
"direction": None,
|
||||
"comment": None,
|
||||
},
|
||||
{
|
||||
"id": "offsite",
|
||||
"kind": "sync",
|
||||
"datastore": "store1",
|
||||
"namespace": "",
|
||||
"schedule": "sat 03:00",
|
||||
"enabled": True,
|
||||
"last_run": 1793502000,
|
||||
"last_state": "error",
|
||||
"last_error": "connection refused",
|
||||
"next_run": 1794106800,
|
||||
"remote": "pbs-offsite",
|
||||
"direction": "push",
|
||||
"comment": None,
|
||||
},
|
||||
]
|
||||
|
||||
:raises Exception: when the server could not be read.
|
||||
"""
|
||||
...
|
||||
|
||||
def get_backup_remotes(self) -> list[BackupRemoteDict]:
|
||||
"""
|
||||
Returns the other backup servers this one syncs with, without their
|
||||
secrets.
|
||||
|
||||
:raises Exception: when the server could not be read.
|
||||
"""
|
||||
...
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Actions
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def start_garbage_collection(self, datastore: str) -> BackupTaskDict:
|
||||
"""
|
||||
Starts garbage collection on *datastore*.
|
||||
|
||||
:raises ValueError: when the datastore does not exist.
|
||||
:raises RuntimeError: when the server refuses (e.g. one is running).
|
||||
"""
|
||||
...
|
||||
|
||||
def run_backup_job(self, kind: BackupJobKind, job_id: str) -> BackupTaskDict:
|
||||
"""
|
||||
Runs a scheduled job now, outside its schedule.
|
||||
|
||||
:param kind: a :data:`~napalm_device_types.backups.BackupJobKind`;
|
||||
``"gc"`` runs the datastore named *job_id*'s garbage collection.
|
||||
:raises ValueError: when there is no such job.
|
||||
:raises RuntimeError: when the server refuses.
|
||||
"""
|
||||
...
|
||||
|
||||
def verify_backup_snapshot(self, snapshot: BackupSnapshotRefDict) -> BackupTaskDict:
|
||||
"""
|
||||
Verifies one snapshot now.
|
||||
|
||||
:raises ValueError: when the snapshot does not exist.
|
||||
"""
|
||||
...
|
||||
|
||||
def set_backup_snapshot_protected(
|
||||
self, snapshot: BackupSnapshotRefDict, protected: bool
|
||||
) -> None:
|
||||
"""
|
||||
Protects a snapshot from pruning and deletion, or lifts that.
|
||||
|
||||
:raises ValueError: when the snapshot does not exist.
|
||||
"""
|
||||
...
|
||||
|
||||
def delete_backup_snapshot(self, snapshot: BackupSnapshotRefDict) -> None:
|
||||
"""
|
||||
Deletes one snapshot. The data is gone once garbage collection has
|
||||
run; there is no undo.
|
||||
|
||||
:raises ValueError: when the snapshot does not exist.
|
||||
:raises RuntimeError: when it is protected.
|
||||
"""
|
||||
...
|
||||
|
||||
def get_backup_task(self, task: str) -> BackupTaskStatusDict:
|
||||
"""
|
||||
Returns the state and the log so far of a task an action started.
|
||||
|
||||
:raises ValueError: when the server does not know the task.
|
||||
"""
|
||||
...
|
||||
@@ -0,0 +1,119 @@
|
||||
"""The shapes of a backup server (``BackupServerDriver``).
|
||||
|
||||
Apart from ``models.py``, which is over the file size limit. A value the
|
||||
platform does not report is None; it is never guessed.
|
||||
"""
|
||||
|
||||
from typing import Literal, TypedDict, get_args
|
||||
|
||||
#: What a backup server's jobs do. Garbage collection is one of them: it has a
|
||||
#: schedule and a last run like the others.
|
||||
BackupJobKind = Literal["gc", "verify", "prune", "sync", "tape"]
|
||||
BACKUP_JOB_KINDS: tuple[str, ...] = get_args(BackupJobKind)
|
||||
|
||||
#: How a job's or a task's run ended; "running" while it has not.
|
||||
BackupRunState = Literal["ok", "warning", "error", "running"]
|
||||
|
||||
#: The outcome of the last verification of a snapshot.
|
||||
BackupVerifyState = Literal["ok", "failed"]
|
||||
|
||||
|
||||
class BackupDatastoreDict(TypedDict):
|
||||
"""A datastore: where a backup server keeps its backups."""
|
||||
|
||||
name: str
|
||||
path: str | None # local path; for an object-store backend the local cache, if any
|
||||
backend: str # "local", "s3", "removable", ... as the platform names it
|
||||
total: int | None # bytes
|
||||
used: int | None # bytes
|
||||
available: int | None # bytes
|
||||
estimated_full: int | None # unix seconds the platform expects it to be full
|
||||
maintenance: str | None # "read-only", "offline", ...; None when in service
|
||||
comment: str | None
|
||||
|
||||
|
||||
class BackupSnapshotDict(TypedDict):
|
||||
"""One backup of a group: the state of its source at one time."""
|
||||
|
||||
time: int # unix seconds the backup was taken
|
||||
size: int | None # bytes
|
||||
verify: BackupVerifyState | None # None: never verified
|
||||
protected: bool # exempt from pruning and deletion
|
||||
comment: str | None
|
||||
|
||||
|
||||
class BackupGroupDict(TypedDict):
|
||||
"""The backups of one source (a VM, a container, a host) in one namespace."""
|
||||
|
||||
datastore: str
|
||||
namespace: str # "" for the datastore's root
|
||||
type: str # "vm", "ct", "host"
|
||||
id: str # the guest's id ("101") or the host's name
|
||||
owner: str | None # the account that writes the group's backups
|
||||
count: int # snapshots in the group
|
||||
newest: BackupSnapshotDict | None # None for a group without a snapshot
|
||||
oldest_time: int | None # unix seconds of the oldest snapshot
|
||||
comment: str | None
|
||||
|
||||
|
||||
class BackupSnapshotRefDict(TypedDict):
|
||||
"""Names one snapshot, for an action on it."""
|
||||
|
||||
datastore: str
|
||||
namespace: str # "" for the datastore's root
|
||||
group_type: str # BackupGroupDict.type
|
||||
group_id: str # BackupGroupDict.id
|
||||
time: int # BackupSnapshotDict.time
|
||||
|
||||
|
||||
class BackupServerJobDict(TypedDict):
|
||||
"""A scheduled job of a backup server."""
|
||||
|
||||
id: str # for "gc", the datastore's name
|
||||
kind: BackupJobKind
|
||||
datastore: str | None
|
||||
namespace: str | None
|
||||
schedule: str | None # as the platform writes it: "daily", "sat 02:00", "*:0/30"
|
||||
enabled: bool
|
||||
last_run: int | None # unix seconds the last run ended (or started, while running)
|
||||
last_state: BackupRunState | None # None: never ran
|
||||
last_error: str | None
|
||||
next_run: int | None # unix seconds
|
||||
remote: str | None # a sync job's BackupRemoteDict.name
|
||||
direction: Literal["pull", "push"] | None # a sync job's
|
||||
comment: str | None
|
||||
|
||||
|
||||
class BackupRemoteDict(TypedDict):
|
||||
"""Another backup server this one syncs with. Never its secret."""
|
||||
|
||||
name: str
|
||||
host: str
|
||||
port: int | None
|
||||
fingerprint: str | None # of the remote's TLS certificate
|
||||
auth_id: str | None # the account the sync logs in as
|
||||
comment: str | None
|
||||
|
||||
|
||||
class BackupTaskDict(TypedDict):
|
||||
"""A task an action started on the server, to follow with ``get_backup_task``."""
|
||||
|
||||
task: str # the platform's task id
|
||||
|
||||
|
||||
class BackupTaskStatusDict(TypedDict):
|
||||
task: str
|
||||
running: bool
|
||||
state: BackupRunState | None # "running" while it runs
|
||||
error: str | None
|
||||
log: list[str] # the task's log lines so far
|
||||
|
||||
|
||||
class BackupServerRefDict(TypedDict):
|
||||
"""Which backup server and datastore a hypervisor's backup storage writes to."""
|
||||
|
||||
server: str # host name or address, as the hypervisor has it configured
|
||||
port: int | None
|
||||
datastore: str
|
||||
namespace: str | None
|
||||
fingerprint: str | None # of the server's TLS certificate, as the hypervisor pins it
|
||||
@@ -5,6 +5,8 @@ Apart from ``models.py``, which is over the file size limit.
|
||||
|
||||
from typing import NotRequired, TypedDict
|
||||
|
||||
from napalm_device_types.backups import BackupServerRefDict
|
||||
|
||||
|
||||
class ClusterNodeDict(TypedDict):
|
||||
"""One host of a cluster, as its cluster sees it. A value the platform does not
|
||||
@@ -34,6 +36,9 @@ class ClusterStorageDict(TypedDict):
|
||||
nodes: list[str] # the nodes it is available on
|
||||
total: int | None # bytes; None where the platform does not report it
|
||||
used: int | None # bytes
|
||||
# A backup storage that writes to a backup server: which one. Left out for every
|
||||
# other storage; None where the platform does not say.
|
||||
backup_server: NotRequired[BackupServerRefDict | None]
|
||||
|
||||
|
||||
class ClusterNetworkDict(TypedDict):
|
||||
|
||||
@@ -0,0 +1,203 @@
|
||||
"""Cron expressions: what they mean, when they fire next, and as systemd ``OnCalendar``.
|
||||
|
||||
A scheduled job is given as a cron expression on every host, and a host with
|
||||
systemd runs it as a timer. So the expression has to mean the same in three
|
||||
places -- on the host's cron, in a preview of the next runs, and in the
|
||||
``OnCalendar`` lines a timer is written with -- and the meaning lives here once.
|
||||
|
||||
**What cron means is Vixie cron's.** Five fields (minute, hour, day of month,
|
||||
month, day of week), each ``*``, a value, a range ``a-b``, a step ``*/n`` or
|
||||
``a-b/n``, or a list of those; month and weekday names in any case; ``7`` is
|
||||
Sunday as well as ``0``; the ``@hourly`` … ``@yearly`` aliases. When both day
|
||||
fields are restricted -- neither starts with ``*`` -- a day matches if *either*
|
||||
does; otherwise both must (``*/2`` counts as starting with ``*``).
|
||||
|
||||
**``OnCalendar`` by expansion.** Every field is written as its values, so a range
|
||||
with a step needs no systemd syntax of its own, and a timer fires on exactly the
|
||||
minutes cron would. The *either* rule is two ``OnCalendar`` lines, which systemd
|
||||
ORs. ``@reboot`` has no calendar; a timer takes it as ``OnBootSec``.
|
||||
|
||||
**Next runs are in the host's time zone.** A local time the clock skips when
|
||||
summer time starts is left out; cron implementations disagree on it, and a
|
||||
preview that promised it would promise too much.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import date, datetime, timedelta
|
||||
from typing import NamedTuple
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
REBOOT = "@reboot"
|
||||
|
||||
ALIASES = {
|
||||
"@yearly": "0 0 1 1 *",
|
||||
"@annually": "0 0 1 1 *",
|
||||
"@monthly": "0 0 1 * *",
|
||||
"@weekly": "0 0 * * 0",
|
||||
"@daily": "0 0 * * *",
|
||||
"@midnight": "0 0 * * *",
|
||||
"@hourly": "0 * * * *",
|
||||
}
|
||||
|
||||
_MONTHS = ("jan", "feb", "mar", "apr", "may", "jun", "jul", "aug", "sep", "oct", "nov", "dec")
|
||||
_WEEKDAYS = ("sun", "mon", "tue", "wed", "thu", "fri", "sat")
|
||||
_SYSTEMD_WEEKDAYS = ("Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat")
|
||||
|
||||
#: (lowest, highest, names) per field; a name stands for its index plus the lowest.
|
||||
_FIELDS = (
|
||||
(0, 59, ()),
|
||||
(0, 23, ()),
|
||||
(1, 31, ()),
|
||||
(1, 12, _MONTHS),
|
||||
(0, 7, _WEEKDAYS),
|
||||
)
|
||||
|
||||
#: How far ahead a next run is looked for: a 29 February comes within eight years.
|
||||
_HORIZON_DAYS = 8 * 366
|
||||
|
||||
|
||||
class CronSchedule(NamedTuple):
|
||||
"""An expression's values per field, and whether a day matches on either day field."""
|
||||
|
||||
minutes: frozenset[int]
|
||||
hours: frozenset[int]
|
||||
days: frozenset[int]
|
||||
months: frozenset[int]
|
||||
weekdays: frozenset[int] # 0 = Sunday
|
||||
either_day: bool
|
||||
|
||||
|
||||
def is_reboot(expr: str) -> bool:
|
||||
"""Whether *expr* is ``@reboot``: once at boot, no calendar."""
|
||||
return expr.strip().lower() == REBOOT
|
||||
|
||||
|
||||
def _value(token: str, low: int, names: tuple[str, ...]) -> int:
|
||||
if token.isdigit():
|
||||
return int(token)
|
||||
if token.lower() in names:
|
||||
return names.index(token.lower()) + low
|
||||
raise ValueError(f"not a value: {token!r}")
|
||||
|
||||
|
||||
def _part(part: str, low: int, high: int, names: tuple[str, ...]) -> set[int]:
|
||||
span, _, step_text = part.partition("/")
|
||||
step = 1
|
||||
if step_text:
|
||||
if not step_text.isdigit() or int(step_text) == 0:
|
||||
raise ValueError(f"not a step: {part!r}")
|
||||
step = int(step_text)
|
||||
if span == "*":
|
||||
first, last = low, high
|
||||
elif "-" in span:
|
||||
a, _, b = span.partition("-")
|
||||
first, last = _value(a, low, names), _value(b, low, names)
|
||||
else:
|
||||
first = _value(span, low, names)
|
||||
last = high if step_text else first
|
||||
if not low <= first <= last <= high:
|
||||
raise ValueError(f"out of range {low}-{high}: {part!r}")
|
||||
return set(range(first, last + 1, step))
|
||||
|
||||
|
||||
def _field(text: str, low: int, high: int, names: tuple[str, ...]) -> frozenset[int]:
|
||||
values: set[int] = set()
|
||||
for part in text.split(","):
|
||||
if not part:
|
||||
raise ValueError(f"empty list item in {text!r}")
|
||||
values |= _part(part, low, high, names)
|
||||
return frozenset(values)
|
||||
|
||||
|
||||
def parse_cron(expr: str) -> CronSchedule:
|
||||
"""The values *expr* stands for.
|
||||
|
||||
:raises ValueError: for anything Vixie cron would not take, and for
|
||||
``@reboot``, which names no time.
|
||||
"""
|
||||
text = expr.strip()
|
||||
if is_reboot(text):
|
||||
raise ValueError("@reboot names no time; it runs once at boot")
|
||||
if text.startswith("@"):
|
||||
if text.lower() not in ALIASES:
|
||||
raise ValueError(f"unknown alias {text!r}")
|
||||
text = ALIASES[text.lower()]
|
||||
fields = text.split()
|
||||
if len(fields) != len(_FIELDS):
|
||||
raise ValueError(f"a cron expression has five fields, not {len(fields)}: {expr!r}")
|
||||
minutes, hours, days, months, weekdays = (
|
||||
_field(f, low, high, names) for f, (low, high, names) in zip(fields, _FIELDS, strict=True)
|
||||
)
|
||||
if 7 in weekdays:
|
||||
weekdays = (weekdays - {7}) | {0}
|
||||
either = not fields[2].startswith("*") and not fields[4].startswith("*")
|
||||
return CronSchedule(minutes, hours, days, months, frozenset(weekdays), either)
|
||||
|
||||
|
||||
def _day_matches(s: CronSchedule, day: date) -> bool:
|
||||
if day.month not in s.months:
|
||||
return False
|
||||
on_day = day.day in s.days
|
||||
on_weekday = (day.isoweekday() % 7) in s.weekdays
|
||||
return (on_day or on_weekday) if s.either_day else (on_day and on_weekday)
|
||||
|
||||
|
||||
def _local(day: date, hour: int, minute: int, tz: ZoneInfo) -> datetime | None:
|
||||
"""*day* at *hour*:*minute* in *tz*; None when the clock skips that time."""
|
||||
naive = datetime(day.year, day.month, day.day, hour, minute)
|
||||
aware = naive.replace(tzinfo=tz)
|
||||
if aware.astimezone(ZoneInfo("UTC")).astimezone(tz).replace(tzinfo=None) != naive:
|
||||
return None
|
||||
return aware
|
||||
|
||||
|
||||
def next_runs(expr: str, after: datetime, tz: ZoneInfo, count: int = 5) -> list[datetime]:
|
||||
"""The next *count* times *expr* fires after *after*, in *tz*.
|
||||
|
||||
``@reboot`` and a day that never comes (31 February) give none.
|
||||
|
||||
:raises ValueError: for an expression :func:`parse_cron` refuses.
|
||||
"""
|
||||
if is_reboot(expr):
|
||||
return []
|
||||
s = parse_cron(expr)
|
||||
start = after.astimezone(tz)
|
||||
runs: list[datetime] = []
|
||||
for offset in range(_HORIZON_DAYS):
|
||||
day = start.date() + timedelta(days=offset)
|
||||
if not _day_matches(s, day):
|
||||
continue
|
||||
for hour in sorted(s.hours):
|
||||
for minute in sorted(s.minutes):
|
||||
at = _local(day, hour, minute, tz)
|
||||
if at is None or at <= after:
|
||||
continue
|
||||
runs.append(at)
|
||||
if len(runs) == count:
|
||||
return runs
|
||||
return runs
|
||||
|
||||
|
||||
def _values(values: frozenset[int], low: int, high: int) -> str:
|
||||
if values == frozenset(range(low, high + 1)):
|
||||
return "*"
|
||||
return ",".join(f"{v:02d}" for v in sorted(values))
|
||||
|
||||
|
||||
def to_on_calendar(expr: str) -> list[str]:
|
||||
"""``OnCalendar`` lines that fire exactly when *expr* does.
|
||||
|
||||
:raises ValueError: for an expression :func:`parse_cron` refuses, ``@reboot``
|
||||
among them -- a timer takes that as ``OnBootSec``.
|
||||
"""
|
||||
s = parse_cron(expr)
|
||||
time = f"{_values(s.hours, 0, 23)}:{_values(s.minutes, 0, 59)}:00"
|
||||
months = _values(s.months, 1, 12)
|
||||
days = _values(s.days, 1, 31)
|
||||
weekdays = ",".join(_SYSTEMD_WEEKDAYS[d] for d in (1, 2, 3, 4, 5, 6, 0) if d in s.weekdays)
|
||||
every_weekday = s.weekdays == frozenset(range(7))
|
||||
if s.either_day:
|
||||
return [f"*-{months}-{days} {time}", f"{weekdays} *-{months}-* {time}"]
|
||||
prefix = "" if every_weekday else f"{weekdays} "
|
||||
return [f"{prefix}*-{months}-{days} {time}"]
|
||||
@@ -0,0 +1,97 @@
|
||||
"""LLDP neighbours as rows, and the two shapes NAPALM asks for.
|
||||
|
||||
A driver reads its neighbours one row per neighbour -- from ``lldpctl``
|
||||
(:func:`~napalm_device_types.lldpctl.parse_lldpctl_keyvalue`), an API or a CLI
|
||||
-- and ``get_lldp_neighbors`` and ``get_lldp_neighbors_detail`` are the same
|
||||
projection of those rows for every driver. A port with several neighbours (a
|
||||
phone and the PC behind it, an unmanaged switch) is several rows, and stays
|
||||
several entries in both shapes.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from collections.abc import Iterable
|
||||
from typing import NotRequired, TypedDict
|
||||
|
||||
from napalm.base.models import LLDPNeighborDetailDict
|
||||
|
||||
#: A chassis id of type ``mac`` that really is a MAC address.
|
||||
_MAC_RE = re.compile(r"^([0-9A-Fa-f]{2}:){5}[0-9A-Fa-f]{2}$")
|
||||
|
||||
|
||||
class LldpNeighborRow(TypedDict):
|
||||
"""One LLDP neighbour, as the local port heard it.
|
||||
|
||||
``chassis_id_type`` and ``port_id_type`` name what the id is: ``mac``,
|
||||
``ip``, ``ifname``, ``ifalias``, ``local`` or ``unhandled`` (an id type
|
||||
lldpd does not decode), ``""`` when the neighbour sent no id.
|
||||
``capabilities`` and ``enabled_capabilities`` use NAPALM's names
|
||||
(``bridge``, ``router``, ``wlan-access-point``, ``telephone``, ...).
|
||||
"""
|
||||
|
||||
local_port: str
|
||||
chassis_id: str
|
||||
chassis_id_type: str
|
||||
port_id: str
|
||||
port_id_type: str
|
||||
port_description: str
|
||||
system_name: str
|
||||
system_description: str
|
||||
management_addresses: list[str]
|
||||
capabilities: list[str]
|
||||
enabled_capabilities: list[str]
|
||||
|
||||
|
||||
class LldpNeighborDict(TypedDict):
|
||||
"""NAPALM's ``get_lldp_neighbors`` entry (``hostname``, ``port``), plus the chassis MAC.
|
||||
|
||||
``mac`` (lower case) is there only when the neighbour names its chassis by a
|
||||
MAC address; an application can match the neighbour by it, where a name
|
||||
may be ambiguous.
|
||||
"""
|
||||
|
||||
hostname: str
|
||||
port: str
|
||||
mac: NotRequired[str]
|
||||
|
||||
|
||||
def lldp_neighbors(rows: Iterable[LldpNeighborRow]) -> dict[str, list[LldpNeighborDict]]:
|
||||
"""``get_lldp_neighbors``: per local port, every neighbour's name, port and chassis MAC."""
|
||||
neighbors: dict[str, list[LldpNeighborDict]] = {}
|
||||
for row in rows:
|
||||
entry: LldpNeighborDict = {"hostname": row["system_name"], "port": row["port_id"]}
|
||||
if row["chassis_id_type"] == "mac" and _MAC_RE.match(row["chassis_id"]):
|
||||
entry["mac"] = row["chassis_id"].lower()
|
||||
neighbors.setdefault(row["local_port"], []).append(entry)
|
||||
return neighbors
|
||||
|
||||
|
||||
def lldp_neighbors_detail(
|
||||
rows: Iterable[LldpNeighborRow], interface: str = ""
|
||||
) -> dict[str, list[LLDPNeighborDetailDict]]:
|
||||
"""``get_lldp_neighbors_detail``: per local port, every neighbour in full.
|
||||
|
||||
With ``interface``, that port only. ``parent_interface`` stays empty: a row
|
||||
does not know which aggregate its port belongs to.
|
||||
"""
|
||||
details: dict[str, list[LLDPNeighborDetailDict]] = {}
|
||||
for row in rows:
|
||||
if interface and row["local_port"] != interface:
|
||||
continue
|
||||
details.setdefault(row["local_port"], []).append(
|
||||
{
|
||||
"parent_interface": "",
|
||||
"remote_port": row["port_id"],
|
||||
"remote_port_description": row["port_description"],
|
||||
"remote_chassis_id": row["chassis_id"],
|
||||
"remote_system_name": row["system_name"],
|
||||
"remote_system_description": row["system_description"],
|
||||
"remote_system_capab": list(row["capabilities"]),
|
||||
"remote_system_enable_capab": list(row["enabled_capabilities"]),
|
||||
}
|
||||
)
|
||||
return details
|
||||
|
||||
|
||||
__all__ = ["LldpNeighborDict", "LldpNeighborRow", "lldp_neighbors", "lldp_neighbors_detail"]
|
||||
@@ -0,0 +1,204 @@
|
||||
"""``lldpctl -f keyvalue``: lldpd's neighbours, one row each.
|
||||
|
||||
Every driver for a host that runs lldpd reads its neighbours the same way, so
|
||||
the command and its parse live here once and a driver only carries the command
|
||||
across. lldpd's keyvalue writer (``src/client/kv_writer.c``) prints one
|
||||
``key=value`` line per field, the key being the field's path joined by dots
|
||||
(a newline inside a value becomes a space)::
|
||||
|
||||
lldp.eth0.1.via=LLDP
|
||||
lldp.eth0.1.chassis.mac=00:11:22:33:44:01
|
||||
lldp.eth0.1.chassis.Tel.enabled=on
|
||||
lldp.eth0.1.port.local=gi1/0/7
|
||||
|
||||
**A neighbour is a block, not a port.** Every neighbour on a port prints the
|
||||
same keys, so grouping the lines by interface merges them. A neighbour opens
|
||||
with ``via`` (after the local interface's ``alias``, when it has one); the next
|
||||
``via`` on the same port is the next neighbour.
|
||||
|
||||
**An interface name may hold dots** (``eth0.1``, ``br-lan.10``). It is
|
||||
everything between ``lldp.`` and the neighbour's first field, never the second
|
||||
dotted part of the key.
|
||||
|
||||
**An id is printed under its type**, ``chassis.<type>=`` and ``port.<type>=``,
|
||||
with lldpd's names for the type (``mac``, ``ip``, ``ifname``, ``ifalias``,
|
||||
``local``, ``unhandled``); there is no ``chassis.id`` key.
|
||||
|
||||
**A failed read raises.** lldpctl exits non-zero when it cannot reach lldpd;
|
||||
:data:`LLDPCTL_KEYVALUE_COMMAND` brings that status and lldpctl's message back,
|
||||
and the parse raises on it instead of reading it as no neighbours. Output that
|
||||
carries no exit status (a driver that runs plain ``lldpctl -f keyvalue``) raises
|
||||
when it holds text but not a single neighbour line, as a shell's ``lldpctl: not
|
||||
found`` does.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
|
||||
from napalm_device_types.lldp import LldpNeighborRow
|
||||
|
||||
_RC = "__LLDPCTL_RC"
|
||||
|
||||
#: One line, POSIX ``sh``, read-only. Without arguments lldpctl shows the
|
||||
#: neighbours in detail; its error text and exit status come back with them.
|
||||
LLDPCTL_KEYVALUE_COMMAND = f"lldpctl -f keyvalue 2>&1; echo {_RC}=$?"
|
||||
|
||||
_RC_RE = re.compile(rf"^{_RC}=(\d+)$")
|
||||
_PREFIX = "lldp."
|
||||
#: The fields a neighbour opens with: the local interface's alias, then ``via``.
|
||||
_OPENERS = frozenset({"alias", "via"})
|
||||
#: The first part after the interface name of every key lldpd prints for a
|
||||
#: neighbour (``src/client/display.c``, ``display_interface``).
|
||||
_CATEGORIES = _OPENERS | {
|
||||
"rid",
|
||||
"age",
|
||||
"status",
|
||||
"chassis",
|
||||
"port",
|
||||
"ttl",
|
||||
"vlan",
|
||||
"ppvid",
|
||||
"pi",
|
||||
"lldp-med",
|
||||
"unknown-tlvs",
|
||||
}
|
||||
#: lldpd's id types (``src/lib/atoms/chassis.c``, ``port.c``), in the order read.
|
||||
_CHASSIS_ID_TYPES = ("mac", "local", "ip", "ifname", "ifalias", "unhandled")
|
||||
_PORT_ID_TYPES = ("ifname", "mac", "local", "ip", "ifalias", "unhandled")
|
||||
#: lldpd's capability symbols in NAPALM's names; one NAPALM lacks is lowercased.
|
||||
_CAPABILITIES = {
|
||||
"Other": "other",
|
||||
"Repeater": "repeater",
|
||||
"Bridge": "bridge",
|
||||
"Router": "router",
|
||||
"Wlan": "wlan-access-point",
|
||||
"Tel": "telephone",
|
||||
"Docsis": "docsis-cable-device",
|
||||
"Station": "station",
|
||||
}
|
||||
_SWITCH = {"on": True, "off": False}
|
||||
|
||||
_Fields = dict[str, list[str]]
|
||||
|
||||
|
||||
def parse_lldpctl_keyvalue(output: str) -> list[LldpNeighborRow]:
|
||||
"""One row per neighbour, in the order lldpctl listed them.
|
||||
|
||||
:raises ValueError: when lldpctl failed -- a non-zero exit status, or text
|
||||
without a single neighbour line and no status at all.
|
||||
"""
|
||||
return [_row(port, fields) for port, fields in _neighbours(_pairs(output))]
|
||||
|
||||
|
||||
def _pairs(output: str) -> list[tuple[str, str]]:
|
||||
"""``(key without "lldp.", value)`` of every neighbour line; raises on a failed run."""
|
||||
pairs: list[tuple[str, str]] = []
|
||||
other: list[str] = []
|
||||
status: int | None = None
|
||||
for line in output.splitlines():
|
||||
line = line.strip()
|
||||
key, sep, value = line.partition("=")
|
||||
if _RC_RE.match(line):
|
||||
status = int(value)
|
||||
elif sep and key.startswith(_PREFIX):
|
||||
pairs.append((key[len(_PREFIX) :], value))
|
||||
elif line:
|
||||
other.append(line)
|
||||
_raise_on_failure(status, other, bool(pairs))
|
||||
return pairs
|
||||
|
||||
|
||||
def _raise_on_failure(status: int | None, other: list[str], listed: bool) -> None:
|
||||
"""A non-zero status, or text without a status and without a neighbour line."""
|
||||
if status or (status is None and other and not listed):
|
||||
said = " ".join(other)[:200] or f"exit status {status}"
|
||||
raise ValueError(f"lldpctl did not list the neighbours: {said}")
|
||||
|
||||
|
||||
def _neighbours(pairs: list[tuple[str, str]]) -> list[tuple[str, _Fields]]:
|
||||
"""Split the lines into neighbours: ``(local interface, {field: [values]})``."""
|
||||
neighbours: list[tuple[str, _Fields]] = []
|
||||
port: str | None = None
|
||||
fields: _Fields = {}
|
||||
for key, value in pairs:
|
||||
split = _split(key, port)
|
||||
if split is None:
|
||||
continue
|
||||
name, field = split
|
||||
if name != port or field == "alias" or (field == "via" and "via" in fields):
|
||||
port, fields = name, {}
|
||||
neighbours.append((port, fields))
|
||||
fields.setdefault(field, []).append(value)
|
||||
return neighbours
|
||||
|
||||
|
||||
def _split(key: str, port: str | None) -> tuple[str, str] | None:
|
||||
"""``(interface, field)`` of a key, or None when it is no neighbour field.
|
||||
|
||||
``via`` and ``alias`` end their key, so they name the interface exactly;
|
||||
the keys after them continue that interface. Only a key that does neither
|
||||
ends its interface name at the first known field.
|
||||
"""
|
||||
head, _, last = key.rpartition(".")
|
||||
if head and last in _OPENERS:
|
||||
return head, last
|
||||
if port is not None and key.startswith(port + "."):
|
||||
field = key[len(port) + 1 :]
|
||||
if field.split(".", 1)[0] in _CATEGORIES:
|
||||
return port, field
|
||||
parts = key.split(".")
|
||||
for i in range(1, len(parts)):
|
||||
if parts[i] in _CATEGORIES:
|
||||
return ".".join(parts[:i]), ".".join(parts[i:])
|
||||
return None
|
||||
|
||||
|
||||
def _row(port: str, fields: _Fields) -> LldpNeighborRow:
|
||||
chassis_type, chassis_id = _id(fields, "chassis", _CHASSIS_ID_TYPES)
|
||||
port_type, port_id = _id(fields, "port", _PORT_ID_TYPES)
|
||||
capabilities, enabled = _capabilities(fields)
|
||||
return {
|
||||
"local_port": port,
|
||||
"chassis_id": chassis_id,
|
||||
"chassis_id_type": chassis_type,
|
||||
"port_id": port_id,
|
||||
"port_id_type": port_type,
|
||||
"port_description": _first(fields, "port.descr"),
|
||||
"system_name": _first(fields, "chassis.name"),
|
||||
"system_description": _first(fields, "chassis.descr"),
|
||||
"management_addresses": list(fields.get("chassis.mgmt-ip", [])),
|
||||
"capabilities": capabilities,
|
||||
"enabled_capabilities": enabled,
|
||||
}
|
||||
|
||||
|
||||
def _first(fields: _Fields, field: str) -> str:
|
||||
return fields.get(field, [""])[0]
|
||||
|
||||
|
||||
def _id(fields: _Fields, category: str, types: tuple[str, ...]) -> tuple[str, str]:
|
||||
"""``(type, id)`` of the chassis or the port; ``("", "")`` when none was sent."""
|
||||
for kind in types:
|
||||
if f"{category}.{kind}" in fields:
|
||||
return kind, _first(fields, f"{category}.{kind}")
|
||||
return "", ""
|
||||
|
||||
|
||||
def _capabilities(fields: _Fields) -> tuple[list[str], list[str]]:
|
||||
"""``chassis.<Symbol>.enabled=on|off``: the capabilities offered, and those enabled."""
|
||||
offered: list[str] = []
|
||||
enabled: list[str] = []
|
||||
for field, values in fields.items():
|
||||
parts = field.split(".")
|
||||
switch = _SWITCH.get(values[0])
|
||||
if len(parts) != 3 or parts[0] != "chassis" or parts[2] != "enabled" or switch is None:
|
||||
continue
|
||||
name = _CAPABILITIES.get(parts[1], parts[1].lower())
|
||||
offered.append(name)
|
||||
if switch:
|
||||
enabled.append(name)
|
||||
return offered, enabled
|
||||
|
||||
|
||||
__all__ = ["LLDPCTL_KEYVALUE_COMMAND", "parse_lldpctl_keyvalue"]
|
||||
@@ -5,6 +5,14 @@ with their public key and, where they are to administer the host, may use
|
||||
sudo with their own password. When they leave, the account is locked, not
|
||||
deleted: their key goes, sudo goes, the login is refused, and the home stays.
|
||||
|
||||
**What the person changes follows them** (NetOrk/netork#2013). A key they
|
||||
replace goes from the account as the new one comes in (``replaces``); a key
|
||||
they delete goes without a new one (``remove_local_account_keys``). Either way
|
||||
only the keys named go: one the person added on the host stays, and the
|
||||
account stays open. A password they remove goes from the account too
|
||||
(``clear_password``): it logs in with a key only and has no sudo, as the sudo
|
||||
rule asks for that password.
|
||||
|
||||
:class:`LocalAccountsMixin` holds what is the same on every POSIX host -- the
|
||||
checks, reading an account back, the order of the steps, the key file and the
|
||||
sudo rule. A driver supplies the commands that differ between operating
|
||||
@@ -48,7 +56,7 @@ from typing import TYPE_CHECKING, ClassVar
|
||||
|
||||
from typing_extensions import TypedDict
|
||||
|
||||
from napalm_device_types.fixes import run_fix_steps
|
||||
from napalm_device_types.fixes import FixStep, run_fix_steps
|
||||
from napalm_device_types.models import FixResultDict, FixStepDict, FixStepOutcomeDict
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - for type checkers only
|
||||
@@ -179,6 +187,18 @@ def check_public_key(key: str) -> None:
|
||||
raise ValueError("not a public key: expected '<type> <base64> [comment]' on one line")
|
||||
|
||||
|
||||
def _checked_keys(keys: Sequence[str]) -> list[str]:
|
||||
"""*keys*, each one stripped.
|
||||
|
||||
:raises ValueError: when *keys* is one string, or one of them no public key.
|
||||
"""
|
||||
if isinstance(keys, str):
|
||||
raise ValueError("expected a list of public keys, not one string")
|
||||
for key in keys:
|
||||
check_public_key(key)
|
||||
return [key.strip() for key in keys]
|
||||
|
||||
|
||||
def _command(argv: Sequence[str]) -> str:
|
||||
return shlex.join(argv)
|
||||
|
||||
@@ -225,9 +245,11 @@ class LocalAccountsMixin:
|
||||
|
||||
# -- what a driver supplies ------------------------------------------------------
|
||||
#
|
||||
# ``_account_*`` are the driver's: the five below it must supply, the key
|
||||
# store (``_account_key_files``, ``_account_read_keys``, ``_account_add_key``,
|
||||
# ``_account_remove_keys``) it may replace. ``_accounts_*`` are this mixin's own.
|
||||
# ``_account_*`` are the driver's: the five below it must supply, the two after
|
||||
# them (``_account_create_with_uid``, ``_account_clear_password``) it may, and
|
||||
# the key store (``_account_key_files``, ``_account_read_keys``,
|
||||
# ``_account_add_key``, ``_account_drop_keys``, ``_account_remove_keys``) it may
|
||||
# replace. ``_accounts_*`` are this mixin's own.
|
||||
|
||||
def _account_create(self, name: str, password_hash: str | None) -> list[list[str]]:
|
||||
"""The commands that create *name* with a home and :attr:`ACCOUNT_SHELL`.
|
||||
@@ -265,6 +287,15 @@ class LocalAccountsMixin:
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
def _account_clear_password(self, name: str) -> list[list[str]]:
|
||||
"""The commands that take *name*'s password away; its key login stays as it is.
|
||||
|
||||
Not an empty password -- PAM's ``nullok`` lets anyone in without one --
|
||||
and not a lock or an expiry, which sshd or PAM apply to a key too.
|
||||
Only ``add_local_account(clear_password=True)`` asks for it.
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
# -- reading ---------------------------------------------------------------------
|
||||
|
||||
def get_local_accounts(self, names: Iterable[str]) -> list[LocalAccountDict]:
|
||||
@@ -339,6 +370,8 @@ class LocalAccountsMixin:
|
||||
*,
|
||||
password_hash: str | None = None,
|
||||
admin: bool = False,
|
||||
clear_password: bool = False,
|
||||
replaces: Sequence[str] = (),
|
||||
) -> LocalAccountChangeDict:
|
||||
"""Give *name* an account that logs in with *public_key*, or open it again.
|
||||
|
||||
@@ -347,28 +380,71 @@ class LocalAccountsMixin:
|
||||
:attr:`ACCOUNT_PASSWORD_SCHEME`) becomes its password; *admin* gives it
|
||||
the sudo rule, which asks for that password -- so an administrator
|
||||
needs one. Without *admin*, the rule and :attr:`ACCOUNT_ADMIN_GROUPS`
|
||||
are taken away.
|
||||
are taken away. Without *password_hash* the account keeps the password
|
||||
it has, unless *clear_password* takes it away: the account then logs
|
||||
in with a key only, and can have no sudo.
|
||||
|
||||
Steps: ``account``, then ``sudo`` and ``keys``, each of which runs when
|
||||
the other failed.
|
||||
*replaces* are keys *public_key* takes the place of -- the person's
|
||||
old ones. They go from the account, matched by their base64 whatever
|
||||
a line's options or comment; every other key stays.
|
||||
|
||||
:raises ValueError: for a name, key or hash no host takes, the login
|
||||
user's own account, or a system account.
|
||||
Steps: ``account``, then ``sudo``, ``password`` (with
|
||||
*clear_password*), ``keys`` and ``old keys`` (with *replaces*), each of
|
||||
which runs when another failed: an old key goes even when the new one
|
||||
could not be added.
|
||||
|
||||
:raises ValueError: for a name, key or hash no host takes, a replaced
|
||||
key that is none, *clear_password* with a *password_hash* or with
|
||||
*admin*, the login user's own account, or a system account.
|
||||
"""
|
||||
check_account_name(name)
|
||||
check_public_key(public_key)
|
||||
if password_hash is not None:
|
||||
self._accounts_check_hash(password_hash)
|
||||
new = _key_blob(public_key)
|
||||
old = [key for key in _checked_keys(replaces) if _key_blob(key) != new]
|
||||
self._accounts_check_password(password_hash, admin=admin, clear=clear_password)
|
||||
existing = self._accounts_changeable(name)
|
||||
first = run_fix_steps(
|
||||
[("account", lambda: self._accounts_open_or_create(name, existing, password_hash))]
|
||||
)
|
||||
parts = [first]
|
||||
if first["success"]:
|
||||
parts += [
|
||||
run_fix_steps([("sudo", lambda: self._accounts_set_sudo(name, admin=admin))]),
|
||||
run_fix_steps([("keys", lambda: self._accounts_add_key(name, public_key))]),
|
||||
]
|
||||
steps = self._accounts_after_open(name, public_key, admin, clear_password, old)
|
||||
parts += [run_fix_steps([step]) for step in steps]
|
||||
return self._accounts_result(name, parts)
|
||||
|
||||
def _accounts_after_open(
|
||||
self, name: str, public_key: str, admin: bool, clear_password: bool, old: list[str]
|
||||
) -> list[FixStep]:
|
||||
"""The steps of ``add_local_account`` once the account is there, each on its own."""
|
||||
steps: list[FixStep] = [("sudo", lambda: self._accounts_set_sudo(name, admin=admin))]
|
||||
if clear_password:
|
||||
steps.append(("password", lambda: self._accounts_clear_password(name)))
|
||||
steps.append(("keys", lambda: self._accounts_add_key(name, public_key)))
|
||||
if old:
|
||||
steps.append(("old keys", lambda: self._accounts_drop_keys(name, old)))
|
||||
return steps
|
||||
|
||||
def remove_local_account_keys(self, name: str, keys: Sequence[str]) -> LocalAccountChangeDict:
|
||||
"""Take *keys* off *name*'s account, for a person who deleted them.
|
||||
|
||||
Only those keys go, matched by their base64 whatever a line's options
|
||||
or comment; the other keys, the password and sudo stay, and the
|
||||
account stays open -- to get a new key once there is one. A locked
|
||||
account has none left to remove: the lock took them all.
|
||||
|
||||
Step: ``keys``.
|
||||
|
||||
:raises ValueError: for a name no host takes, no keys or one that is
|
||||
none, the login user's own account, or a system account.
|
||||
"""
|
||||
check_account_name(name)
|
||||
wanted = _checked_keys(keys)
|
||||
if not wanted:
|
||||
raise ValueError("no public key to remove")
|
||||
if self._accounts_changeable(name) is None:
|
||||
step: FixStepDict = {"step": "keys", "ok": False, "output": f"no account {name}"}
|
||||
return {**_joined([step]), "account": None}
|
||||
parts = [run_fix_steps([("keys", lambda: self._accounts_drop_keys(name, wanted))])]
|
||||
return self._accounts_result(name, parts)
|
||||
|
||||
def lock_local_account(self, name: str) -> LocalAccountChangeDict:
|
||||
@@ -490,6 +566,20 @@ class LocalAccountsMixin:
|
||||
raise RuntimeError(f"cannot ask who has uid {uid}: {_said(result)}")
|
||||
return result.stdout.split(":", 1)[0].strip() or None
|
||||
|
||||
def _accounts_check_password(
|
||||
self, password_hash: str | None, *, admin: bool, clear: bool
|
||||
) -> None:
|
||||
""":raises ValueError: for a hash the host does not take, or a password to
|
||||
clear that is to be set, or that sudo would ask for."""
|
||||
if clear and password_hash is not None:
|
||||
raise ValueError(
|
||||
"a password to clear cannot be set too: clear_password or password_hash"
|
||||
)
|
||||
if clear and admin:
|
||||
raise ValueError("an account without a password cannot use sudo: its rule asks for it")
|
||||
if password_hash is not None:
|
||||
self._accounts_check_hash(password_hash)
|
||||
|
||||
def _accounts_check_hash(self, password_hash: str) -> None:
|
||||
prefixes = PASSWORD_SCHEMES[self.ACCOUNT_PASSWORD_SCHEME]
|
||||
if not _CRYPT_RE.fullmatch(password_hash) or not password_hash.startswith(prefixes):
|
||||
@@ -550,6 +640,16 @@ class LocalAccountsMixin:
|
||||
return {"ok": False, "output": _said(failed)}
|
||||
return {"ok": True, "output": done + (", password set" if password_hash else "")}
|
||||
|
||||
def _accounts_clear_password(self, name: str) -> FixStepOutcomeDict:
|
||||
try:
|
||||
commands = self._account_clear_password(name)
|
||||
except NotImplementedError:
|
||||
return {"ok": False, "output": "this driver cannot remove a password"}
|
||||
failed = self._accounts_run(commands)
|
||||
if failed is not None:
|
||||
return {"ok": False, "output": _said(failed)}
|
||||
return {"ok": True, "output": f"{name} has no password: it logs in with a key only"}
|
||||
|
||||
def _accounts_lock(self, name: str) -> FixStepOutcomeDict:
|
||||
passwd = self._accounts_passwd(name)
|
||||
if passwd is None:
|
||||
@@ -683,9 +783,23 @@ class LocalAccountsMixin:
|
||||
mine = self._accounts_key_file(name, path)
|
||||
if mine is None:
|
||||
return {"ok": False, "output": f"cannot read {path}"}
|
||||
staged = self._accounts_stage("".join(f"{line}\n" for line in [*mine, public_key.strip()]))
|
||||
failed = self._accounts_write_key_file(name, home, path, [*mine, public_key.strip()])
|
||||
if failed is not None:
|
||||
return {"ok": False, "output": f"{path}: {_said(failed)}"}
|
||||
return {"ok": True, "output": f"key added to {path}"}
|
||||
|
||||
def _accounts_write_key_file(
|
||||
self, name: str, home: str, path: str, lines: list[str]
|
||||
) -> CommandResult | None:
|
||||
"""*lines* as *name*'s key file *path*, written as *name*; None, or what failed.
|
||||
|
||||
Without lines the file goes: nothing is staged from an empty stdin.
|
||||
"""
|
||||
if not lines:
|
||||
return self._accounts_run_as(name, [["rm", "-f", "--", path]])
|
||||
staged = self._accounts_stage("".join(f"{line}\n" for line in lines))
|
||||
try:
|
||||
failed = self._accounts_run_as(
|
||||
return self._accounts_run_as(
|
||||
name,
|
||||
[
|
||||
["mkdir", "-p", "-m", "700", "--", f"{home}/.ssh"],
|
||||
@@ -694,9 +808,49 @@ class LocalAccountsMixin:
|
||||
)
|
||||
finally:
|
||||
self._accounts_unstage(staged)
|
||||
|
||||
def _accounts_drop_from(
|
||||
self, name: str, home: str, path: str, blobs: set[str | None]
|
||||
) -> FixStepOutcomeDict:
|
||||
"""The keys of *blobs* out of *name*'s key file *path*; no output when none was in it."""
|
||||
lines = self._accounts_key_file(name, path)
|
||||
if lines is None:
|
||||
return {"ok": False, "output": f"cannot read {path}"}
|
||||
kept = [line for line in lines if _key_blob(line) not in blobs]
|
||||
if len(kept) == len(lines):
|
||||
return {"ok": True, "output": ""}
|
||||
failed = self._accounts_write_key_file(name, home, path, kept)
|
||||
if failed is not None:
|
||||
return {"ok": False, "output": f"{path}: {_said(failed)}"}
|
||||
return {"ok": True, "output": f"key added to {path}"}
|
||||
return {"ok": True, "output": f"{len(lines) - len(kept)} removed from {path}"}
|
||||
|
||||
def _accounts_drop_keys(self, name: str, keys: list[str]) -> FixStepOutcomeDict:
|
||||
home = self._accounts_home(name)
|
||||
if home is None:
|
||||
return {"ok": False, "output": f"no account {name}"}
|
||||
return self._account_drop_keys(name, home, keys)
|
||||
|
||||
def _account_drop_keys(self, name: str, home: str, keys: list[str]) -> FixStepOutcomeDict:
|
||||
"""Remove exactly *keys* from *name*'s key files in *home*, as *name*.
|
||||
|
||||
A key is matched by its base64, whatever a line's options or comment;
|
||||
every other line stays. Afterwards none of *keys* may be left in
|
||||
:meth:`_account_key_files`. A driver whose host keeps keys elsewhere
|
||||
overrides this, with :meth:`_account_add_key`.
|
||||
"""
|
||||
blobs = {_key_blob(key) for key in keys}
|
||||
said: list[str] = []
|
||||
for path in authorized_keys_paths(home):
|
||||
outcome = self._accounts_drop_from(name, home, path, blobs)
|
||||
if not outcome["ok"]:
|
||||
return outcome
|
||||
said += [outcome["output"]] if outcome["output"] else []
|
||||
left = self._account_read_keys(name, home)
|
||||
if left is None:
|
||||
return {"ok": False, "output": f"cannot read the key files of {name}"}
|
||||
if any(_key_blob(line) in blobs for line in left):
|
||||
return {"ok": False, "output": f"{name} still has a key that was to go"}
|
||||
return {"ok": True, "output": "\n".join(said) or f"none of the keys is in {home}/.ssh"}
|
||||
|
||||
def _accounts_key_file(self, name: str, path: str) -> list[str] | None:
|
||||
there = self.run_command(_command(["test", "-e", path]), privileged=True, timeout=30)
|
||||
|
||||
@@ -1038,15 +1038,6 @@ class ProcessDict(TypedDict):
|
||||
command: str # full command line
|
||||
|
||||
|
||||
class CronJobDict(TypedDict):
|
||||
"""A scheduled cron task."""
|
||||
|
||||
user: str # owner of the crontab entry
|
||||
schedule: str # five-field cron expression, e.g. "0 * * * *"
|
||||
command: str # shell command to execute
|
||||
description: NotRequired[str] # optional inline comment
|
||||
|
||||
|
||||
class ApplyUpdatesResultDict(TypedDict):
|
||||
"""Result of an ``apply_updates()`` call."""
|
||||
|
||||
|
||||
@@ -15,7 +15,6 @@ from typing import TYPE_CHECKING
|
||||
from napalm_device_types.base import DeviceTypeDriver
|
||||
from napalm_device_types.health_metrics import HealthMetricsMixin
|
||||
from napalm_device_types.models import (
|
||||
CronJobDict,
|
||||
DeviceActionResultDict,
|
||||
ProcessDict,
|
||||
SNMPConfigDict,
|
||||
@@ -131,34 +130,6 @@ class OSDriver(
|
||||
"""
|
||||
...
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Cron jobs
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def get_cron_jobs(self) -> list[CronJobDict]:
|
||||
"""
|
||||
Returns scheduled cron tasks from all user crontabs and ``/etc/cron.d``.
|
||||
|
||||
Each entry contains:
|
||||
|
||||
* user (string) - owner of the crontab entry
|
||||
* schedule (string) - five-field cron expression (e.g. ``"0 * * * *"``)
|
||||
* command (string) - shell command to execute
|
||||
* description (string, optional) - inline comment text if present
|
||||
|
||||
Example::
|
||||
|
||||
[
|
||||
{
|
||||
"user": "root",
|
||||
"schedule": "0 4 * * *",
|
||||
"command": "/usr/local/bin/backup.sh",
|
||||
"description": "nightly backup",
|
||||
},
|
||||
]
|
||||
"""
|
||||
...
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# SNMP
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
@@ -28,9 +28,12 @@ APT_UPGRADABLE_COMMAND = "{ LC_ALL=C apt list --upgradable 2>/dev/null; echo __A
|
||||
|
||||
#: Read-only. Lists the packages that a pending security advisory covers. A
|
||||
#: failed dnf prints nothing, the same as no advisories, so its exit status is
|
||||
#: printed inside the group, as apt's is.
|
||||
#: printed inside the group, as apt's is. It reads the cached metadata only, as
|
||||
#: apt reads its local index: an expired cache otherwise makes dnf download the
|
||||
#: repositories' metadata first, which can take minutes (#76).
|
||||
DNF_SECURITY_COMMAND = (
|
||||
"{ LC_ALL=C dnf updateinfo list --security --quiet 2>/dev/null; echo __DNF_RC=$?; } | cat"
|
||||
"{ LC_ALL=C dnf --cacheonly updateinfo list --security --quiet 2>/dev/null;"
|
||||
" echo __DNF_RC=$?; } | cat"
|
||||
)
|
||||
|
||||
# openssl/noble-updates,noble-security 3.0.13-0ubuntu3.6 amd64 [upgradable from: 3.0.13-0ubuntu3.5]
|
||||
|
||||
@@ -0,0 +1,196 @@
|
||||
"""Scheduled jobs: what a host runs on its own schedule, and switching it on, off and now.
|
||||
|
||||
A host runs work on its own: cron lines, run-parts scripts, anacron, systemd
|
||||
timers, FreeBSD's ``periodic``, a firewall's or a NAS's own scheduled jobs, the
|
||||
Windows Task Scheduler. :class:`ScheduledJobsMixin` reads all of them as one
|
||||
shape, :class:`~napalm_device_types.models.ScheduledJobDict`, and acts on them.
|
||||
Declared under ``if TYPE_CHECKING`` like a role: nothing exists at runtime until
|
||||
a driver implements it. :class:`~napalm_device_types.scheduled_jobs_posix.PosixScheduledJobsMixin`
|
||||
is the implementation for a host reached through the command channel.
|
||||
|
||||
**Who owns a definition** is on every job (``managed_by``): ``package`` (installed
|
||||
by one), ``admin`` (written on the host), ``platform`` (generated by the device's
|
||||
own configuration -- OPNsense, OpenMediaVault, QTS, Windows), ``application`` (written
|
||||
through :meth:`put_scheduled_job`, under the ``name`` the application gave it).
|
||||
What the application makes of that is the application's.
|
||||
|
||||
**What may be done is on every job too** (``actions``): every job a driver can
|
||||
switch takes ``enable``, ``disable`` and ``run``; only an ``application`` job takes
|
||||
``edit`` and ``delete``. A driver never edits or removes a job it did not write.
|
||||
|
||||
**Never over a change it did not see.** Every write names the ``digest`` the caller
|
||||
saw. The driver reads the job again right before it writes and raises
|
||||
:class:`ScheduledJobChanged` when the definition differs.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import re
|
||||
from typing import TYPE_CHECKING, Any, NotRequired, TypedDict
|
||||
|
||||
from napalm_device_types.cron import is_reboot, parse_cron
|
||||
|
||||
#: What :meth:`ScheduledJobsMixin.manage_scheduled_job` takes.
|
||||
ACTIONS = ("enable", "disable", "run")
|
||||
|
||||
#: What a job's ``actions`` may list beyond :data:`ACTIONS`.
|
||||
EDIT = "edit"
|
||||
DELETE = "delete"
|
||||
|
||||
KINDS = ("cron", "run_parts", "anacron", "systemd_timer", "periodic", "task")
|
||||
MANAGED_BY = ("package", "admin", "platform", "application")
|
||||
|
||||
#: Seconds a run now may take on the host before ``timeout`` stops it.
|
||||
RUN_TIMEOUT = 300
|
||||
|
||||
#: A job's name becomes a file and a unit name: lower case, digits, dashes.
|
||||
_NAME = re.compile(r"[a-z0-9][a-z0-9-]{0,62}")
|
||||
#: A user name as useradd and the BSDs take it.
|
||||
_USER = re.compile(r"[A-Za-z_][A-Za-z0-9_.-]{0,31}\$?")
|
||||
|
||||
#: The parts of a job that are its definition; what the host says of its runs is not.
|
||||
_DEFINITION = ("kind", "source", "run_as", "schedule", "command", "enabled")
|
||||
|
||||
|
||||
class ScheduledJobDict(TypedDict):
|
||||
"""A job the host runs on its own schedule (``get_scheduled_jobs``).
|
||||
|
||||
A cron line, a run-parts script, an anacron job, a systemd timer, a FreeBSD
|
||||
``periodic`` script, a platform's own job, a Windows task -- one shape for all
|
||||
of them (``napalm_device_types.scheduled_jobs``).
|
||||
"""
|
||||
|
||||
key: str # stable name on this host: "timer:apt-daily.timer", "cron:/etc/crontab:1a2b3c4d"
|
||||
kind: str # cron | run_parts | anacron | systemd_timer | periodic | task
|
||||
source: str # the file, unit file or configuration it is defined in
|
||||
run_as: str # the user it runs as
|
||||
schedule: str # as the host has it: "*/5 * * * *", "OnCalendar=daily", "daily"
|
||||
schedule_kind: str # cron | calendar | monotonic | period | triggers
|
||||
command: str # as the host has it
|
||||
enabled: bool
|
||||
managed_by: str # package | admin | platform | application
|
||||
digest: str # job_digest(): the definition as it is now
|
||||
actions: list[str] # what the driver can do to this job: enable, disable, run, edit, delete
|
||||
name: NotRequired[str] # the name an application gave a job it wrote (put_scheduled_job)
|
||||
description: NotRequired[str]
|
||||
last_run: NotRequired[str | None] # ISO 8601, UTC; where the host knows it
|
||||
last_result: NotRequired[str | None] # "success", or what the host says went wrong
|
||||
next_run: NotRequired[str | None] # ISO 8601, UTC; where the host knows it
|
||||
|
||||
|
||||
class ScheduledJobsReadingDict(TypedDict):
|
||||
"""Every scheduled job of a host, and the time zone its schedules are in."""
|
||||
|
||||
jobs: list[ScheduledJobDict]
|
||||
time_zone: str | None # IANA name ("Europe/Berlin"); None when the host does not say
|
||||
|
||||
|
||||
class ScheduledJobSpecDict(TypedDict):
|
||||
"""A job an application wants on the host (``put_scheduled_job``)."""
|
||||
|
||||
name: str # the job's name on the host, chosen by the application: [a-z0-9][a-z0-9-]*
|
||||
schedule: str # a cron expression (napalm_device_types.cron), "@reboot" included
|
||||
run_as: str
|
||||
command: str # one or more lines of POSIX sh; a platform may take its own (OPNsense: configd)
|
||||
enabled: bool
|
||||
description: NotRequired[str]
|
||||
|
||||
|
||||
class ScheduledJobResultDict(TypedDict):
|
||||
"""What an action on a scheduled job did."""
|
||||
|
||||
success: bool
|
||||
output: str # what the host said; for a run, what the job printed
|
||||
|
||||
|
||||
class ScheduledJobChanged(Exception): # noqa: N818 - a state, as the callers word it
|
||||
"""The job's definition differs from the digest the caller saw: nothing was written."""
|
||||
|
||||
|
||||
class ScheduledJobRefused(ValueError):
|
||||
"""The job cannot take what was asked of it, or the request itself is not valid."""
|
||||
|
||||
|
||||
def job_digest(job: dict[str, Any], *, content: str = "") -> str:
|
||||
"""The definition of *job* in 16 hex characters.
|
||||
|
||||
Over kind, source, run_as, schedule, command and enabled -- so enabling a job
|
||||
changes it -- and *content*: what the definition points at that the job's
|
||||
fields do not show, such as a run-parts script's checksum.
|
||||
"""
|
||||
parts = [job.get(field) for field in _DEFINITION] + [content]
|
||||
return hashlib.sha256(json.dumps(parts).encode()).hexdigest()[:16]
|
||||
|
||||
|
||||
def check_job_name(name: str) -> None:
|
||||
""":raises ScheduledJobRefused: for a name a file or a unit could not carry safely."""
|
||||
if not _NAME.fullmatch(name or ""):
|
||||
raise ScheduledJobRefused(f"not a job name: {name!r} (a-z, 0-9 and -, at most 63)")
|
||||
|
||||
|
||||
def check_spec(spec: ScheduledJobSpecDict) -> None:
|
||||
""":raises ScheduledJobRefused: for a spec no host could be given as it is."""
|
||||
check_job_name(spec["name"])
|
||||
if not _USER.fullmatch(spec.get("run_as") or ""):
|
||||
raise ScheduledJobRefused(f"not a user name: {spec.get('run_as')!r}")
|
||||
if not (spec.get("command") or "").strip():
|
||||
raise ScheduledJobRefused("a job needs a command")
|
||||
if "\n" in (spec.get("description") or ""):
|
||||
raise ScheduledJobRefused("a description is one line")
|
||||
if not is_reboot(spec["schedule"]):
|
||||
try:
|
||||
parse_cron(spec["schedule"])
|
||||
except ValueError as exc:
|
||||
raise ScheduledJobRefused(str(exc)) from exc
|
||||
|
||||
|
||||
class ScheduledJobsMixin:
|
||||
"""Declares the scheduled-jobs contract; a driver that can implements it.
|
||||
|
||||
A verb a driver cannot do at all is ``None`` on the driver, as ``manage_service``
|
||||
is; a verb one job cannot take is missing from that job's ``actions``.
|
||||
"""
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def get_scheduled_jobs(self) -> ScheduledJobsReadingDict:
|
||||
"""Every scheduled job of the host.
|
||||
|
||||
:raises ValueError: when the reading came back cut short -- never a
|
||||
shorter list.
|
||||
"""
|
||||
...
|
||||
|
||||
def manage_scheduled_job(
|
||||
self, key: str, action: str, *, digest: str
|
||||
) -> ScheduledJobResultDict:
|
||||
"""Enable, disable or run the job *key* now.
|
||||
|
||||
:raises ScheduledJobChanged: when the job is no longer as *digest* says.
|
||||
:raises ScheduledJobRefused: when the job does not take *action*.
|
||||
:raises LookupError: when the host has no job *key*.
|
||||
"""
|
||||
...
|
||||
|
||||
def put_scheduled_job(
|
||||
self, spec: ScheduledJobSpecDict, *, digest: str | None = None
|
||||
) -> ScheduledJobResultDict:
|
||||
"""Create the job ``spec["name"]`` (*digest* None) or change it.
|
||||
|
||||
:raises ScheduledJobChanged: when the job is no longer as *digest*
|
||||
says, or when a job of that name is there although *digest* is None.
|
||||
:raises ScheduledJobRefused: for a spec :func:`check_spec` refuses, or
|
||||
one the host cannot take.
|
||||
"""
|
||||
...
|
||||
|
||||
def delete_scheduled_job(self, key: str, *, digest: str) -> ScheduledJobResultDict:
|
||||
"""Delete the job *key*, which an application wrote.
|
||||
|
||||
:raises ScheduledJobChanged: when the job is no longer as *digest* says.
|
||||
:raises ScheduledJobRefused: when the job is not one an application wrote.
|
||||
:raises LookupError: when the host has no job *key*.
|
||||
"""
|
||||
...
|
||||
@@ -0,0 +1,493 @@
|
||||
"""Scheduled jobs on a POSIX host, over the command channel: cron and systemd timers.
|
||||
|
||||
:class:`PosixScheduledJobsMixin` implements
|
||||
:class:`~napalm_device_types.scheduled_jobs.ScheduledJobsMixin` for a driver that has
|
||||
:meth:`~napalm_device_types.channel.CommandChannelMixin.run_command`. The driver
|
||||
mixes it in and sets the paths its host keeps jobs in; the defaults are Debian's.
|
||||
|
||||
**Reading** is one framed report as root (:mod:`napalm_device_types._scheduled_reading`).
|
||||
|
||||
**Switching** a cron or anacron line comments it out behind ``#napalm-disabled: ``
|
||||
and takes exactly that marker away again; a run-parts script loses or gets its
|
||||
execute bit; a timer is ``systemctl disable --now`` / ``enable --now``. A timer that
|
||||
cannot be switched (``static``) only runs.
|
||||
|
||||
**Writing** stages the new text as the login user and installs it as root with one
|
||||
fixed command: ``crontab -u <user> <file>`` for a user's crontab, ``install`` with
|
||||
the file's own mode and owner for a system file. Root never reads stdin. Before any
|
||||
write the job is read again and compared with the digest the caller saw.
|
||||
|
||||
**An application's own job** (:meth:`put_scheduled_job`) is a timer on a host with
|
||||
systemd -- ``<name>.timer`` and ``<name>.service`` in :attr:`UNIT_DIR`, the command
|
||||
in :attr:`JOB_SCRIPT_DIR` ``/<name>`` behind a marker and its cron expression --
|
||||
else a file in :attr:`OWN_CRON_D_DIR`, else marked lines in the user's crontab.
|
||||
|
||||
**Running now** is bounded by :data:`~napalm_device_types.scheduled_jobs.RUN_TIMEOUT`:
|
||||
a timer's unit by ``systemctl start`` and its journal since then, a cron line's
|
||||
command as its user the way cron runs it (``_crontab.run_script``).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import fnmatch
|
||||
import shlex
|
||||
from collections.abc import Sequence
|
||||
from typing import TYPE_CHECKING, ClassVar
|
||||
|
||||
from napalm_device_types._crontab import (
|
||||
JOB_MARKER,
|
||||
CronLine,
|
||||
marked_lines,
|
||||
parse_anacrontab,
|
||||
parse_crontab,
|
||||
run_script,
|
||||
switched,
|
||||
with_job,
|
||||
without_job,
|
||||
)
|
||||
from napalm_device_types._scheduled_reading import (
|
||||
OWNERS_SH,
|
||||
SCHEDULE_HEADER,
|
||||
TIME_ZONE_SH,
|
||||
Paths,
|
||||
Reading,
|
||||
parse_reading,
|
||||
reading_command,
|
||||
)
|
||||
from napalm_device_types.cron import is_reboot, to_on_calendar
|
||||
from napalm_device_types.scheduled_jobs import (
|
||||
ACTIONS,
|
||||
RUN_TIMEOUT,
|
||||
ScheduledJobChanged,
|
||||
ScheduledJobRefused,
|
||||
ScheduledJobResultDict,
|
||||
ScheduledJobsMixin,
|
||||
ScheduledJobSpecDict,
|
||||
ScheduledJobsReadingDict,
|
||||
check_spec,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from napalm_device_types.channel import CommandResult
|
||||
|
||||
_STAGE = 't=$(mktemp) && chmod 644 "$t" && cat > "$t" && printf %s "$t"'
|
||||
#: What a run keeps of the job's output.
|
||||
_OUTPUT_LIMIT = 64 * 1024
|
||||
_READ_TIMEOUT = 120
|
||||
|
||||
|
||||
def _command(argv: Sequence[str]) -> str:
|
||||
return shlex.join(argv)
|
||||
|
||||
|
||||
def _said(result: CommandResult) -> str:
|
||||
return "\n".join(p for p in (result.stdout.strip(), result.stderr.strip()) if p)[
|
||||
-_OUTPUT_LIMIT:
|
||||
]
|
||||
|
||||
|
||||
def _result(result: CommandResult, done: str) -> ScheduledJobResultDict:
|
||||
if result.exit_code != 0:
|
||||
return {"success": False, "output": _said(result) or f"exit {result.exit_code}"}
|
||||
return {"success": True, "output": _said(result) or done}
|
||||
|
||||
|
||||
class PosixScheduledJobsMixin(ScheduledJobsMixin):
|
||||
"""Scheduled jobs of a host reached through ``run_command``; the paths are the driver's."""
|
||||
|
||||
CRON_SYSTEM_FILES: ClassVar[tuple[str, ...]] = ("/etc/crontab",)
|
||||
CRON_D_DIRS: ClassVar[tuple[str, ...]] = ("/etc/cron.d",)
|
||||
CRON_SPOOL_DIRS: ClassVar[tuple[str, ...]] = (
|
||||
"/var/spool/cron/crontabs",
|
||||
"/var/spool/cron",
|
||||
"/etc/crontabs",
|
||||
)
|
||||
RUN_PARTS_DIRS: ClassVar[tuple[tuple[str, str], ...]] = (
|
||||
("hourly", "/etc/cron.hourly"),
|
||||
("daily", "/etc/cron.daily"),
|
||||
("weekly", "/etc/cron.weekly"),
|
||||
("monthly", "/etc/cron.monthly"),
|
||||
)
|
||||
ANACRONTAB: ClassVar[str] = "/etc/anacrontab"
|
||||
JOB_SCRIPT_DIR: ClassVar[str] = "/usr/local/libexec/scheduled-jobs"
|
||||
UNIT_DIR: ClassVar[str] = "/etc/systemd/system"
|
||||
#: Where a host without systemd takes an application's job as a file, if the directory
|
||||
#: is there; else, or None, the user's crontab.
|
||||
OWN_CRON_D_DIR: ClassVar[str | None] = "/etc/cron.d"
|
||||
ROOT_GROUP: ClassVar[str] = "root"
|
||||
#: Prints "<octal mode> <owner> <group>" of the path appended.
|
||||
STAT_ARGV: ClassVar[tuple[str, ...]] = ("stat", "-c", "%a %U %G", "--")
|
||||
#: ``install`` and how it makes missing directories (FreeBSD's ``-D`` is a destdir).
|
||||
INSTALL: ClassVar[tuple[str, ...]] = ("install", "-D")
|
||||
#: ``sh`` that reads definition paths on stdin and prints ``<package>: <path>``.
|
||||
OWNERS_SH: ClassVar[str] = OWNERS_SH
|
||||
#: ``sh`` that prints the time zone: an IANA name, or a path ending in ``zoneinfo/<name>``.
|
||||
TIME_ZONE_SH: ClassVar[str] = TIME_ZONE_SH
|
||||
#: ``fnmatch`` patterns on a job's ``source``: what the platform generates is ``platform``.
|
||||
PLATFORM_SOURCES: ClassVar[tuple[str, ...]] = ()
|
||||
#: Run as root after a user's crontab was written; for a host that keeps it elsewhere too.
|
||||
CRONTAB_AFTER_WRITE: ClassVar[tuple[str, ...] | None] = None
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def run_command(
|
||||
self,
|
||||
command: str,
|
||||
*,
|
||||
privileged: bool = False,
|
||||
timeout: float = 60,
|
||||
stdin: bytes | None = None,
|
||||
) -> CommandResult: ...
|
||||
|
||||
# -- reading ---------------------------------------------------------------------------------
|
||||
|
||||
def _sj_paths(self) -> Paths:
|
||||
return Paths(
|
||||
system_files=self.CRON_SYSTEM_FILES,
|
||||
cron_d_dirs=self.CRON_D_DIRS,
|
||||
spool_dirs=self.CRON_SPOOL_DIRS,
|
||||
run_parts=self.RUN_PARTS_DIRS,
|
||||
anacrontab=self.ANACRONTAB,
|
||||
script_dir=self.JOB_SCRIPT_DIR,
|
||||
owners_sh=self.OWNERS_SH,
|
||||
time_zone_sh=self.TIME_ZONE_SH,
|
||||
)
|
||||
|
||||
def _sj_read(self) -> Reading:
|
||||
result = self.run_command(
|
||||
reading_command(self._sj_paths()), privileged=True, timeout=_READ_TIMEOUT
|
||||
)
|
||||
reading = parse_reading(result.stdout)
|
||||
for job in reading.jobs:
|
||||
source = str(job["source"])
|
||||
if job["managed_by"] != "application" and any(
|
||||
fnmatch.fnmatchcase(source, p) for p in self.PLATFORM_SOURCES
|
||||
):
|
||||
job["managed_by"] = "platform"
|
||||
return reading
|
||||
|
||||
def get_scheduled_jobs(self) -> ScheduledJobsReadingDict:
|
||||
"""Every scheduled job of the host, in one round trip.
|
||||
|
||||
:raises ValueError: when the report came back cut short.
|
||||
"""
|
||||
reading = self._sj_read()
|
||||
return {"jobs": reading.jobs, "time_zone": reading.time_zone} # type: ignore[typeddict-item]
|
||||
|
||||
def _sj_job(self, reading: Reading, key: str, digest: str) -> dict[str, object]:
|
||||
job = next((j for j in reading.jobs if j["key"] == key), None)
|
||||
if job is None:
|
||||
raise LookupError(f"no scheduled job {key}")
|
||||
if job["digest"] != digest:
|
||||
raise ScheduledJobChanged(f"{key} changed on the host since it was read")
|
||||
return job
|
||||
|
||||
# -- commands --------------------------------------------------------------------------------
|
||||
|
||||
def _sj_root(self, argv: Sequence[str], timeout: float = 60) -> CommandResult:
|
||||
return self.run_command(_command(argv), privileged=True, timeout=timeout)
|
||||
|
||||
def _sj_checked(self, argv: Sequence[str]) -> CommandResult:
|
||||
result = self._sj_root(argv)
|
||||
if result.exit_code != 0:
|
||||
raise RuntimeError(f"{_command(argv)}: {_said(result) or result.exit_code}")
|
||||
return result
|
||||
|
||||
def _sj_stage(self, content: str) -> str:
|
||||
result = self.run_command(_STAGE, timeout=30, stdin=content.encode())
|
||||
path = result.stdout.strip().splitlines()[-1:] or [""]
|
||||
if result.exit_code != 0 or not path[0].startswith("/"):
|
||||
raise RuntimeError(f"cannot stage a file: {_said(result)}")
|
||||
return path[0]
|
||||
|
||||
def _sj_install(self, content: str, path: str, mode: str, owner: str, group: str) -> None:
|
||||
staged = self._sj_stage(content)
|
||||
try:
|
||||
self._sj_checked(
|
||||
[*self.INSTALL, "-m", mode, "-o", owner, "-g", group, "--", staged, path]
|
||||
)
|
||||
finally:
|
||||
self.run_command(_command(["rm", "-f", "--", staged]), timeout=30)
|
||||
|
||||
# -- crontab files ---------------------------------------------------------------------------
|
||||
|
||||
def _sj_spool_user(self, path: str) -> str | None:
|
||||
"""The user whose crontab *path* is; None for a system file."""
|
||||
directory, _, user = path.rpartition("/")
|
||||
return user if directory in self.CRON_SPOOL_DIRS else None
|
||||
|
||||
def _sj_crontab(self, user: str) -> str:
|
||||
result = self._sj_root(["crontab", "-u", user, "-l"])
|
||||
if result.exit_code != 0:
|
||||
if "no crontab" in (result.stderr + result.stdout).lower():
|
||||
return ""
|
||||
raise RuntimeError(f"cannot read the crontab of {user}: {_said(result)}")
|
||||
return result.stdout
|
||||
|
||||
def _sj_text(self, path: str) -> str:
|
||||
user = self._sj_spool_user(path)
|
||||
if user is not None:
|
||||
return self._sj_crontab(user)
|
||||
return self._sj_checked(["cat", "--", path]).stdout
|
||||
|
||||
def _sj_write_crontab(self, user: str, text: str) -> None:
|
||||
staged = self._sj_stage(text)
|
||||
try:
|
||||
self._sj_checked(["crontab", "-u", user, staged])
|
||||
finally:
|
||||
self.run_command(_command(["rm", "-f", "--", staged]), timeout=30)
|
||||
if self.CRONTAB_AFTER_WRITE:
|
||||
self._sj_checked(self.CRONTAB_AFTER_WRITE)
|
||||
|
||||
def _sj_write(self, path: str, text: str) -> None:
|
||||
user = self._sj_spool_user(path)
|
||||
if user is not None:
|
||||
self._sj_write_crontab(user, text)
|
||||
return
|
||||
# Through a symbolic link to the file it names, with that file's mode: the
|
||||
# link's own mode is 777, and cron ignores a file anyone may write.
|
||||
real = self._sj_checked(["readlink", "-f", "--", path]).stdout.strip() or path
|
||||
mode, owner, group = self._sj_checked([*self.STAT_ARGV, real]).stdout.split()[:3]
|
||||
self._sj_install(text, real, mode, owner, group)
|
||||
|
||||
def _sj_line(self, job: dict[str, object], text: str) -> CronLine:
|
||||
"""The job's line in *text* as it is now, read again for the write."""
|
||||
path = str(job["source"])
|
||||
if job["kind"] == "anacron":
|
||||
found = parse_anacrontab(text)
|
||||
else:
|
||||
user = self._sj_spool_user(path)
|
||||
found = parse_crontab(path, text, system=user is None, user=user or "")
|
||||
line = next((ln for ln in found if ln.key == job["key"]), None)
|
||||
if line is None:
|
||||
raise ScheduledJobChanged(f"{job['key']} is no longer in {path}")
|
||||
return line
|
||||
|
||||
# -- switching and running -------------------------------------------------------------------
|
||||
|
||||
def _sj_timer_unit(self, job: dict[str, object]) -> str:
|
||||
key = str(job["key"])
|
||||
return f"{job['name']}.timer" if key.startswith("job:") else key.removeprefix("timer:")
|
||||
|
||||
def _sj_switch(self, job: dict[str, object], enabled: bool) -> ScheduledJobResultDict:
|
||||
word = "enabled" if enabled else "disabled"
|
||||
if job["kind"] == "systemd_timer":
|
||||
verb = "enable" if enabled else "disable"
|
||||
return _result(
|
||||
self._sj_root(["systemctl", verb, "--now", "--", self._sj_timer_unit(job)]), word
|
||||
)
|
||||
if job["kind"] == "run_parts":
|
||||
return _result(
|
||||
self._sj_root(["chmod", "+x" if enabled else "-x", "--", str(job["source"])]), word
|
||||
)
|
||||
path = str(job["source"])
|
||||
text = self._sj_text(path)
|
||||
self._sj_write(path, switched(text, self._sj_line(job, text), enabled=enabled))
|
||||
return {"success": True, "output": f"{job['key']} {word}"}
|
||||
|
||||
def _sj_as(self, user: str, script: str) -> list[str]:
|
||||
"""*script* run by ``sh`` as *user*, from root."""
|
||||
if user == "root":
|
||||
return ["sh", "-c", script]
|
||||
return ["su", "-s", "/bin/sh", user, "-c", script]
|
||||
|
||||
def _sj_run(self, job: dict[str, object], reading: Reading) -> ScheduledJobResultDict:
|
||||
bound = ["timeout", str(RUN_TIMEOUT)]
|
||||
wait = RUN_TIMEOUT + 30
|
||||
if job["kind"] == "systemd_timer":
|
||||
unit = reading.units[str(job["key"])]
|
||||
since = self.run_command("date +%s", timeout=30).stdout.strip()
|
||||
started = self._sj_root([*bound, "systemctl", "start", "--", unit], timeout=wait)
|
||||
journal = self._sj_root(
|
||||
[
|
||||
"journalctl",
|
||||
"-u",
|
||||
unit,
|
||||
"--since",
|
||||
f"@{since}",
|
||||
"--no-pager",
|
||||
"-o",
|
||||
"cat",
|
||||
"-n",
|
||||
"200",
|
||||
]
|
||||
)
|
||||
output = "\n".join(p for p in (_said(started), journal.stdout.strip()) if p)
|
||||
return {"success": started.exit_code == 0, "output": output or f"{unit} ran"}
|
||||
if job["kind"] == "run_parts":
|
||||
return _result(self._sj_root([*bound, str(job["source"])], timeout=wait), "ran")
|
||||
path = str(job["source"])
|
||||
line = self._sj_line(job, self._sj_text(path))
|
||||
script = run_script(line.command, line.env)
|
||||
return _result(
|
||||
self._sj_root([*bound, *self._sj_as(line.run_as, script)], timeout=wait), "ran"
|
||||
)
|
||||
|
||||
def manage_scheduled_job(self, key: str, action: str, *, digest: str) -> ScheduledJobResultDict:
|
||||
"""Enable, disable or run the job *key* now; see :class:`ScheduledJobsMixin`."""
|
||||
if action not in ACTIONS:
|
||||
raise ScheduledJobRefused(f"unknown action {action!r}; allowed: {list(ACTIONS)}")
|
||||
reading = self._sj_read()
|
||||
job = self._sj_job(reading, key, digest)
|
||||
if action not in job["actions"]: # type: ignore[operator]
|
||||
raise ScheduledJobRefused(f"{key} cannot be told to {action} on this host")
|
||||
if action == "run":
|
||||
return self._sj_run(job, reading)
|
||||
return self._sj_switch(job, action == "enable")
|
||||
|
||||
# -- an application's own jobs ---------------------------------------------------------------
|
||||
|
||||
def _sj_own(
|
||||
self, reading: Reading, spec: ScheduledJobSpecDict, digest: str | None
|
||||
) -> dict[str, object] | None:
|
||||
current = next((j for j in reading.jobs if j["key"] == f"job:{spec['name']}"), None)
|
||||
if digest is None:
|
||||
if current is not None:
|
||||
raise ScheduledJobChanged(f"a job {spec['name']} is there already")
|
||||
return None
|
||||
if current is None:
|
||||
raise LookupError(f"no scheduled job job:{spec['name']}")
|
||||
if current["digest"] != digest:
|
||||
raise ScheduledJobChanged(f"job:{spec['name']} changed on the host since it was read")
|
||||
return current
|
||||
|
||||
def _sj_group(self, user: str) -> str:
|
||||
result = self.run_command(_command(["id", "-gn", "--", user]), timeout=30)
|
||||
if result.exit_code != 0:
|
||||
raise ScheduledJobRefused(f"the host has no user {user}")
|
||||
return result.stdout.strip()
|
||||
|
||||
def _sj_calendar(self, schedule: str) -> list[str]:
|
||||
if is_reboot(schedule):
|
||||
return ["OnBootSec=1min"]
|
||||
lines = to_on_calendar(schedule)
|
||||
for line in lines:
|
||||
checked = self.run_command(
|
||||
_command(["systemd-analyze", "calendar", "--", line]), timeout=30
|
||||
)
|
||||
if checked.exit_code != 0:
|
||||
raise ScheduledJobRefused(f"systemd does not take {line!r}: {_said(checked)}")
|
||||
return [f"OnCalendar={line}" for line in lines]
|
||||
|
||||
def _sj_put_timer(self, spec: ScheduledJobSpecDict) -> None:
|
||||
name, description = spec["name"], spec.get("description") or spec["name"]
|
||||
script = f"{self.JOB_SCRIPT_DIR}/{name}"
|
||||
unit_description = description.replace("%", "%%")
|
||||
timer = "\n".join(
|
||||
[
|
||||
"[Unit]",
|
||||
f"Description={unit_description}",
|
||||
"",
|
||||
"[Timer]",
|
||||
*self._sj_calendar(spec["schedule"]),
|
||||
f"Unit={name}.service",
|
||||
"",
|
||||
"[Install]",
|
||||
"WantedBy=timers.target",
|
||||
"",
|
||||
]
|
||||
)
|
||||
service = "\n".join(
|
||||
[
|
||||
"[Unit]",
|
||||
f"Description={unit_description}",
|
||||
"",
|
||||
"[Service]",
|
||||
"Type=oneshot",
|
||||
f"User={spec['run_as']}",
|
||||
f"ExecStart={script}",
|
||||
"",
|
||||
]
|
||||
)
|
||||
body = f"#!/bin/sh\n{JOB_MARKER}{name} {spec.get('description') or ''}".rstrip()
|
||||
body += f"\n{SCHEDULE_HEADER}{spec['schedule']}\n{spec['command'].strip()}\n"
|
||||
self._sj_install(body, script, "750", "root", self._sj_group(spec["run_as"]))
|
||||
self._sj_install(service, f"{self.UNIT_DIR}/{name}.service", "644", "root", self.ROOT_GROUP)
|
||||
self._sj_install(timer, f"{self.UNIT_DIR}/{name}.timer", "644", "root", self.ROOT_GROUP)
|
||||
self._sj_checked(["systemctl", "daemon-reload"])
|
||||
verb = "enable" if spec["enabled"] else "disable"
|
||||
self._sj_checked(["systemctl", verb, "--now", "--", f"{name}.timer"])
|
||||
|
||||
def _sj_cron_text(self, spec: ScheduledJobSpecDict, *, system: bool) -> list[str]:
|
||||
if "\n" in spec["command"].strip():
|
||||
raise ScheduledJobRefused("a cron line holds one line of command")
|
||||
owner = f" {spec['run_as']}" if system else ""
|
||||
line = f"{spec['schedule']}{owner} {spec['command'].strip()}"
|
||||
return marked_lines(
|
||||
spec["name"], spec.get("description") or "", line, enabled=spec["enabled"]
|
||||
)
|
||||
|
||||
def _sj_cron_d(self) -> str | None:
|
||||
""":attr:`OWN_CRON_D_DIR` where the host has it; BusyBox crond reads none."""
|
||||
directory = self.OWN_CRON_D_DIR
|
||||
if directory is None:
|
||||
return None
|
||||
found = self.run_command(_command(["test", "-d", directory]), timeout=30)
|
||||
return directory if found.exit_code == 0 else None
|
||||
|
||||
def _sj_put_cron(self, spec: ScheduledJobSpecDict, current: dict[str, object] | None) -> None:
|
||||
if self._sj_cron_d() is not None:
|
||||
self._sj_group(spec["run_as"])
|
||||
text = "\n".join(self._sj_cron_text(spec, system=True)) + "\n"
|
||||
self._sj_install(
|
||||
text, f"{self.OWN_CRON_D_DIR}/{spec['name']}", "644", "root", self.ROOT_GROUP
|
||||
)
|
||||
return
|
||||
if current is not None and current["run_as"] != spec["run_as"]:
|
||||
self._sj_delete_cron(current)
|
||||
user = spec["run_as"]
|
||||
text = self._sj_crontab(user)
|
||||
jobs = parse_crontab("", text, system=False, user=user)
|
||||
self._sj_write_crontab(
|
||||
user, with_job(text, jobs, spec["name"], self._sj_cron_text(spec, system=False))
|
||||
)
|
||||
|
||||
def put_scheduled_job(
|
||||
self, spec: ScheduledJobSpecDict, *, digest: str | None = None
|
||||
) -> ScheduledJobResultDict:
|
||||
"""Create or change the application's own job; see :class:`ScheduledJobsMixin`."""
|
||||
check_spec(spec)
|
||||
reading = self._sj_read()
|
||||
current = self._sj_own(reading, spec, digest)
|
||||
if reading.systemd:
|
||||
self._sj_put_timer(spec)
|
||||
else:
|
||||
self._sj_put_cron(spec, current)
|
||||
return {
|
||||
"success": True,
|
||||
"output": f"job:{spec['name']} {'changed' if current else 'created'}",
|
||||
}
|
||||
|
||||
def _sj_delete_cron(self, job: dict[str, object]) -> None:
|
||||
path = str(job["source"])
|
||||
if self._sj_spool_user(path) is None:
|
||||
self._sj_checked(["rm", "-f", "--", path])
|
||||
return
|
||||
text = self._sj_text(path)
|
||||
jobs = parse_crontab(path, text, system=False, user=str(job["run_as"]))
|
||||
self._sj_write(path, without_job(text, jobs, str(job["name"])))
|
||||
|
||||
def delete_scheduled_job(self, key: str, *, digest: str) -> ScheduledJobResultDict:
|
||||
"""Delete the application's job *key*; see :class:`ScheduledJobsMixin`."""
|
||||
job = self._sj_job(self._sj_read(), key, digest)
|
||||
if job["managed_by"] != "application":
|
||||
raise ScheduledJobRefused(
|
||||
f"{key} was not written by an application; it is only switched"
|
||||
)
|
||||
name = str(job["name"])
|
||||
if job["kind"] != "systemd_timer":
|
||||
self._sj_delete_cron(job)
|
||||
return {"success": True, "output": f"{key} deleted"}
|
||||
self._sj_root(["systemctl", "disable", "--now", "--", f"{name}.timer"])
|
||||
self._sj_checked(
|
||||
[
|
||||
"rm",
|
||||
"-f",
|
||||
"--",
|
||||
f"{self.UNIT_DIR}/{name}.timer",
|
||||
f"{self.UNIT_DIR}/{name}.service",
|
||||
f"{self.JOB_SCRIPT_DIR}/{name}",
|
||||
]
|
||||
)
|
||||
self._sj_checked(["systemctl", "daemon-reload"])
|
||||
return {"success": True, "output": f"{key} deleted"}
|
||||
@@ -15,7 +15,9 @@ accident.
|
||||
``/etc/ssh/sshd_config.d/*.conf`` -- Debian and Ubuntu do, at the top -- the
|
||||
lists go to :data:`DROPIN_PATH`, which sorts first and so wins. sshd_config
|
||||
stays as the distribution shipped it, and removing the file undoes the change.
|
||||
Only without such an ``Include`` is sshd_config itself rewritten, after a copy.
|
||||
Only without such an ``Include`` is sshd_config itself rewritten, after a copy --
|
||||
and not at all on a platform that generates it
|
||||
(:attr:`OpenSshServerMixin.SSHD_CONFIG_GENERATED`).
|
||||
|
||||
**Nobody is locked out, by three checks.**
|
||||
|
||||
@@ -426,9 +428,16 @@ class OpenSshServerMixin:
|
||||
whose platform writes sshd_config itself (an appliance that regenerates it
|
||||
from its own database) must not inherit this, or must opt out with
|
||||
``harden_ssh_server = None``: its file would be overwritten on the next
|
||||
apply.
|
||||
apply. Where that generated file still includes the drop-in directory, the
|
||||
driver sets :attr:`SSHD_CONFIG_GENERATED` instead and keeps the hardening.
|
||||
"""
|
||||
|
||||
#: True where the platform writes sshd_config from its own database
|
||||
#: (OpenMediaVault): only the drop-in directory is written, and a
|
||||
#: sshd_config that does not include it is left alone, since the platform
|
||||
#: would write over a change to it on its next apply.
|
||||
SSHD_CONFIG_GENERATED: bool = False
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def run_command(
|
||||
@@ -565,6 +574,13 @@ class OpenSshServerMixin:
|
||||
main = self._ssh_read(MAIN_CONFIG)
|
||||
if main is None:
|
||||
raise RuntimeError(f"cannot read {MAIN_CONFIG}")
|
||||
if self.SSHD_CONFIG_GENERATED and not has_dropin_include(main):
|
||||
result.message = (
|
||||
f"Not changed: {MAIN_CONFIG} does not include {DROPIN_DIR}, and this"
|
||||
" platform writes it from its own settings."
|
||||
)
|
||||
say(result.message)
|
||||
return result
|
||||
restore = self._write_change(plan, main, result, say)
|
||||
return self._apply_or_restore(
|
||||
result,
|
||||
|
||||
+1
-1
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "napalm-device-types"
|
||||
version = "4.23.0"
|
||||
version = "4.28.0"
|
||||
description = "Abstract device-type base classes for NAPALM drivers"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.12"
|
||||
|
||||
@@ -32,7 +32,7 @@
|
||||
},
|
||||
"mypy_relaxed": [],
|
||||
"python_file_lines": {
|
||||
"napalm_device_types/models.py": 1463
|
||||
"napalm_device_types/models.py": 1454
|
||||
},
|
||||
"untyped_public": []
|
||||
}
|
||||
@@ -0,0 +1,234 @@
|
||||
"""A backup server is a role of its own (NAPALM/napalm-device-types#82).
|
||||
|
||||
What makes a device a backup server -- datastores, the backup groups in them,
|
||||
the jobs that keep them (garbage collection, verify, prune, sync, tape) and the
|
||||
remotes it syncs with -- has no counterpart in ``StorageDriver``. A backup
|
||||
server that also manages its own disks lists ``StorageDriver`` as a second role.
|
||||
|
||||
The declarations sit under ``TYPE_CHECKING`` (see test_role_contracts), so the
|
||||
signatures are read from the source rather than from the class.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import ast
|
||||
import inspect
|
||||
from typing import NotRequired, get_args, get_type_hints
|
||||
|
||||
import pytest
|
||||
|
||||
import napalm_device_types.backup_server as backup_server_module
|
||||
from napalm_device_types import (
|
||||
BACKUP_JOB_KINDS,
|
||||
BackupDatastoreDict,
|
||||
BackupGroupDict,
|
||||
BackupRemoteDict,
|
||||
BackupServerDriver,
|
||||
BackupServerJobDict,
|
||||
BackupServerRefDict,
|
||||
BackupSnapshotDict,
|
||||
BackupSnapshotRefDict,
|
||||
BackupTaskDict,
|
||||
BackupTaskStatusDict,
|
||||
ClusterStorageDict,
|
||||
OSDriver,
|
||||
StorageDriver,
|
||||
)
|
||||
from napalm_device_types.backups import BackupJobKind, BackupRunState, BackupVerifyState
|
||||
from napalm_device_types.roles import primary_role_of, role_keys_of
|
||||
|
||||
|
||||
def _declared(name: str) -> ast.FunctionDef:
|
||||
tree = ast.parse(inspect.getsource(backup_server_module))
|
||||
for node in ast.walk(tree):
|
||||
if isinstance(node, ast.FunctionDef) and node.name == name:
|
||||
return node
|
||||
raise AssertionError(f"BackupServerDriver does not declare {name}()")
|
||||
|
||||
|
||||
def _signature(name: str) -> tuple[list[str], str]:
|
||||
fn = _declared(name)
|
||||
params = ast.unparse(fn.args).split(", ")[1:]
|
||||
return params, ast.unparse(fn.returns) if fn.returns else ""
|
||||
|
||||
|
||||
class TestRole:
|
||||
def test_role_key_and_label(self):
|
||||
assert vars(BackupServerDriver)["ROLE"] == "backup_server"
|
||||
assert BackupServerDriver.TYPE_LABEL == "Backup Server"
|
||||
|
||||
def test_backup_server_first_storage_second(self):
|
||||
"""A PBS manages its disks and ZFS pools too; that part is storage."""
|
||||
|
||||
class Pbs(BackupServerDriver, StorageDriver, OSDriver):
|
||||
pass
|
||||
|
||||
assert role_keys_of(Pbs) == ["backup_server", "storage", "linux"]
|
||||
assert primary_role_of(Pbs) == "backup_server"
|
||||
|
||||
|
||||
class TestDeclarations:
|
||||
@pytest.mark.parametrize(
|
||||
("name", "params", "returns"),
|
||||
[
|
||||
("get_backup_datastores", [], "list[BackupDatastoreDict]"),
|
||||
("get_backup_groups", ["datastore: str=''"], "list[BackupGroupDict]"),
|
||||
(
|
||||
"get_backup_snapshots",
|
||||
["datastore: str", "namespace: str", "group_type: str", "group_id: str"],
|
||||
"list[BackupSnapshotDict]",
|
||||
),
|
||||
("get_backup_jobs", [], "list[BackupServerJobDict]"),
|
||||
("get_backup_remotes", [], "list[BackupRemoteDict]"),
|
||||
("start_garbage_collection", ["datastore: str"], "BackupTaskDict"),
|
||||
("run_backup_job", ["kind: BackupJobKind", "job_id: str"], "BackupTaskDict"),
|
||||
("verify_backup_snapshot", ["snapshot: BackupSnapshotRefDict"], "BackupTaskDict"),
|
||||
(
|
||||
"set_backup_snapshot_protected",
|
||||
["snapshot: BackupSnapshotRefDict", "protected: bool"],
|
||||
"None",
|
||||
),
|
||||
("delete_backup_snapshot", ["snapshot: BackupSnapshotRefDict"], "None"),
|
||||
("get_backup_task", ["task: str"], "BackupTaskStatusDict"),
|
||||
],
|
||||
)
|
||||
def test_declared_with_its_signature(self, name, params, returns):
|
||||
got_params, got_returns = _signature(name)
|
||||
assert [p.replace(" ", "") for p in got_params] == [p.replace(" ", "") for p in params]
|
||||
assert got_returns == returns
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"name",
|
||||
[
|
||||
"get_backup_datastores",
|
||||
"get_backup_groups",
|
||||
"get_backup_snapshots",
|
||||
"get_backup_jobs",
|
||||
"get_backup_remotes",
|
||||
"start_garbage_collection",
|
||||
"run_backup_job",
|
||||
"verify_backup_snapshot",
|
||||
"set_backup_snapshot_protected",
|
||||
"delete_backup_snapshot",
|
||||
"get_backup_task",
|
||||
],
|
||||
)
|
||||
def test_absent_until_a_driver_implements_it(self, name):
|
||||
assert not hasattr(BackupServerDriver, name)
|
||||
|
||||
def test_every_declaration_has_a_docstring(self):
|
||||
tree = ast.parse(inspect.getsource(backup_server_module))
|
||||
missing = [
|
||||
n.name
|
||||
for n in ast.walk(tree)
|
||||
if isinstance(n, ast.FunctionDef) and not ast.get_docstring(n)
|
||||
]
|
||||
assert missing == []
|
||||
|
||||
|
||||
class TestShapes:
|
||||
"""Neutral names: nothing here is Proxmox's. A value the platform does not
|
||||
report is None; it is never guessed."""
|
||||
|
||||
def test_datastore(self):
|
||||
assert set(get_type_hints(BackupDatastoreDict)) == {
|
||||
"name",
|
||||
"path",
|
||||
"backend",
|
||||
"total",
|
||||
"used",
|
||||
"available",
|
||||
"estimated_full",
|
||||
"maintenance",
|
||||
"comment",
|
||||
}
|
||||
|
||||
def test_group_carries_its_newest_snapshot_not_all_of_them(self):
|
||||
assert set(get_type_hints(BackupGroupDict)) == {
|
||||
"datastore",
|
||||
"namespace",
|
||||
"type",
|
||||
"id",
|
||||
"owner",
|
||||
"count",
|
||||
"newest",
|
||||
"oldest_time",
|
||||
"comment",
|
||||
}
|
||||
assert get_type_hints(BackupGroupDict)["newest"] == BackupSnapshotDict | None
|
||||
|
||||
def test_snapshot(self):
|
||||
assert set(get_type_hints(BackupSnapshotDict)) == {
|
||||
"time",
|
||||
"size",
|
||||
"verify",
|
||||
"protected",
|
||||
"comment",
|
||||
}
|
||||
assert set(get_args(BackupVerifyState)) == {"ok", "failed"}
|
||||
|
||||
def test_snapshot_reference_names_one_snapshot(self):
|
||||
assert set(get_type_hints(BackupSnapshotRefDict)) == {
|
||||
"datastore",
|
||||
"namespace",
|
||||
"group_type",
|
||||
"group_id",
|
||||
"time",
|
||||
}
|
||||
|
||||
def test_job_kinds_include_garbage_collection(self):
|
||||
"""GC is a job with a schedule and a last run like the others, so it is
|
||||
listed with them rather than restated on the datastore."""
|
||||
assert set(get_args(BackupJobKind)) == {"gc", "verify", "prune", "sync", "tape"}
|
||||
assert set(BACKUP_JOB_KINDS) == set(get_args(BackupJobKind))
|
||||
|
||||
def test_job(self):
|
||||
assert set(get_type_hints(BackupServerJobDict)) == {
|
||||
"id",
|
||||
"kind",
|
||||
"datastore",
|
||||
"namespace",
|
||||
"schedule",
|
||||
"enabled",
|
||||
"last_run",
|
||||
"last_state",
|
||||
"last_error",
|
||||
"next_run",
|
||||
"remote",
|
||||
"direction",
|
||||
"comment",
|
||||
}
|
||||
assert set(get_args(BackupRunState)) == {"ok", "warning", "error", "running"}
|
||||
|
||||
def test_remote_never_carries_its_secret(self):
|
||||
fields = set(get_type_hints(BackupRemoteDict))
|
||||
assert fields == {"name", "host", "port", "fingerprint", "auth_id", "comment"}
|
||||
assert not fields & {"password", "secret", "token"}
|
||||
|
||||
def test_task(self):
|
||||
assert set(get_type_hints(BackupTaskDict)) == {"task"}
|
||||
assert set(get_type_hints(BackupTaskStatusDict)) == {
|
||||
"task",
|
||||
"running",
|
||||
"state",
|
||||
"error",
|
||||
"log",
|
||||
}
|
||||
|
||||
|
||||
class TestClusterStorageNamesItsBackupServer:
|
||||
"""A hypervisor's backup storage says which server and datastore it points
|
||||
at, so an application can tie the server's groups to the cluster's guests."""
|
||||
|
||||
def test_backup_server_reference_is_optional_on_a_storage(self):
|
||||
hints = get_type_hints(ClusterStorageDict, include_extras=True)
|
||||
assert hints["backup_server"] == NotRequired[BackupServerRefDict | None]
|
||||
|
||||
def test_reference(self):
|
||||
assert set(get_type_hints(BackupServerRefDict)) == {
|
||||
"server",
|
||||
"port",
|
||||
"datastore",
|
||||
"namespace",
|
||||
"fingerprint",
|
||||
}
|
||||
@@ -25,6 +25,7 @@ import napalm_device_types.hypervisor as hypervisor_module
|
||||
from napalm_device_types import (
|
||||
BackupDict,
|
||||
BackupJobDict,
|
||||
BackupServerRefDict,
|
||||
ClusterDict,
|
||||
ClusterNetworkDict,
|
||||
ClusterNodeDict,
|
||||
@@ -92,7 +93,8 @@ class TestClusterShape:
|
||||
assert "votes" not in ClusterNodeDict.__required_keys__
|
||||
|
||||
def test_storage(self):
|
||||
assert get_type_hints(ClusterStorageDict) == {
|
||||
hints = get_type_hints(ClusterStorageDict, include_extras=True)
|
||||
assert hints == {
|
||||
"name": str,
|
||||
"type": str,
|
||||
"shared": bool,
|
||||
@@ -100,7 +102,10 @@ class TestClusterShape:
|
||||
"nodes": list[str],
|
||||
"total": int | None,
|
||||
"used": int | None,
|
||||
"backup_server": NotRequired[BackupServerRefDict | None],
|
||||
}
|
||||
# A driver written before backup servers were part of the report stays valid.
|
||||
assert "backup_server" not in ClusterStorageDict.__required_keys__
|
||||
|
||||
def test_network(self):
|
||||
hints = get_type_hints(ClusterNetworkDict, include_extras=True)
|
||||
|
||||
@@ -0,0 +1,141 @@
|
||||
"""Cron expressions: what they mean, when they fire next, and as systemd ``OnCalendar``.
|
||||
|
||||
One module answers all three, because a job netOrk writes as a timer must fire
|
||||
exactly when the cron expression the user gave says, and the preview must show
|
||||
the same runs the host will make (NetOrk/netork#1933).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import UTC, datetime
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types.cron import is_reboot, next_runs, parse_cron, to_on_calendar
|
||||
|
||||
BERLIN = ZoneInfo("Europe/Berlin")
|
||||
|
||||
|
||||
class TestParse:
|
||||
def test_every_field_is_expanded_to_its_values(self):
|
||||
s = parse_cron("*/15 6-8 1,15 * *")
|
||||
assert s.minutes == frozenset({0, 15, 30, 45})
|
||||
assert s.hours == frozenset({6, 7, 8})
|
||||
assert s.days == frozenset({1, 15})
|
||||
assert s.months == frozenset(range(1, 13))
|
||||
assert s.weekdays == frozenset(range(7))
|
||||
|
||||
def test_a_range_takes_a_step(self):
|
||||
assert parse_cron("1-30/10 * * * *").minutes == frozenset({1, 11, 21})
|
||||
|
||||
def test_names_are_months_and_weekdays_in_any_case(self):
|
||||
s = parse_cron("0 0 * Jan,MAR mon-Fri")
|
||||
assert s.months == frozenset({1, 3})
|
||||
assert s.weekdays == frozenset({1, 2, 3, 4, 5})
|
||||
|
||||
def test_seven_is_sunday_too(self):
|
||||
assert parse_cron("0 0 * * 7").weekdays == frozenset({0})
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("alias", "expanded"),
|
||||
[
|
||||
("@hourly", "0 * * * *"),
|
||||
("@daily", "0 0 * * *"),
|
||||
("@midnight", "0 0 * * *"),
|
||||
("@weekly", "0 0 * * 0"),
|
||||
("@monthly", "0 0 1 * *"),
|
||||
("@yearly", "0 0 1 1 *"),
|
||||
("@annually", "0 0 1 1 *"),
|
||||
],
|
||||
)
|
||||
def test_an_alias_is_its_expression(self, alias, expanded):
|
||||
assert parse_cron(alias) == parse_cron(expanded)
|
||||
|
||||
def test_reboot_is_no_calendar(self):
|
||||
assert is_reboot("@reboot")
|
||||
assert not is_reboot("0 0 * * *")
|
||||
with pytest.raises(ValueError, match="reboot"):
|
||||
parse_cron("@reboot")
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"bad",
|
||||
[
|
||||
"",
|
||||
"* * * *",
|
||||
"* * * * * *",
|
||||
"60 * * * *",
|
||||
"* 24 * * *",
|
||||
"* * 0 * *",
|
||||
"* * * 13 *",
|
||||
"* * * * 8",
|
||||
"*/0 * * * *",
|
||||
"5-1 * * * *",
|
||||
"a * * * *",
|
||||
"@often",
|
||||
"1,,2 * * * *",
|
||||
],
|
||||
)
|
||||
def test_what_cron_would_not_take_is_refused(self, bad):
|
||||
with pytest.raises(ValueError):
|
||||
parse_cron(bad)
|
||||
|
||||
def test_day_of_month_and_weekday_are_or_only_when_both_are_restricted(self):
|
||||
assert parse_cron("0 0 1 * 1").either_day
|
||||
assert not parse_cron("0 0 */2 * 1").either_day
|
||||
assert not parse_cron("0 0 1 * *").either_day
|
||||
|
||||
|
||||
class TestNextRuns:
|
||||
def test_runs_after_the_instant_in_the_hosts_time(self):
|
||||
after = datetime(2026, 11, 3, 5, 50, tzinfo=UTC) # 06:50 in Berlin
|
||||
runs = next_runs("0 7 * * *", after, BERLIN, count=2)
|
||||
assert runs == [
|
||||
datetime(2026, 11, 3, 7, 0, tzinfo=BERLIN),
|
||||
datetime(2026, 11, 4, 7, 0, tzinfo=BERLIN),
|
||||
]
|
||||
|
||||
def test_the_instant_itself_is_not_a_next_run(self):
|
||||
after = datetime(2026, 11, 3, 6, 0, tzinfo=UTC)
|
||||
assert next_runs("0 6 * * *", after, ZoneInfo("UTC"), count=1)[0].day == 4
|
||||
|
||||
def test_either_day_fires_on_both(self):
|
||||
after = datetime(2026, 11, 30, 12, 0, tzinfo=UTC) # a Monday
|
||||
runs = next_runs("0 0 1 * 1", after, ZoneInfo("UTC"), count=3)
|
||||
assert [r.day for r in runs] == [1, 7, 14]
|
||||
|
||||
def test_a_day_that_never_comes_gives_no_runs(self):
|
||||
assert next_runs("0 0 31 2 *", datetime(2026, 1, 1, tzinfo=UTC), BERLIN) == []
|
||||
|
||||
def test_a_time_the_clock_skips_is_left_out(self):
|
||||
# 2026-03-29 02:30 does not exist in Berlin.
|
||||
after = datetime(2026, 3, 28, 12, 0, tzinfo=UTC)
|
||||
runs = next_runs("30 2 * * *", after, BERLIN, count=2)
|
||||
assert [r.day for r in runs] == [30, 31]
|
||||
|
||||
def test_reboot_has_no_next_run(self):
|
||||
assert next_runs("@reboot", datetime(2026, 1, 1, tzinfo=UTC), BERLIN) == []
|
||||
|
||||
|
||||
class TestOnCalendar:
|
||||
def test_every_field_as_its_values(self):
|
||||
assert to_on_calendar("*/15 6 * * *") == ["*-*-* 06:00,15,30,45:00"]
|
||||
|
||||
def test_a_restricted_weekday_leads(self):
|
||||
assert to_on_calendar("30 4 * * 1-5") == ["Mon,Tue,Wed,Thu,Fri *-*-* 04:30:00"]
|
||||
|
||||
def test_months_and_days(self):
|
||||
assert to_on_calendar("0 0 1,15 1,7 *") == ["*-01,07-01,15 00:00:00"]
|
||||
|
||||
def test_either_day_is_two_lines(self):
|
||||
assert to_on_calendar("0 0 1 * 0") == ["*-*-01 00:00:00", "Sun *-*-* 00:00:00"]
|
||||
|
||||
def test_both_days_with_a_star_are_one_line(self):
|
||||
assert to_on_calendar("0 0 */10 * 6") == ["Sat *-*-01,11,21,31 00:00:00"]
|
||||
|
||||
def test_an_alias(self):
|
||||
assert to_on_calendar("@daily") == ["*-*-* 00:00:00"]
|
||||
|
||||
def test_reboot_is_refused(self):
|
||||
with pytest.raises(ValueError, match="reboot"):
|
||||
to_on_calendar("@reboot")
|
||||
@@ -0,0 +1,145 @@
|
||||
"""Crontabs as text: the jobs in them, and changing one line (NetOrk/netork#1933)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types._crontab import (
|
||||
DISABLED_MARKER,
|
||||
marked_lines,
|
||||
parse_anacrontab,
|
||||
parse_crontab,
|
||||
run_script,
|
||||
switched,
|
||||
with_job,
|
||||
without_job,
|
||||
)
|
||||
|
||||
SYSTEM = """\
|
||||
# /etc/crontab: system-wide crontab
|
||||
SHELL=/bin/sh
|
||||
PATH="/usr/local/sbin:/usr/bin:/bin"
|
||||
17 * * * * root cd / && run-parts --report /etc/cron.hourly
|
||||
@reboot root /usr/local/bin/warm-cache # keeps the # in the command
|
||||
#napalm-disabled: 0 4 * * * backup /usr/local/bin/backup
|
||||
garbage line
|
||||
0 4 * * * backup /usr/local/bin/backup
|
||||
"""
|
||||
|
||||
USER = """\
|
||||
MAILTO=ops@example.org
|
||||
*/5 * * * * /usr/bin/php /var/www/cron.php
|
||||
# napalm-scheduled-job: netork-cleanup tidy the cache
|
||||
0 3 * * * find /tmp -mtime +7 -delete
|
||||
"""
|
||||
|
||||
|
||||
class TestParse:
|
||||
def test_a_system_crontab_names_the_user(self):
|
||||
jobs = parse_crontab("/etc/crontab", SYSTEM, system=True)
|
||||
first = jobs[0]
|
||||
assert (first.schedule, first.run_as) == ("17 * * * *", "root")
|
||||
assert first.command == "cd / && run-parts --report /etc/cron.hourly"
|
||||
assert first.env == {"SHELL": "/bin/sh", "PATH": "/usr/local/sbin:/usr/bin:/bin"}
|
||||
assert first.key.startswith("cron:/etc/crontab:")
|
||||
|
||||
def test_an_alias_and_a_hash_inside_the_command(self):
|
||||
reboot = parse_crontab("/etc/crontab", SYSTEM, system=True)[1]
|
||||
assert reboot.schedule == "@reboot"
|
||||
assert reboot.command == "/usr/local/bin/warm-cache # keeps the # in the command"
|
||||
|
||||
def test_a_line_switched_off_is_a_job_that_is_off(self):
|
||||
jobs = parse_crontab("/etc/crontab", SYSTEM, system=True)
|
||||
off = jobs[2]
|
||||
assert not off.enabled
|
||||
assert off.run_as == "backup"
|
||||
|
||||
def test_equal_lines_get_their_own_keys_and_switching_off_keeps_the_key(self):
|
||||
jobs = parse_crontab("/etc/crontab", SYSTEM, system=True)
|
||||
assert jobs[3].key == jobs[2].key + ":2"
|
||||
assert len(jobs) == 4 # the garbage line is left out
|
||||
|
||||
def test_a_user_crontab_runs_as_its_user(self):
|
||||
jobs = parse_crontab("/var/spool/cron/crontabs/www", USER, system=False, user="www-data")
|
||||
assert jobs[0].run_as == "www-data"
|
||||
assert jobs[0].env == {"MAILTO": "ops@example.org"}
|
||||
|
||||
def test_an_applications_job_is_keyed_by_its_name(self):
|
||||
job = parse_crontab("/x", USER, system=False, user="root")[1]
|
||||
assert (job.key, job.name, job.description) == (
|
||||
"job:netork-cleanup",
|
||||
"netork-cleanup",
|
||||
"tidy the cache",
|
||||
)
|
||||
|
||||
def test_an_anacrontab(self):
|
||||
text = "SHELL=/bin/sh\n1\t5\tcron.daily\trun-parts --report /etc/cron.daily\n@monthly 15 cron.monthly run-parts /etc/cron.monthly\nbad line here\n"
|
||||
jobs = parse_anacrontab(text)
|
||||
assert [(j.key, j.schedule, j.run_as) for j in jobs] == [
|
||||
("anacron:cron.daily", "1", "root"),
|
||||
("anacron:cron.monthly", "@monthly", "root"),
|
||||
]
|
||||
assert jobs[0].command == "run-parts --report /etc/cron.daily"
|
||||
|
||||
|
||||
class TestEdit:
|
||||
def test_switching_off_and_on_again_gives_the_text_back(self):
|
||||
jobs = parse_crontab("/etc/crontab", SYSTEM, system=True)
|
||||
off = switched(SYSTEM, jobs[0], enabled=False)
|
||||
assert DISABLED_MARKER + "17 *\t* * *\troot" in off
|
||||
again = parse_crontab("/etc/crontab", off, system=True)
|
||||
assert again[0].key == jobs[0].key and not again[0].enabled
|
||||
assert switched(off, again[0], enabled=True) == SYSTEM.replace(
|
||||
"17 *\t* * *\troot\tcd", "17 *\t* * *\troot\tcd"
|
||||
)
|
||||
|
||||
def test_an_applications_job_is_added_replaced_and_removed(self):
|
||||
lines = marked_lines("netork-x", "a job", "0 1 * * * /bin/true", enabled=True)
|
||||
added = with_job("A=1\n", [], "netork-x", lines)
|
||||
assert added == "A=1\n# napalm-scheduled-job: netork-x a job\n0 1 * * * /bin/true\n"
|
||||
jobs = parse_crontab("/x", added, system=False, user="root")
|
||||
changed = with_job(
|
||||
added,
|
||||
jobs,
|
||||
"netork-x",
|
||||
marked_lines("netork-x", "", "0 2 * * * /bin/true", enabled=False),
|
||||
)
|
||||
assert (
|
||||
changed
|
||||
== "A=1\n# napalm-scheduled-job: netork-x\n#napalm-disabled: 0 2 * * * /bin/true\n"
|
||||
)
|
||||
jobs = parse_crontab("/x", changed, system=False, user="root")
|
||||
assert without_job(changed, jobs, "netork-x") == "A=1\n"
|
||||
|
||||
def test_adding_to_a_text_without_a_last_newline(self):
|
||||
assert with_job("A=1", [], "n", ["# m", "x"]) == "A=1\n# m\nx\n"
|
||||
|
||||
def test_removing_the_only_job_leaves_nothing(self):
|
||||
text = "# napalm-scheduled-job: n\n0 1 * * * x\n"
|
||||
jobs = parse_crontab("/x", text, system=False, user="root")
|
||||
assert without_job(text, jobs, "n") == ""
|
||||
|
||||
|
||||
class TestRunScript:
|
||||
def _run(self, script: str) -> str:
|
||||
return subprocess.run(
|
||||
["sh", "-c", script], capture_output=True, text=True, check=True
|
||||
).stdout
|
||||
|
||||
def test_the_crontabs_variables_are_set(self):
|
||||
assert (
|
||||
self._run(run_script('echo "$GREETING"', {"GREETING": "hi there", "MAILTO": "x"}))
|
||||
== "hi there\n"
|
||||
)
|
||||
|
||||
def test_a_percent_starts_stdin_and_further_ones_are_newlines(self):
|
||||
assert self._run(run_script("cat%one%two", {})) == "one\ntwo\n"
|
||||
|
||||
def test_an_escaped_percent_is_a_percent(self):
|
||||
assert self._run(run_script("date -d @0 -u +\\%Y", {})) == "1970\n"
|
||||
|
||||
@pytest.mark.parametrize("command", ["echo plain", "echo 'quoted'"])
|
||||
def test_without_percent_the_command_is_as_it_is(self, command):
|
||||
assert run_script(command, {}) == command
|
||||
@@ -11,6 +11,7 @@ import pytest
|
||||
|
||||
from napalm_device_types import (
|
||||
AccessPointDriver,
|
||||
BackupServerDriver,
|
||||
EnsurePackageMixin,
|
||||
FirewallDriver,
|
||||
HypervisorDriver,
|
||||
@@ -28,7 +29,7 @@ from napalm_device_types import (
|
||||
)
|
||||
|
||||
ROLE_BASES = [
|
||||
AccessPointDriver, FirewallDriver, HypervisorDriver, ManagementControllerDriver, MediaDriver,
|
||||
AccessPointDriver, BackupServerDriver, FirewallDriver, HypervisorDriver, ManagementControllerDriver, MediaDriver,
|
||||
OSDriver, PhoneDriver, ResidentialGatewayDriver, StorageDriver, SwitchDriver,
|
||||
] # fmt: skip
|
||||
|
||||
|
||||
@@ -0,0 +1,127 @@
|
||||
"""LLDP rows projected into NAPALM's ``get_lldp_neighbors`` and ``get_lldp_neighbors_detail``.
|
||||
|
||||
A driver reads its neighbours into :class:`LldpNeighborRow` (from ``lldpctl``,
|
||||
an API, a CLI) and the two NAPALM shapes come from those rows the same way for
|
||||
every driver (NAPALM/napalm-device-types#10).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import get_type_hints
|
||||
|
||||
from napalm_device_types import LldpNeighborRow, lldp_neighbors, lldp_neighbors_detail
|
||||
|
||||
|
||||
def _row(local_port: str, name: str, **fields: object) -> LldpNeighborRow:
|
||||
row: LldpNeighborRow = {
|
||||
"local_port": local_port,
|
||||
"chassis_id": "",
|
||||
"chassis_id_type": "",
|
||||
"port_id": "",
|
||||
"port_id_type": "",
|
||||
"port_description": "",
|
||||
"system_name": name,
|
||||
"system_description": "",
|
||||
"management_addresses": [],
|
||||
"capabilities": [],
|
||||
"enabled_capabilities": [],
|
||||
}
|
||||
row.update(fields) # type: ignore[typeddict-item]
|
||||
return row
|
||||
|
||||
|
||||
PHONE = _row(
|
||||
"eth0.1",
|
||||
"desk-phone",
|
||||
chassis_id="00:11:22:33:44:AB",
|
||||
chassis_id_type="mac",
|
||||
port_id="00:11:22:33:44:01",
|
||||
port_id_type="mac",
|
||||
port_description="WAN",
|
||||
system_description="SIP phone",
|
||||
capabilities=["bridge", "telephone"],
|
||||
enabled_capabilities=["telephone"],
|
||||
)
|
||||
PC = _row("eth0.1", "desk-pc", port_id="eno1", port_id_type="ifname")
|
||||
SWITCH = _row("eth1", "access-sw", port_id="gi1/0/7", port_id_type="local")
|
||||
|
||||
|
||||
def test_a_row_names_its_fields():
|
||||
assert set(get_type_hints(LldpNeighborRow)) == {
|
||||
"local_port",
|
||||
"chassis_id",
|
||||
"chassis_id_type",
|
||||
"port_id",
|
||||
"port_id_type",
|
||||
"port_description",
|
||||
"system_name",
|
||||
"system_description",
|
||||
"management_addresses",
|
||||
"capabilities",
|
||||
"enabled_capabilities",
|
||||
}
|
||||
|
||||
|
||||
class TestNeighbors:
|
||||
def test_every_neighbour_of_a_port_is_listed(self):
|
||||
assert lldp_neighbors([PHONE, PC, SWITCH]) == {
|
||||
"eth0.1": [
|
||||
{"hostname": "desk-phone", "port": "00:11:22:33:44:01", "mac": "00:11:22:33:44:ab"},
|
||||
{"hostname": "desk-pc", "port": "eno1"},
|
||||
],
|
||||
"eth1": [{"hostname": "access-sw", "port": "gi1/0/7"}],
|
||||
}
|
||||
|
||||
def test_no_rows_is_no_neighbour(self):
|
||||
assert lldp_neighbors([]) == {}
|
||||
|
||||
def test_a_chassis_mac_is_reported_in_lower_case(self):
|
||||
# netOrk matches a neighbour by this MAC first (NAPALM/napalm-openwrt#31).
|
||||
assert lldp_neighbors([PHONE])["eth0.1"][0]["mac"] == "00:11:22:33:44:ab"
|
||||
|
||||
def test_a_chassis_named_otherwise_has_no_mac(self):
|
||||
switch = _row("eth1", "sw", chassis_id="00:11:22:33:44:55", chassis_id_type="local")
|
||||
|
||||
assert "mac" not in lldp_neighbors([switch])["eth1"][0]
|
||||
|
||||
def test_a_mac_id_that_is_no_mac_address_is_left_out(self):
|
||||
odd = _row("eth1", "sw", chassis_id="00:11", chassis_id_type="mac")
|
||||
|
||||
assert "mac" not in lldp_neighbors([odd])["eth1"][0]
|
||||
|
||||
|
||||
class TestNeighborsDetail:
|
||||
def test_every_field_napalm_names(self):
|
||||
assert lldp_neighbors_detail([PHONE]) == {
|
||||
"eth0.1": [
|
||||
{
|
||||
"parent_interface": "",
|
||||
"remote_port": "00:11:22:33:44:01",
|
||||
"remote_port_description": "WAN",
|
||||
"remote_chassis_id": "00:11:22:33:44:AB",
|
||||
"remote_system_name": "desk-phone",
|
||||
"remote_system_description": "SIP phone",
|
||||
"remote_system_capab": ["bridge", "telephone"],
|
||||
"remote_system_enable_capab": ["telephone"],
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
def test_every_neighbour_of_a_port_is_listed(self):
|
||||
detail = lldp_neighbors_detail([PHONE, PC, SWITCH])
|
||||
|
||||
assert [n["remote_system_name"] for n in detail["eth0.1"]] == ["desk-phone", "desk-pc"]
|
||||
assert [n["remote_system_name"] for n in detail["eth1"]] == ["access-sw"]
|
||||
|
||||
def test_one_interface_only(self):
|
||||
assert list(lldp_neighbors_detail([PHONE, PC, SWITCH], "eth1")) == ["eth1"]
|
||||
|
||||
def test_an_interface_without_neighbours_is_empty(self):
|
||||
assert lldp_neighbors_detail([PHONE], "eth9") == {}
|
||||
|
||||
def test_the_lists_are_the_rows_own_copies(self):
|
||||
detail = lldp_neighbors_detail([PHONE])
|
||||
|
||||
detail["eth0.1"][0]["remote_system_capab"].append("router")
|
||||
|
||||
assert PHONE["capabilities"] == ["bridge", "telephone"]
|
||||
@@ -0,0 +1,308 @@
|
||||
"""``lldpctl -f keyvalue``: one row per LLDP neighbour, for every driver that reads lldpd.
|
||||
|
||||
napalm-linux, napalm-openwrt and napalm-proxmox each parsed this output on their
|
||||
own, and each copy grouped the lines by interface, so two neighbours on one port
|
||||
merged (NAPALM/napalm-openwrt#9, NAPALM/napalm-linux#58), split ``eth0.1`` at its
|
||||
dot (NAPALM/napalm-openwrt#26) and looked for ``chassis.id.value``, a key lldpd
|
||||
never prints. The parse lives here once now.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import LLDPCTL_KEYVALUE_COMMAND, parse_lldpctl_keyvalue
|
||||
|
||||
# The recording napalm-openwrt's tests have used since the driver was written
|
||||
# (tests/unit/test_driver.py, ``LLDPCTL_KV``): one neighbour on eth0. Its two
|
||||
# ``chassis.cap.*`` lines are not lldpd's format (lldpd prints one
|
||||
# ``chassis.<Capability>.enabled=on|off`` line per capability), so they yield none.
|
||||
OPENWRT_ONE_NEIGHBOUR = """\
|
||||
lldp.eth0.via=LLDP
|
||||
lldp.eth0.rid=1
|
||||
lldp.eth0.age=0 day, 01:23:45
|
||||
lldp.eth0.chassis.mac=00:aa:bb:cc:dd:ee
|
||||
lldp.eth0.chassis.name=core-router
|
||||
lldp.eth0.chassis.descr=RouterOS 7.x
|
||||
lldp.eth0.chassis.mgmt-ip=10.0.0.1
|
||||
lldp.eth0.chassis.cap.available=Router, Bridge
|
||||
lldp.eth0.chassis.cap.enabled=Router
|
||||
lldp.eth0.port.ifname=ether1
|
||||
lldp.eth0.port.descr=uplink
|
||||
"""
|
||||
|
||||
# Laid out the way lldpd 1.0's keyvalue writer prints ``show neighbors details``
|
||||
# (src/client/display.c ``display_interface``, ``display_chassis``,
|
||||
# ``display_port``, ``display_vlans``, ``display_med``; src/client/kv_writer.c):
|
||||
# a neighbour opens with the local interface's optional ``alias`` and ``via``,
|
||||
# an id is printed under its type (``chassis.mac``, ``port.local``), a
|
||||
# capability as ``chassis.<Symbol>.enabled``, a management address once per
|
||||
# address. Not recorded from a device.
|
||||
#
|
||||
# On ``eth0.1`` (a VLAN interface, its name holds a dot) a phone and a PC behind
|
||||
# an unmanaged switch; on ``eth1`` a switch that names its port by a local id and
|
||||
# itself by an address.
|
||||
TWO_ON_ONE_DOTTED_PORT = """\
|
||||
lldp.eth0.1.alias=desk
|
||||
lldp.eth0.1.via=LLDP
|
||||
lldp.eth0.1.rid=1
|
||||
lldp.eth0.1.age=0 day, 00:10:00
|
||||
lldp.eth0.1.chassis.mac=00:11:22:33:44:01
|
||||
lldp.eth0.1.chassis.name=desk-phone
|
||||
lldp.eth0.1.chassis.descr=SIP phone, firmware a=1
|
||||
lldp.eth0.1.chassis.mgmt-ip=10.0.0.21
|
||||
lldp.eth0.1.chassis.mgmt-iface=2
|
||||
lldp.eth0.1.chassis.mgmt-ip=fe80::211:22ff:fe33:4401
|
||||
lldp.eth0.1.chassis.mgmt-iface=2
|
||||
lldp.eth0.1.chassis.Bridge.enabled=off
|
||||
lldp.eth0.1.chassis.Tel.enabled=on
|
||||
lldp.eth0.1.port.mac=00:11:22:33:44:01
|
||||
lldp.eth0.1.port.descr=WAN
|
||||
lldp.eth0.1.port.ttl=120
|
||||
lldp.eth0.1.port.auto-negotiation.supported=yes
|
||||
lldp.eth0.1.port.auto-negotiation.enabled=yes
|
||||
lldp.eth0.1.port.auto-negotiation.100Base-TX.hd=yes
|
||||
lldp.eth0.1.port.auto-negotiation.100Base-TX.fd=yes
|
||||
lldp.eth0.1.port.auto-negotiation.current=100BaseTXFD - 2 pair category 5 UTP, full duplex mode
|
||||
lldp.eth0.1.vlan.vlan-id=10
|
||||
lldp.eth0.1.vlan.pvid=yes
|
||||
lldp.eth0.1.vlan=voice
|
||||
lldp.eth0.1.lldp-med.device-type=Communication Device Endpoint (Class III)
|
||||
lldp.eth0.1.lldp-med.Capabilities.available=yes
|
||||
lldp.eth0.1.lldp-med.inventory.software=2.1.4
|
||||
lldp.eth0.1.alias=desk
|
||||
lldp.eth0.1.via=LLDP
|
||||
lldp.eth0.1.rid=2
|
||||
lldp.eth0.1.age=0 day, 00:09:00
|
||||
lldp.eth0.1.chassis.mac=00:11:22:33:44:02
|
||||
lldp.eth0.1.chassis.name=desk-pc
|
||||
lldp.eth0.1.chassis.descr=Debian GNU/Linux 12 (bookworm) Linux 6.1.0-25-amd64
|
||||
lldp.eth0.1.chassis.Bridge.enabled=off
|
||||
lldp.eth0.1.chassis.Router.enabled=off
|
||||
lldp.eth0.1.chassis.Wlan.enabled=off
|
||||
lldp.eth0.1.chassis.Station.enabled=on
|
||||
lldp.eth0.1.port.ifname=eno1
|
||||
lldp.eth0.1.port.ttl=120
|
||||
lldp.eth0.1.unknown-tlvs.unknown-tlv.oui=00,80,C2
|
||||
lldp.eth0.1.unknown-tlvs.unknown-tlv.subtype=9
|
||||
lldp.eth0.1.unknown-tlvs.unknown-tlv.len=1
|
||||
lldp.eth0.1.unknown-tlvs.unknown-tlv=01
|
||||
lldp.eth1.via=LLDP
|
||||
lldp.eth1.rid=3
|
||||
lldp.eth1.age=1 day, 02:00:00
|
||||
lldp.eth1.chassis.ip=192.0.2.7
|
||||
lldp.eth1.chassis.name=access-sw
|
||||
lldp.eth1.chassis.Bridge.enabled=on
|
||||
lldp.eth1.chassis.Router.enabled=off
|
||||
lldp.eth1.chassis.CVLAN.enabled=on
|
||||
lldp.eth1.port.local=gi1/0/7
|
||||
lldp.eth1.port.descr=GigabitEthernet1/0/7
|
||||
lldp.eth1.ppvid.supported=no
|
||||
lldp.eth1.ppvid.enabled=no
|
||||
lldp.eth1.pi=88cc
|
||||
"""
|
||||
|
||||
|
||||
def _rows(output: str = TWO_ON_ONE_DOTTED_PORT) -> list[dict[str, object]]:
|
||||
return [dict(row) for row in parse_lldpctl_keyvalue(output)]
|
||||
|
||||
|
||||
class TestOneRowPerNeighbour:
|
||||
def test_the_openwrt_recording_is_one_neighbour_with_its_fields(self):
|
||||
assert _rows(OPENWRT_ONE_NEIGHBOUR) == [
|
||||
{
|
||||
"local_port": "eth0",
|
||||
"chassis_id": "00:aa:bb:cc:dd:ee",
|
||||
"chassis_id_type": "mac",
|
||||
"port_id": "ether1",
|
||||
"port_id_type": "ifname",
|
||||
"port_description": "uplink",
|
||||
"system_name": "core-router",
|
||||
"system_description": "RouterOS 7.x",
|
||||
"management_addresses": ["10.0.0.1"],
|
||||
"capabilities": [],
|
||||
"enabled_capabilities": [],
|
||||
}
|
||||
]
|
||||
|
||||
def test_two_neighbours_on_one_port_stay_apart(self):
|
||||
rows = _rows()
|
||||
|
||||
assert [(r["local_port"], r["system_name"]) for r in rows] == [
|
||||
("eth0.1", "desk-phone"),
|
||||
("eth0.1", "desk-pc"),
|
||||
("eth1", "access-sw"),
|
||||
]
|
||||
|
||||
def test_a_neighbour_keeps_only_its_own_fields(self):
|
||||
pc = _rows()[1]
|
||||
|
||||
assert pc["chassis_id"] == "00:11:22:33:44:02"
|
||||
assert pc["port_description"] == ""
|
||||
assert pc["management_addresses"] == []
|
||||
|
||||
def test_neighbours_without_alias_are_split_at_via(self):
|
||||
output = OPENWRT_ONE_NEIGHBOUR + OPENWRT_ONE_NEIGHBOUR.replace("core-router", "edge")
|
||||
|
||||
assert [r["system_name"] for r in _rows(output)] == ["core-router", "edge"]
|
||||
|
||||
|
||||
class TestInterfaceName:
|
||||
def test_a_dotted_interface_name_survives(self):
|
||||
assert {r["local_port"] for r in _rows()} == {"eth0.1", "eth1"}
|
||||
|
||||
def test_an_interface_named_like_a_field_survives(self):
|
||||
output = (
|
||||
"lldp.vlan.100.via=LLDP\n"
|
||||
"lldp.vlan.100.chassis.mac=00:aa:bb:cc:dd:01\n"
|
||||
"lldp.vlan.100.port.ifname=port1\n"
|
||||
)
|
||||
|
||||
(row,) = _rows(output)
|
||||
|
||||
assert (row["local_port"], row["chassis_id"], row["port_id"]) == (
|
||||
"vlan.100",
|
||||
"00:aa:bb:cc:dd:01",
|
||||
"port1",
|
||||
)
|
||||
|
||||
def test_without_via_the_name_ends_at_the_first_field(self):
|
||||
output = "lldp.br-lan.10.chassis.name=sw\nlldp.br-lan.10.port.ifname=1\n"
|
||||
|
||||
(row,) = _rows(output)
|
||||
|
||||
assert (row["local_port"], row["system_name"], row["port_id"]) == ("br-lan.10", "sw", "1")
|
||||
|
||||
|
||||
class TestIds:
|
||||
def test_a_port_given_by_its_mac_is_read(self):
|
||||
phone = _rows()[0]
|
||||
|
||||
assert (phone["port_id"], phone["port_id_type"]) == ("00:11:22:33:44:01", "mac")
|
||||
|
||||
def test_a_port_given_by_a_local_id_is_read(self):
|
||||
switch = _rows()[2]
|
||||
|
||||
assert (switch["port_id"], switch["port_id_type"]) == ("gi1/0/7", "local")
|
||||
|
||||
def test_a_chassis_given_by_its_address_is_read(self):
|
||||
switch = _rows()[2]
|
||||
|
||||
assert (switch["chassis_id"], switch["chassis_id_type"]) == ("192.0.2.7", "ip")
|
||||
|
||||
@pytest.mark.parametrize("kind", ["local", "ifname", "ifalias", "unhandled"])
|
||||
def test_every_id_type_lldpd_names_is_read(self, kind):
|
||||
output = f"lldp.eth0.via=LLDP\nlldp.eth0.chassis.{kind}=c1\nlldp.eth0.port.{kind}=p1\n"
|
||||
|
||||
(row,) = _rows(output)
|
||||
|
||||
assert (row["chassis_id"], row["chassis_id_type"]) == ("c1", kind)
|
||||
assert (row["port_id"], row["port_id_type"]) == ("p1", kind)
|
||||
|
||||
def test_a_neighbour_without_ids_has_empty_ones(self):
|
||||
(row,) = _rows("lldp.eth0.via=CDPv2\nlldp.eth0.chassis.name=sw\n")
|
||||
|
||||
assert (row["chassis_id"], row["chassis_id_type"]) == ("", "")
|
||||
assert (row["port_id"], row["port_id_type"]) == ("", "")
|
||||
|
||||
|
||||
class TestOtherFields:
|
||||
def test_every_management_address_is_kept(self):
|
||||
phone = _rows()[0]
|
||||
|
||||
assert phone["management_addresses"] == ["10.0.0.21", "fe80::211:22ff:fe33:4401"]
|
||||
|
||||
def test_a_value_may_hold_an_equals_sign(self):
|
||||
assert _rows()[0]["system_description"] == "SIP phone, firmware a=1"
|
||||
|
||||
def test_capabilities_in_napalm_names_and_which_are_enabled(self):
|
||||
phone, pc, switch = _rows()
|
||||
|
||||
assert phone["capabilities"] == ["bridge", "telephone"]
|
||||
assert phone["enabled_capabilities"] == ["telephone"]
|
||||
assert pc["capabilities"] == ["bridge", "router", "wlan-access-point", "station"]
|
||||
assert pc["enabled_capabilities"] == ["station"]
|
||||
assert switch["capabilities"] == ["bridge", "router", "cvlan"]
|
||||
assert switch["enabled_capabilities"] == ["bridge", "cvlan"]
|
||||
|
||||
def test_every_napalm_capability_name(self):
|
||||
symbols = ["Other", "Repeater", "Bridge", "Router", "Wlan", "Tel", "Docsis", "Station"]
|
||||
output = "lldp.eth0.via=LLDP\n" + "".join(
|
||||
f"lldp.eth0.chassis.{s}.enabled=on\n" for s in symbols
|
||||
)
|
||||
|
||||
(row,) = _rows(output)
|
||||
|
||||
assert row["capabilities"] == [
|
||||
"other",
|
||||
"repeater",
|
||||
"bridge",
|
||||
"router",
|
||||
"wlan-access-point",
|
||||
"telephone",
|
||||
"docsis-cable-device",
|
||||
"station",
|
||||
]
|
||||
assert row["enabled_capabilities"] == row["capabilities"]
|
||||
|
||||
def test_a_capability_line_needs_on_or_off(self):
|
||||
(row,) = _rows(OPENWRT_ONE_NEIGHBOUR)
|
||||
|
||||
assert row["capabilities"] == []
|
||||
|
||||
|
||||
class TestReadOrFailure:
|
||||
"""A failed read raises; ``[]`` only when lldpctl ran and listed nobody."""
|
||||
|
||||
def test_no_output_is_no_neighbour(self):
|
||||
assert parse_lldpctl_keyvalue("") == []
|
||||
|
||||
def test_a_zero_exit_status_without_neighbours_is_no_neighbour(self):
|
||||
assert parse_lldpctl_keyvalue("__LLDPCTL_RC=0\n") == []
|
||||
|
||||
def test_a_missing_lldpctl_raises(self):
|
||||
with pytest.raises(ValueError, match="lldpctl: not found"):
|
||||
parse_lldpctl_keyvalue("sh: lldpctl: not found\n")
|
||||
|
||||
def test_a_non_zero_exit_status_raises_with_what_lldpctl_said(self):
|
||||
output = (
|
||||
"[WARN/control] unable to connect to socket /var/run/lldpd.socket: "
|
||||
"No such file or directory\n"
|
||||
"__LLDPCTL_RC=1\n"
|
||||
)
|
||||
|
||||
with pytest.raises(ValueError, match="unable to connect to socket"):
|
||||
parse_lldpctl_keyvalue(output)
|
||||
|
||||
def test_a_non_zero_exit_status_without_a_message_names_the_status(self):
|
||||
with pytest.raises(ValueError, match="exit status 127"):
|
||||
parse_lldpctl_keyvalue("__LLDPCTL_RC=127\n")
|
||||
|
||||
def test_with_status_zero_a_warning_beside_the_neighbours_is_ignored(self):
|
||||
output = (
|
||||
"lldpctl -f keyvalue 2>&1; echo __LLDPCTL_RC=$?\n"
|
||||
"[WARN/lldpctl] something lldpd wants to say\n"
|
||||
+ OPENWRT_ONE_NEIGHBOUR
|
||||
+ "__LLDPCTL_RC=0\n"
|
||||
)
|
||||
|
||||
assert [r["system_name"] for r in _rows(output)] == ["core-router"]
|
||||
|
||||
def test_a_line_outside_lldp_is_no_neighbour(self):
|
||||
output = "lldpd.config.tx-delay=30\n" + OPENWRT_ONE_NEIGHBOUR
|
||||
|
||||
assert len(_rows(output)) == 1
|
||||
|
||||
def test_a_key_without_a_known_field_is_skipped(self):
|
||||
output = "lldp.summary=1\nlldp.eth9.future-field=x\n" + OPENWRT_ONE_NEIGHBOUR
|
||||
|
||||
assert [r["local_port"] for r in _rows(output)] == ["eth0"]
|
||||
|
||||
|
||||
class TestCommand:
|
||||
def test_the_command_reads_the_neighbours_in_keyvalue_form(self):
|
||||
assert LLDPCTL_KEYVALUE_COMMAND.startswith("lldpctl -f keyvalue ")
|
||||
|
||||
def test_the_command_brings_back_the_error_text_and_the_exit_status(self):
|
||||
assert "2>&1" in LLDPCTL_KEYVALUE_COMMAND
|
||||
assert LLDPCTL_KEYVALUE_COMMAND.endswith("echo __LLDPCTL_RC=$?")
|
||||
@@ -23,6 +23,8 @@ from napalm_device_types.local_accounts import (
|
||||
from napalm_device_types.models import FixStepOutcomeDict
|
||||
|
||||
ALICE_KEY = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAlice alice@laptop"
|
||||
#: The key Alice had before, on the laptop she lost.
|
||||
OLD_KEY = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOld alice@lost-laptop"
|
||||
OWN_KEY = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQOwn own@desktop"
|
||||
HASH = "$6$salt$" + "x" * 86
|
||||
#: Shell syntax a command for root must not hold: each is one fixed command.
|
||||
@@ -61,6 +63,10 @@ class _Host:
|
||||
self.ran: list[tuple[str, bool]] = []
|
||||
self.staged = 0
|
||||
|
||||
def ran_words(self, first: str) -> list[str]:
|
||||
"""The words of the last command run whose first word is *first*."""
|
||||
return next(w for w in (shlex.split(c) for c, _ in reversed(self.ran)) if w[0] == first)
|
||||
|
||||
def is_admin(self, name: str) -> bool:
|
||||
rule = self.files.get(sudoers_rule_path(_Driver.SUDOERS_DIR, name))
|
||||
return name in self.other_rules or (self.includes_dropins and rule is not None)
|
||||
@@ -200,6 +206,11 @@ class _Host:
|
||||
self.accounts[args[0]].groups.remove(args[1])
|
||||
return CommandResult("", "", 0)
|
||||
|
||||
def _clearpw(self, args: list[str], privileged: bool) -> CommandResult:
|
||||
assert privileged
|
||||
self.accounts[args[0]].password = None
|
||||
return CommandResult("", "", 0)
|
||||
|
||||
|
||||
class _Driver(LocalAccountsMixin):
|
||||
ACCOUNT_SHELL = "/bin/sh"
|
||||
@@ -238,6 +249,9 @@ class _Driver(LocalAccountsMixin):
|
||||
def _account_leave_group(self, name: str, group: str, groups: list[str]) -> list[str]:
|
||||
return ["leavegroup", name, group]
|
||||
|
||||
def _account_clear_password(self, name: str) -> list[list[str]]:
|
||||
return [["clearpw", name]]
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def host() -> _Host:
|
||||
@@ -362,7 +376,9 @@ def test_a_new_account_gets_the_key_and_the_sudo_rule(host: _Host, driver: _Driv
|
||||
|
||||
|
||||
def test_every_command_root_runs_is_one_fixed_command(host: _Host, driver: _Driver) -> None:
|
||||
driver.add_local_account("alice", ALICE_KEY, password_hash=HASH, admin=True)
|
||||
driver.add_local_account("alice", OLD_KEY, password_hash=HASH, admin=True)
|
||||
driver.add_local_account("alice", ALICE_KEY, replaces=[OLD_KEY], clear_password=True)
|
||||
driver.remove_local_account_keys("alice", [ALICE_KEY])
|
||||
driver.lock_local_account("alice")
|
||||
|
||||
for command in _root_commands(host):
|
||||
@@ -690,3 +706,278 @@ def test_a_key_file_the_host_adds_is_read_too(host: _Host) -> None:
|
||||
[account] = _ExtraFile(host).get_local_accounts(["alice"])
|
||||
|
||||
assert account["authorized_keys"] == [OWN_KEY, ALICE_KEY]
|
||||
|
||||
|
||||
# -- replacing a key (NetOrk/netork#2013) ----------------------------------------------
|
||||
|
||||
|
||||
def _alice_keys(host: _Host) -> str:
|
||||
return host.files.get("/home/alice/.ssh/authorized_keys", "")
|
||||
|
||||
|
||||
def test_a_replaced_key_goes_and_the_persons_own_keys_stay(host: _Host, driver: _Driver) -> None:
|
||||
_add_alice(host, keys=f"{OLD_KEY}\n{OWN_KEY}\n")
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY, replaces=[OLD_KEY])
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert [s["step"] for s in result["steps"]] == ["account", "sudo", "keys", "old keys"]
|
||||
assert _alice_keys(host) == f"{OWN_KEY}\n{ALICE_KEY}\n"
|
||||
account = result["account"]
|
||||
assert account is not None and account["authorized_keys"] == [OWN_KEY, ALICE_KEY]
|
||||
|
||||
|
||||
def test_a_replaced_key_goes_from_every_key_file_with_any_options_or_comment(
|
||||
host: _Host, driver: _Driver
|
||||
) -> None:
|
||||
# The same key with options in front and another comment is the same key.
|
||||
blob = OLD_KEY.rsplit(" ", 1)[0]
|
||||
_add_alice(host, keys=f'from="10.0.0.0/8" {blob} renamed\n# the laptop\n')
|
||||
host.files["/home/alice/.ssh/authorized_keys2"] = f"{OWN_KEY}\n{blob}\n"
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY, replaces=[OLD_KEY])
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert _alice_keys(host) == f"# the laptop\n{ALICE_KEY}\n"
|
||||
assert host.files["/home/alice/.ssh/authorized_keys2"] == f"{OWN_KEY}\n"
|
||||
assert not [p for p in host.files if p.startswith("/tmp/")]
|
||||
|
||||
|
||||
def test_a_replaced_key_that_is_not_there_writes_nothing(host: _Host, driver: _Driver) -> None:
|
||||
_add_alice(host, keys=f"{ALICE_KEY}\n")
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY, replaces=[OLD_KEY])
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert host.staged == 0
|
||||
assert "none of the keys" in result["steps"][-1]["output"]
|
||||
|
||||
|
||||
def test_the_new_key_is_never_taken_for_a_replaced_one(host: _Host, driver: _Driver) -> None:
|
||||
_add_alice(host, keys=f"{OLD_KEY}\n")
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY, replaces=[OLD_KEY, ALICE_KEY])
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert _alice_keys(host) == f"{ALICE_KEY}\n"
|
||||
|
||||
|
||||
def test_without_replaces_no_key_goes(host: _Host, driver: _Driver) -> None:
|
||||
_add_alice(host, keys=f"{OLD_KEY}\n")
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY)
|
||||
|
||||
assert [s["step"] for s in result["steps"]] == ["account", "sudo", "keys"]
|
||||
assert _alice_keys(host) == f"{OLD_KEY}\n{ALICE_KEY}\n"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("replaces", [["not a key"], [OLD_KEY, ""], OLD_KEY])
|
||||
def test_a_replaced_key_that_is_none_is_refused(
|
||||
host: _Host, driver: _Driver, replaces: list[str]
|
||||
) -> None:
|
||||
with pytest.raises(ValueError, match="key"):
|
||||
driver.add_local_account("alice", ALICE_KEY, replaces=replaces)
|
||||
assert _root_commands(host) == []
|
||||
|
||||
|
||||
def test_the_old_key_goes_even_when_the_new_one_could_not_be_added(host: _Host) -> None:
|
||||
class _AddFails(_Driver):
|
||||
def _account_add_key(self, name: str, home: str, public_key: str) -> FixStepOutcomeDict:
|
||||
return {"ok": False, "output": "disk full"}
|
||||
|
||||
_add_alice(host, keys=f"{OLD_KEY}\n{OWN_KEY}\n")
|
||||
|
||||
result = _AddFails(host).add_local_account("alice", ALICE_KEY, replaces=[OLD_KEY])
|
||||
|
||||
assert not result["success"]
|
||||
assert next(s for s in result["steps"] if s["step"] == "old keys")["ok"]
|
||||
assert _alice_keys(host) == f"{OWN_KEY}\n"
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("failing", "ignored", "said"),
|
||||
[
|
||||
({"as cat"}, set(), "cannot read"),
|
||||
({"as install"}, set(), "authorized_keys"),
|
||||
(set(), {"as install"}, "still has"),
|
||||
],
|
||||
)
|
||||
def test_a_replaced_key_that_did_not_go_says_so(
|
||||
host: _Host, driver: _Driver, failing: set[str], ignored: set[str], said: str
|
||||
) -> None:
|
||||
_add_alice(host, keys=f"{OLD_KEY}\n{ALICE_KEY}\n")
|
||||
host.failing |= failing
|
||||
host.ignored |= ignored
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY, replaces=[OLD_KEY])
|
||||
|
||||
old = next(s for s in result["steps"] if s["step"] == "old keys")
|
||||
assert not old["ok"] and said in old["output"]
|
||||
assert not result["success"]
|
||||
|
||||
|
||||
# -- removing the password (NetOrk/netork#2013) ----------------------------------------
|
||||
|
||||
|
||||
def test_clearing_the_password_leaves_a_key_only_account_without_sudo(
|
||||
host: _Host, driver: _Driver
|
||||
) -> None:
|
||||
_add_alice(host, keys=f"{ALICE_KEY}\n")
|
||||
host.files["/etc/sudoers.d/80-account-alice"] = sudoers_rule("alice")
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY, clear_password=True)
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert [s["step"] for s in result["steps"]] == ["account", "sudo", "password", "keys"]
|
||||
a = host.accounts["alice"]
|
||||
assert a.password is None and not a.locked and a.groups == ["alice"]
|
||||
assert "/etc/sudoers.d/80-account-alice" not in host.files
|
||||
assert _alice_keys(host) == f"{ALICE_KEY}\n"
|
||||
|
||||
|
||||
def test_a_new_account_without_a_password_gets_it_cleared_too(host: _Host, driver: _Driver) -> None:
|
||||
result = driver.add_local_account("alice", ALICE_KEY, clear_password=True)
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert ["clearpw", "alice"] == host.ran_words("clearpw")
|
||||
|
||||
|
||||
def test_without_clear_password_the_password_stays(host: _Host, driver: _Driver) -> None:
|
||||
_add_alice(host)
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY)
|
||||
|
||||
assert "password" not in [s["step"] for s in result["steps"]]
|
||||
assert host.accounts["alice"].password == HASH
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"kwargs",
|
||||
[{"password_hash": HASH}, {"admin": True}],
|
||||
)
|
||||
def test_clearing_the_password_refuses_a_password_or_sudo(
|
||||
host: _Host, driver: _Driver, kwargs: dict[str, object]
|
||||
) -> None:
|
||||
# The sudo rule asks for the password: an account without one cannot use it.
|
||||
with pytest.raises(ValueError, match="password"):
|
||||
driver.add_local_account("alice", ALICE_KEY, clear_password=True, **kwargs) # type: ignore[arg-type]
|
||||
assert _root_commands(host) == []
|
||||
|
||||
|
||||
def test_a_password_that_cannot_be_cleared_fails_its_step_only(
|
||||
host: _Host, driver: _Driver
|
||||
) -> None:
|
||||
_add_alice(host)
|
||||
host.failing.add("clearpw")
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY, clear_password=True)
|
||||
|
||||
assert not result["success"]
|
||||
password = next(s for s in result["steps"] if s["step"] == "password")
|
||||
assert not password["ok"] and "clearpw: failed" in password["output"]
|
||||
# sudo and the key do not wait for it.
|
||||
assert next(s for s in result["steps"] if s["step"] == "sudo")["ok"]
|
||||
assert _alice_keys(host) == f"{ALICE_KEY}\n"
|
||||
|
||||
|
||||
def test_a_driver_that_cannot_clear_a_password_says_so(host: _Host) -> None:
|
||||
class _NoClear(_Driver):
|
||||
_account_clear_password = LocalAccountsMixin._account_clear_password
|
||||
|
||||
_add_alice(host)
|
||||
|
||||
result = _NoClear(host).add_local_account("alice", ALICE_KEY, clear_password=True)
|
||||
|
||||
password = next(s for s in result["steps"] if s["step"] == "password")
|
||||
assert not password["ok"] and "cannot remove" in password["output"]
|
||||
assert host.accounts["alice"].password == HASH
|
||||
|
||||
|
||||
# -- removing keys without a new one (NetOrk/netork#2013) -------------------------------
|
||||
|
||||
|
||||
def test_removing_keys_takes_only_those_and_leaves_the_account_open(
|
||||
host: _Host, driver: _Driver
|
||||
) -> None:
|
||||
_add_alice(host, keys=f"{ALICE_KEY}\n{OWN_KEY}\n")
|
||||
host.files["/etc/sudoers.d/80-account-alice"] = sudoers_rule("alice")
|
||||
|
||||
result = driver.remove_local_account_keys("alice", [ALICE_KEY])
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert [s["step"] for s in result["steps"]] == ["keys"]
|
||||
assert _alice_keys(host) == f"{OWN_KEY}\n"
|
||||
a = host.accounts["alice"]
|
||||
assert not a.locked and a.password == HASH and a.groups == ["alice", "wheel"]
|
||||
assert "/etc/sudoers.d/80-account-alice" in host.files
|
||||
account = result["account"]
|
||||
assert account is not None and account["authorized_keys"] == [OWN_KEY]
|
||||
|
||||
|
||||
def test_removing_the_last_key_leaves_an_empty_key_file(host: _Host, driver: _Driver) -> None:
|
||||
_add_alice(host, keys=f"{ALICE_KEY}\n")
|
||||
|
||||
result = driver.remove_local_account_keys("alice", [ALICE_KEY])
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert _alice_keys(host) == ""
|
||||
assert not host.accounts["alice"].locked
|
||||
|
||||
|
||||
def test_removing_keys_changes_nothing_as_root(host: _Host, driver: _Driver) -> None:
|
||||
# Root only asks whether a key file is there, and reads the account back
|
||||
# (its lock, sudo -l); the files are written as the person.
|
||||
_add_alice(host, keys=f"{ALICE_KEY}\n")
|
||||
|
||||
driver.remove_local_account_keys("alice", [ALICE_KEY])
|
||||
|
||||
for command in _root_commands(host):
|
||||
words = shlex.split(command)
|
||||
assert words[0] in {"test", "islocked", "sudo"}, command
|
||||
if words[0] == "sudo":
|
||||
assert words[:3] in (["sudo", "-n", "-u"], ["sudo", "-l", "-U"]), command
|
||||
|
||||
|
||||
def test_removing_keys_of_an_account_that_is_not_there_fails(driver: _Driver) -> None:
|
||||
result = driver.remove_local_account_keys("bob", [ALICE_KEY])
|
||||
|
||||
assert not result["success"]
|
||||
assert "no account" in result["output"]
|
||||
assert result["account"] is None
|
||||
|
||||
|
||||
@pytest.mark.parametrize("keys", [[], ["not a key"], ALICE_KEY])
|
||||
def test_removing_keys_refuses_what_is_no_list_of_keys(
|
||||
host: _Host, driver: _Driver, keys: list[str]
|
||||
) -> None:
|
||||
_add_alice(host, keys=f"{ALICE_KEY}\n")
|
||||
|
||||
with pytest.raises(ValueError, match="key"):
|
||||
driver.remove_local_account_keys("alice", keys)
|
||||
assert _alice_keys(host) == f"{ALICE_KEY}\n"
|
||||
|
||||
|
||||
@pytest.mark.parametrize(("name", "said"), [("netadmin", "logs in"), ("daemon", "system")])
|
||||
def test_removing_keys_never_touches_the_login_user_or_a_system_account(
|
||||
host: _Host, driver: _Driver, name: str, said: str
|
||||
) -> None:
|
||||
with pytest.raises(ValueError, match=said):
|
||||
driver.remove_local_account_keys(name, [ALICE_KEY])
|
||||
assert _root_commands(host) == []
|
||||
|
||||
|
||||
def test_a_driver_with_its_own_key_store_drops_keys_there(host: _Host) -> None:
|
||||
class _Store(_OwnKeyStore):
|
||||
def _account_drop_keys(self, name: str, home: str, keys: list[str]) -> FixStepOutcomeDict:
|
||||
gone = {k.split()[1] for k in keys}
|
||||
self.store[name] = [k for k in self.store.get(name, []) if k.split()[2] not in gone]
|
||||
return {"ok": True, "output": "dropped some"}
|
||||
|
||||
driver = _Store(host)
|
||||
driver.add_local_account("alice", OLD_KEY)
|
||||
driver.add_local_account("alice", ALICE_KEY, replaces=[OLD_KEY])
|
||||
assert driver.store == {"alice": [f"/home/alice {ALICE_KEY}"]}
|
||||
|
||||
removed = driver.remove_local_account_keys("alice", [ALICE_KEY])
|
||||
assert removed["success"], removed["output"]
|
||||
assert driver.store == {"alice": []}
|
||||
@@ -176,6 +176,18 @@ __DNF_RC=0
|
||||
with pytest.raises(ValueError, match="exit status 1"):
|
||||
parse_dnf_security(out)
|
||||
|
||||
def test_the_command_reads_only_the_cached_metadata(self, tmp_path):
|
||||
"""An expired cache made dnf download the repositories' metadata inside the
|
||||
poll, which outlasted its whole budget (#76)."""
|
||||
args = tmp_path / "args"
|
||||
dnf = tmp_path / "dnf"
|
||||
dnf.write_text(f"#!/bin/sh\nprintf '%s\\n' \"$@\" > {args}\n")
|
||||
dnf.chmod(0o755)
|
||||
env = {"PATH": f"{tmp_path}:{os.environ.get('PATH', '/usr/bin:/bin')}"}
|
||||
subprocess.run(["sh", "-c", DNF_SECURITY_COMMAND], capture_output=True, env=env)
|
||||
|
||||
assert "--cacheonly" in args.read_text().split()
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("nevra", "name"),
|
||||
[
|
||||
|
||||
@@ -18,6 +18,7 @@ import pytest
|
||||
|
||||
from napalm_device_types import (
|
||||
AccessPointDriver,
|
||||
BackupServerDriver,
|
||||
DeviceTypeDriver,
|
||||
FirewallDriver,
|
||||
HypervisorDriver,
|
||||
@@ -33,6 +34,7 @@ from napalm_device_types.roles import primary_role_of, roles_of
|
||||
|
||||
ROLE_BASES = [
|
||||
AccessPointDriver,
|
||||
BackupServerDriver,
|
||||
FirewallDriver,
|
||||
HypervisorDriver,
|
||||
ManagementControllerDriver,
|
||||
@@ -75,6 +77,8 @@ class TestDeclaredMethodsDoNotExistAtRuntime:
|
||||
@pytest.mark.parametrize(
|
||||
("base", "method"),
|
||||
[
|
||||
(BackupServerDriver, "get_backup_datastores"),
|
||||
(BackupServerDriver, "run_backup_job"),
|
||||
(StorageDriver, "get_disks"),
|
||||
(StorageDriver, "get_volumes"),
|
||||
(HypervisorDriver, "get_vms"),
|
||||
|
||||
@@ -0,0 +1,108 @@
|
||||
"""The scheduled-jobs contract: its digest, the names and specs it takes (NetOrk/netork#1933)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import ScheduledJobsMixin
|
||||
from napalm_device_types.scheduled_jobs import (
|
||||
ACTIONS,
|
||||
ScheduledJobChanged,
|
||||
ScheduledJobRefused,
|
||||
check_job_name,
|
||||
check_spec,
|
||||
job_digest,
|
||||
)
|
||||
|
||||
JOB = {
|
||||
"kind": "cron",
|
||||
"source": "/etc/crontab",
|
||||
"run_as": "root",
|
||||
"schedule": "*/5 * * * *",
|
||||
"command": "/usr/local/bin/backup",
|
||||
"enabled": True,
|
||||
}
|
||||
|
||||
|
||||
class TestDigest:
|
||||
def test_sixteen_hex_characters(self):
|
||||
digest = job_digest(JOB)
|
||||
assert len(digest) == 16
|
||||
int(digest, 16)
|
||||
|
||||
def test_the_same_definition_has_the_same_digest(self):
|
||||
assert job_digest(JOB) == job_digest(dict(JOB))
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"field, value",
|
||||
[
|
||||
("schedule", "0 * * * *"),
|
||||
("command", "/usr/local/bin/other"),
|
||||
("run_as", "nobody"),
|
||||
("enabled", False),
|
||||
("source", "/etc/cron.d/backup"),
|
||||
("kind", "anacron"),
|
||||
],
|
||||
)
|
||||
def test_every_part_of_the_definition_changes_it(self, field, value):
|
||||
assert job_digest({**JOB, field: value}) != job_digest(JOB)
|
||||
|
||||
def test_content_the_definition_points_at_changes_it(self):
|
||||
assert job_digest(JOB, content="12345 678") != job_digest(JOB)
|
||||
|
||||
def test_what_the_host_says_of_its_runs_does_not(self):
|
||||
assert job_digest({**JOB, "last_run": "2026-11-03T06:00:00Z"}) == job_digest(JOB)
|
||||
|
||||
|
||||
class TestNames:
|
||||
@pytest.mark.parametrize("name", ["backup", "netork-backup-db", "a", "x1"])
|
||||
def test_a_good_name(self, name):
|
||||
check_job_name(name)
|
||||
|
||||
@pytest.mark.parametrize("name", ["", "-x", "Backup", "a b", "a/b", "a.timer", "a" * 64])
|
||||
def test_a_name_a_file_or_unit_could_not_carry_safely(self, name):
|
||||
with pytest.raises(ScheduledJobRefused):
|
||||
check_job_name(name)
|
||||
|
||||
|
||||
class TestSpec:
|
||||
SPEC = {
|
||||
"name": "backup",
|
||||
"schedule": "0 3 * * *",
|
||||
"run_as": "root",
|
||||
"command": "/usr/local/bin/backup",
|
||||
"enabled": True,
|
||||
}
|
||||
|
||||
def test_a_good_spec(self):
|
||||
check_spec(self.SPEC)
|
||||
check_spec({**self.SPEC, "schedule": "@reboot"})
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"field, value",
|
||||
[
|
||||
("schedule", "every day"),
|
||||
("run_as", "root; rm"),
|
||||
("run_as", ""),
|
||||
("command", ""),
|
||||
("command", " "),
|
||||
("description", "two\nlines"),
|
||||
],
|
||||
)
|
||||
def test_a_spec_that_cannot_be_written(self, field, value):
|
||||
with pytest.raises(ScheduledJobRefused):
|
||||
check_spec({**self.SPEC, field: value})
|
||||
|
||||
|
||||
class TestContract:
|
||||
def test_the_mixin_declares_nothing_at_runtime(self):
|
||||
driver = ScheduledJobsMixin()
|
||||
for method in ("get_scheduled_jobs", "manage_scheduled_job", "put_scheduled_job"):
|
||||
assert not hasattr(driver, method)
|
||||
|
||||
def test_the_actions(self):
|
||||
assert ACTIONS == ("enable", "disable", "run")
|
||||
|
||||
def test_a_refusal_is_a_value_error_and_a_change_is_not(self):
|
||||
assert issubclass(ScheduledJobRefused, ValueError)
|
||||
assert not issubclass(ScheduledJobChanged, ValueError)
|
||||
@@ -0,0 +1,540 @@
|
||||
"""Scheduled jobs on a POSIX host: the report as root, switching, running, an application's jobs.
|
||||
|
||||
The reading command is run by a real ``sh`` over a directory tree that stands in
|
||||
for the host, with ``systemctl``, ``crontab``, ``dpkg``, ``install`` and
|
||||
``journalctl`` replaced by scripts that keep a log -- so the shell itself is what
|
||||
is tested, not a hand-written copy of its output (NetOrk/netork#1933).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import PosixScheduledJobsMixin
|
||||
from napalm_device_types._scheduled_reading import parse_reading
|
||||
from napalm_device_types.channel import CommandResult
|
||||
from napalm_device_types.scheduled_jobs import ScheduledJobChanged, ScheduledJobRefused
|
||||
|
||||
TIMERS = """\
|
||||
Id=apt-daily.timer
|
||||
Unit=apt-daily.service
|
||||
Description=Daily apt download activities
|
||||
TimersCalendar={ OnCalendar=*-*-* 06,18:00:00 ; next_elapse=@1762192800 }
|
||||
NextElapseUSecRealtime=@1762192800
|
||||
LastTriggerUSec=@1762149600
|
||||
UnitFileState=enabled
|
||||
ActiveState=active
|
||||
FragmentPath=/usr/lib/systemd/system/apt-daily.timer
|
||||
|
||||
Id=fstrim.timer
|
||||
Unit=fstrim.service
|
||||
Description=Discard unused blocks once a week
|
||||
TimersMonotonic={ OnUnitActiveUSec=1w ; next_elapse=n/a }
|
||||
NextElapseUSecRealtime=n/a
|
||||
LastTriggerUSec=n/a
|
||||
UnitFileState=static
|
||||
ActiveState=inactive
|
||||
FragmentPath=/etc/systemd/system/fstrim.timer
|
||||
|
||||
Id=backup.timer
|
||||
Unit=backup.service
|
||||
Description=netork backup
|
||||
TimersCalendar={ OnCalendar=*-*-* 03:00:00 ; next_elapse=n/a }
|
||||
NextElapseUSecRealtime=Tue 2026-11-04 03:00:00 UTC
|
||||
LastTriggerUSec=n/a
|
||||
UnitFileState=enabled
|
||||
ActiveState=active
|
||||
FragmentPath=/etc/systemd/system/backup.timer
|
||||
"""
|
||||
|
||||
UNITS = """\
|
||||
Id=apt-daily.service
|
||||
ExecStart={ path=/usr/lib/apt/apt.systemd.daily ; argv[]=/usr/lib/apt/apt.systemd.daily update ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }
|
||||
User=
|
||||
Result=success
|
||||
FragmentPath=/usr/lib/systemd/system/apt-daily.service
|
||||
|
||||
Id=fstrim.service
|
||||
ExecStart={ path=/sbin/fstrim ; argv[]=/sbin/fstrim --listed-in /etc/fstab ; ignore_errors=no }
|
||||
Result=success
|
||||
FragmentPath=/usr/lib/systemd/system/fstrim.service
|
||||
|
||||
Id=backup.service
|
||||
ExecStart={ path=SCRIPTS/backup ; argv[]=SCRIPTS/backup ; ignore_errors=no }
|
||||
User=backup
|
||||
Result=exit-code
|
||||
FragmentPath=/etc/systemd/system/backup.service
|
||||
"""
|
||||
|
||||
SYSTEMCTL = """#!/bin/sh
|
||||
echo "systemctl $*" >> "$LOG"
|
||||
case "$1" in
|
||||
list-unit-files) printf 'apt-daily.timer enabled enabled\\nfstrim.timer static -\\nbackup.timer enabled enabled\\ngetty@.timer static -\\n' ;;
|
||||
show) case "$*" in *Unit,Description*) sed "s#SCRIPTS#$SCRIPTS#g" "$FIX/timers" ;; *) sed "s#SCRIPTS#$SCRIPTS#g" "$FIX/units" ;; esac ;;
|
||||
esac
|
||||
"""
|
||||
|
||||
CRONTAB = """#!/bin/sh
|
||||
echo "crontab $*" >> "$LOG"
|
||||
[ "$1" = -u ] || exit 2
|
||||
if [ "$3" = -l ]; then
|
||||
[ -f "$SPOOL/$2" ] || { echo "no crontab for $2" >&2; exit 1; }
|
||||
cat "$SPOOL/$2"
|
||||
else
|
||||
cp "$3" "$SPOOL/$2"
|
||||
fi
|
||||
"""
|
||||
|
||||
DPKG = """#!/bin/sh
|
||||
for p in "$@"; do case " $OWNED " in *" $p "*) echo "cron: $p" ;; esac; done
|
||||
"""
|
||||
|
||||
INSTALL = """#!/bin/sh
|
||||
echo "install $*" >> "$LOG"
|
||||
args=""
|
||||
while [ $# -gt 0 ]; do case "$1" in -o|-g) shift 2 ;; *) args="$args '$1'"; shift ;; esac; done
|
||||
eval "/usr/bin/install $args"
|
||||
"""
|
||||
|
||||
ECHO_LOG = """#!/bin/sh
|
||||
echo "$(basename "$0") $*" >> "$LOG"
|
||||
[ "$(basename "$0")" = journalctl ] && echo "backup ran fine"
|
||||
exit 0
|
||||
"""
|
||||
|
||||
|
||||
class Host(PosixScheduledJobsMixin):
|
||||
"""A host in a directory; every command is run by sh, root or not."""
|
||||
|
||||
def __init__(self, root: Path, *, systemd: bool = True, own_cron_d: bool = True) -> None:
|
||||
self.root = root
|
||||
etc = root / "etc"
|
||||
self.CRON_SYSTEM_FILES = (str(etc / "crontab"),) # type: ignore[misc]
|
||||
self.CRON_D_DIRS = (str(etc / "cron.d"),) # type: ignore[misc]
|
||||
self.CRON_SPOOL_DIRS = (str(root / "spool"),) # type: ignore[misc]
|
||||
self.RUN_PARTS_DIRS = (("daily", str(etc / "cron.daily")),) # type: ignore[misc]
|
||||
self.ANACRONTAB = str(etc / "anacrontab") # type: ignore[misc]
|
||||
self.JOB_SCRIPT_DIR = str(root / "scripts") # type: ignore[misc]
|
||||
self.UNIT_DIR = str(root / "units") # type: ignore[misc]
|
||||
self.OWN_CRON_D_DIR = str(etc / "cron.d") if own_cron_d else None # type: ignore[misc]
|
||||
self.ROOT_GROUP = subprocess.run(
|
||||
["id", "-gn"], capture_output=True, text=True
|
||||
).stdout.strip() # type: ignore[misc]
|
||||
self.systemd_dir = root / ("run-systemd" if systemd else "no-systemd")
|
||||
self.commands: list[str] = []
|
||||
|
||||
def _sj_paths(self): # type: ignore[no-untyped-def]
|
||||
paths = super()._sj_paths()
|
||||
from dataclasses import replace
|
||||
|
||||
return replace(paths, systemd_marker=str(self.systemd_dir))
|
||||
|
||||
def run_command(self, command, *, privileged=False, timeout=60, stdin=None): # type: ignore[no-untyped-def]
|
||||
self.commands.append(command)
|
||||
env = {
|
||||
**os.environ,
|
||||
"PATH": f"{self.root / 'bin'}:{os.environ['PATH']}",
|
||||
"LOG": str(self.root / "log"),
|
||||
"SPOOL": str(self.root / "spool"),
|
||||
"FIX": str(self.root / "fix"),
|
||||
"SCRIPTS": str(self.root / "scripts"),
|
||||
"OWNED": str(self.root / "etc" / "crontab"),
|
||||
}
|
||||
done = subprocess.run(
|
||||
["sh", "-c", command], input=stdin, capture_output=True, env=env, timeout=30
|
||||
)
|
||||
return CommandResult(done.stdout.decode(), done.stderr.decode(), done.returncode)
|
||||
|
||||
def log(self) -> str:
|
||||
log = self.root / "log"
|
||||
return log.read_text() if log.exists() else ""
|
||||
|
||||
|
||||
def _tool(root: Path, name: str, body: str) -> None:
|
||||
path = root / "bin" / name
|
||||
path.write_text(body)
|
||||
path.chmod(0o755)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def root(tmp_path: Path) -> Path:
|
||||
for d in (
|
||||
"etc/cron.d",
|
||||
"etc/cron.daily",
|
||||
"spool",
|
||||
"scripts",
|
||||
"units",
|
||||
"bin",
|
||||
"fix",
|
||||
"run-systemd",
|
||||
):
|
||||
(tmp_path / d).mkdir(parents=True)
|
||||
(tmp_path / "etc/crontab").write_text(
|
||||
"SHELL=/bin/sh\nOUT="
|
||||
+ str(tmp_path / "out")
|
||||
+ "\n17 * * * * root cd / && run-parts /etc/cron.hourly\n"
|
||||
'*/5 * * * * root echo "$OUT" > ' + str(tmp_path / "ran") + "\n"
|
||||
)
|
||||
(tmp_path / "etc/cron.d/certbot").write_text("0 */12 * * * root certbot -q renew\n")
|
||||
(tmp_path / "etc/anacrontab").write_text("1 5 cron.daily run-parts /etc/cron.daily\n")
|
||||
(tmp_path / "spool/www-data").write_text("*/5 * * * * php /var/www/cron.php\n")
|
||||
for name, mode in (("logrotate", 0o755), ("off", 0o644), ("x.dpkg-old", 0o755)):
|
||||
script = tmp_path / "etc/cron.daily" / name
|
||||
script.write_text("#!/bin/sh\necho rotated\n")
|
||||
script.chmod(mode)
|
||||
(tmp_path / "scripts/backup").write_text(
|
||||
"#!/bin/sh\n# napalm-scheduled-job: backup nightly backup\n# schedule: 0 3 * * *\n/usr/local/bin/backup --all\n"
|
||||
)
|
||||
(tmp_path / "fix/timers").write_text(TIMERS)
|
||||
(tmp_path / "fix/units").write_text(UNITS)
|
||||
for name, body in (
|
||||
("systemctl", SYSTEMCTL),
|
||||
("crontab", CRONTAB),
|
||||
("dpkg", DPKG),
|
||||
("install", INSTALL),
|
||||
("journalctl", ECHO_LOG),
|
||||
("systemd-analyze", ECHO_LOG),
|
||||
):
|
||||
_tool(tmp_path, name, body)
|
||||
return tmp_path
|
||||
|
||||
|
||||
def _jobs(host: Host) -> dict[str, dict]:
|
||||
return {j["key"]: j for j in host.get_scheduled_jobs()["jobs"]}
|
||||
|
||||
|
||||
class TestReading:
|
||||
def test_every_source_is_read(self, root: Path):
|
||||
jobs = _jobs(Host(root))
|
||||
kinds = sorted({j["kind"] for j in jobs.values()})
|
||||
assert kinds == ["anacron", "cron", "run_parts", "systemd_timer"]
|
||||
users = {j["run_as"] for j in jobs.values() if j["kind"] == "cron"}
|
||||
assert users == {"root", "www-data"}
|
||||
|
||||
def test_a_package_owns_what_it_installed(self, root: Path):
|
||||
jobs = _jobs(Host(root))
|
||||
crontab = [j for j in jobs.values() if j["source"] == str(root / "etc/crontab")]
|
||||
assert {j["managed_by"] for j in crontab} == {"package"}
|
||||
certbot = next(j for j in jobs.values() if j["source"].endswith("certbot"))
|
||||
assert certbot["managed_by"] == "admin"
|
||||
|
||||
def test_run_parts_by_execute_bit_and_the_names_run_parts_runs(self, root: Path):
|
||||
jobs = _jobs(Host(root))
|
||||
daily = root / "etc/cron.daily"
|
||||
assert jobs[f"run-parts:{daily}/logrotate"]["enabled"]
|
||||
assert not jobs[f"run-parts:{daily}/off"]["enabled"]
|
||||
assert f"run-parts:{daily}/x.dpkg-old" not in jobs
|
||||
|
||||
def test_a_changed_script_changes_its_digest(self, root: Path):
|
||||
host = Host(root)
|
||||
key = f"run-parts:{root}/etc/cron.daily/logrotate"
|
||||
before = _jobs(host)[key]["digest"]
|
||||
(root / "etc/cron.daily/logrotate").write_text("#!/bin/sh\necho something else\n")
|
||||
assert _jobs(host)[key]["digest"] != before
|
||||
|
||||
def test_timers_with_their_unit_and_runs(self, root: Path):
|
||||
apt = _jobs(Host(root))["timer:apt-daily.timer"]
|
||||
assert apt["schedule"] == "OnCalendar=*-*-* 06,18:00:00"
|
||||
assert apt["schedule_kind"] == "calendar"
|
||||
assert apt["command"] == "/usr/lib/apt/apt.systemd.daily update"
|
||||
assert (apt["run_as"], apt["managed_by"], apt["enabled"]) == ("root", "package", True)
|
||||
assert apt["last_run"] == "2025-11-03T06:00:00+00:00"
|
||||
assert apt["next_run"] == "2025-11-03T18:00:00+00:00"
|
||||
assert apt["last_result"] == "success"
|
||||
assert apt["actions"] == ["enable", "disable", "run"]
|
||||
|
||||
def test_a_static_timer_only_runs(self, root: Path):
|
||||
fstrim = _jobs(Host(root))["timer:fstrim.timer"]
|
||||
assert fstrim["schedule"] == "OnUnitActiveSec=1w"
|
||||
assert fstrim["schedule_kind"] == "monotonic"
|
||||
assert fstrim["actions"] == ["run"]
|
||||
assert fstrim.get("last_run") is None
|
||||
assert fstrim["managed_by"] == "admin"
|
||||
|
||||
def test_an_applications_timer_by_its_script(self, root: Path):
|
||||
backup = _jobs(Host(root))["job:backup"]
|
||||
assert backup["managed_by"] == "application"
|
||||
assert (backup["name"], backup["description"]) == ("backup", "nightly backup")
|
||||
assert (backup["schedule"], backup["schedule_kind"]) == ("0 3 * * *", "cron")
|
||||
assert backup["command"] == "/usr/local/bin/backup --all"
|
||||
assert backup["run_as"] == "backup"
|
||||
assert "edit" in backup["actions"] and "delete" in backup["actions"]
|
||||
assert backup["next_run"] == "2026-11-04T03:00:00+00:00"
|
||||
assert "last_result" not in backup
|
||||
|
||||
def test_a_file_without_a_last_newline_does_not_swallow_the_next(self, root: Path):
|
||||
(root / "etc/cron.d/certbot").write_text("0 */12 * * * root certbot -q renew")
|
||||
jobs = _jobs(Host(root))
|
||||
assert any(j["source"].endswith("certbot") for j in jobs.values())
|
||||
assert any(j["source"].endswith("spool/www-data") for j in jobs.values())
|
||||
|
||||
def test_without_systemd_no_timers(self, root: Path):
|
||||
jobs = _jobs(Host(root, systemd=False))
|
||||
assert not [j for j in jobs.values() if j["kind"] == "systemd_timer"]
|
||||
|
||||
def test_a_report_cut_short_raises(self, root: Path):
|
||||
host = Host(root)
|
||||
full = host.run_command
|
||||
host.run_command = lambda c, **kw: full(c, **kw)._replace(
|
||||
stdout=full(c, **kw).stdout.split("SJ_END")[0][:-3]
|
||||
) # type: ignore[method-assign]
|
||||
with pytest.raises(ValueError, match="cut short"):
|
||||
host.get_scheduled_jobs()
|
||||
|
||||
|
||||
class TestDriverSettings:
|
||||
def test_what_the_platform_generates(self, root: Path):
|
||||
host = Host(root)
|
||||
host.PLATFORM_SOURCES = ("*/cron.d/cert*",) # type: ignore[misc]
|
||||
certbot = next(j for j in _jobs(host).values() if j["source"].endswith("certbot"))
|
||||
assert certbot["managed_by"] == "platform"
|
||||
|
||||
def test_the_time_zone_and_owners_commands_are_the_drivers(self, root: Path):
|
||||
host = Host(root)
|
||||
host.TIME_ZONE_SH = "echo America/New_York" # type: ignore[misc]
|
||||
host.OWNERS_SH = "cat >/dev/null" # type: ignore[misc]
|
||||
reading = host.get_scheduled_jobs()
|
||||
assert reading["time_zone"] == "America/New_York"
|
||||
assert {j["managed_by"] for j in reading["jobs"] if j["kind"] == "cron"} == {"admin"}
|
||||
|
||||
|
||||
class TestParseReading:
|
||||
def test_the_time_zone_from_either_file(self):
|
||||
out = "SJ_BEGIN\n[tz]\n/usr/share/zoneinfo/Europe/Berlin\nSJ_END\n"
|
||||
assert parse_reading(out).time_zone == "Europe/Berlin"
|
||||
assert parse_reading("SJ_BEGIN\n[tz]\nEtc/UTC\nSJ_END").time_zone == "Etc/UTC"
|
||||
assert parse_reading("SJ_BEGIN\n[tz]\nSJ_END").time_zone is None
|
||||
|
||||
def test_owners_skip_diversions(self):
|
||||
out = "SJ_BEGIN\n[owners]\ndiversion by x from: /etc/crontab\ncron, cron-daemon: /etc/cron.d/x\nSJ_END"
|
||||
reading = parse_reading(out + "")
|
||||
assert reading.jobs == []
|
||||
|
||||
def test_a_time_not_in_utc_is_left_unknown(self):
|
||||
from napalm_device_types._scheduled_reading import _timestamp
|
||||
|
||||
assert _timestamp("Tue 2026-11-04 03:00:00 CET") is None
|
||||
assert _timestamp("@0") is None
|
||||
|
||||
|
||||
class TestSwitching:
|
||||
def _cron(self, host: Host, path: str) -> dict:
|
||||
return next(j for j in _jobs(host).values() if j["source"] == path and j["kind"] == "cron")
|
||||
|
||||
def test_a_system_line_switched_off_and_on(self, root: Path):
|
||||
host = Host(root)
|
||||
path = str(root / "etc/cron.d/certbot")
|
||||
job = self._cron(host, path)
|
||||
mode = os.stat(path).st_mode
|
||||
assert host.manage_scheduled_job(job["key"], "disable", digest=job["digest"])["success"]
|
||||
assert (
|
||||
root / "etc/cron.d/certbot"
|
||||
).read_text() == "#napalm-disabled: 0 */12 * * * root certbot -q renew\n"
|
||||
off = _jobs(host)[job["key"]]
|
||||
assert not off["enabled"]
|
||||
host.manage_scheduled_job(job["key"], "enable", digest=off["digest"])
|
||||
assert (root / "etc/cron.d/certbot").read_text() == "0 */12 * * * root certbot -q renew\n"
|
||||
assert f"install -D -m {mode & 0o777:o}" in host.log()
|
||||
assert os.stat(path).st_mode == mode
|
||||
|
||||
def test_a_linked_file_is_written_through_its_link(self, root: Path):
|
||||
real = root / "etc/vzdump.cron"
|
||||
real.write_text("0 1 * * * root vzdump --all\n")
|
||||
real.chmod(0o640)
|
||||
link = root / "etc/cron.d/vzdump"
|
||||
link.symlink_to(real)
|
||||
host = Host(root)
|
||||
job = self._cron(host, str(link))
|
||||
host.manage_scheduled_job(job["key"], "disable", digest=job["digest"])
|
||||
assert link.is_symlink()
|
||||
assert real.read_text() == "#napalm-disabled: 0 1 * * * root vzdump --all\n"
|
||||
assert real.stat().st_mode & 0o777 == 0o640
|
||||
|
||||
def test_a_users_line_goes_through_crontab(self, root: Path):
|
||||
host = Host(root)
|
||||
job = self._cron(host, str(root / "spool/www-data"))
|
||||
host.manage_scheduled_job(job["key"], "disable", digest=job["digest"])
|
||||
assert (root / "spool/www-data").read_text().startswith("#napalm-disabled: */5")
|
||||
assert "crontab -u www-data /" in host.log()
|
||||
|
||||
def test_a_change_it_did_not_see_is_refused(self, root: Path):
|
||||
host = Host(root)
|
||||
job = self._cron(host, str(root / "etc/cron.d/certbot"))
|
||||
with pytest.raises(ScheduledJobChanged):
|
||||
host.manage_scheduled_job(job["key"], "disable", digest="0" * 16)
|
||||
|
||||
def test_an_unknown_job_and_an_unknown_action(self, root: Path):
|
||||
host = Host(root)
|
||||
with pytest.raises(LookupError):
|
||||
host.manage_scheduled_job("cron:/nowhere:1", "disable", digest="x")
|
||||
with pytest.raises(ScheduledJobRefused):
|
||||
host.manage_scheduled_job("cron:/nowhere:1", "explode", digest="x")
|
||||
|
||||
def test_a_static_timer_is_not_disabled(self, root: Path):
|
||||
host = Host(root)
|
||||
fstrim = _jobs(host)["timer:fstrim.timer"]
|
||||
with pytest.raises(ScheduledJobRefused):
|
||||
host.manage_scheduled_job("timer:fstrim.timer", "disable", digest=fstrim["digest"])
|
||||
|
||||
def test_a_timer_and_a_run_parts_script(self, root: Path):
|
||||
host = Host(root)
|
||||
jobs = _jobs(host)
|
||||
apt = jobs["timer:apt-daily.timer"]
|
||||
host.manage_scheduled_job(apt["key"], "disable", digest=apt["digest"])
|
||||
assert "systemctl disable --now -- apt-daily.timer" in host.log()
|
||||
key = f"run-parts:{root}/etc/cron.daily/off"
|
||||
host.manage_scheduled_job(key, "enable", digest=jobs[key]["digest"])
|
||||
assert os.access(root / "etc/cron.daily/off", os.X_OK)
|
||||
|
||||
def test_an_applications_timer_is_switched_by_its_name(self, root: Path):
|
||||
host = Host(root)
|
||||
backup = _jobs(host)["job:backup"]
|
||||
host.manage_scheduled_job("job:backup", "disable", digest=backup["digest"])
|
||||
assert "systemctl disable --now -- backup.timer" in host.log()
|
||||
|
||||
|
||||
class TestRunning:
|
||||
def test_a_cron_line_runs_as_cron_would(self, root: Path):
|
||||
host = Host(root)
|
||||
job = next(j for j in _jobs(host).values() if "ran" in j["command"])
|
||||
result = host.manage_scheduled_job(job["key"], "run", digest=job["digest"])
|
||||
assert result["success"]
|
||||
assert (root / "ran").read_text() == f"{root / 'out'}\n"
|
||||
assert "timeout 300 sh -c" in host.commands[-1]
|
||||
|
||||
def test_a_timer_runs_its_unit_and_says_what_its_journal_said(self, root: Path):
|
||||
host = Host(root)
|
||||
backup = _jobs(host)["job:backup"]
|
||||
result = host.manage_scheduled_job("job:backup", "run", digest=backup["digest"])
|
||||
assert result == {"success": True, "output": "backup ran fine"}
|
||||
assert "systemctl start -- backup.service" in host.log()
|
||||
|
||||
def test_a_run_parts_script_runs(self, root: Path):
|
||||
host = Host(root)
|
||||
key = f"run-parts:{root}/etc/cron.daily/logrotate"
|
||||
result = host.manage_scheduled_job(key, "run", digest=_jobs(host)[key]["digest"])
|
||||
assert result == {"success": True, "output": "rotated"}
|
||||
|
||||
def test_another_user_runs_through_su(self):
|
||||
assert Host(Path("/x"))._sj_as("www-data", "true") == [
|
||||
"su",
|
||||
"-s",
|
||||
"/bin/sh",
|
||||
"www-data",
|
||||
"-c",
|
||||
"true",
|
||||
]
|
||||
|
||||
|
||||
SPEC = {
|
||||
"name": "netork-tidy",
|
||||
"schedule": "*/30 1-2 * * *",
|
||||
"run_as": "root",
|
||||
"command": "find /tmp -mtime +7 -delete",
|
||||
"enabled": True,
|
||||
"description": "tidy %tmp",
|
||||
}
|
||||
|
||||
|
||||
class TestApplicationJobs:
|
||||
def test_a_timer_on_a_host_with_systemd(self, root: Path):
|
||||
host = Host(root)
|
||||
assert host.put_scheduled_job(SPEC)["output"] == "job:netork-tidy created"
|
||||
timer = (root / "units/netork-tidy.timer").read_text()
|
||||
assert "OnCalendar=*-*-* 01,02:00,30:00" in timer
|
||||
assert "Description=tidy %%tmp" in timer
|
||||
service = (root / "units/netork-tidy.service").read_text()
|
||||
assert f"ExecStart={root}/scripts/netork-tidy" in service and "User=root" in service
|
||||
script = (root / "scripts/netork-tidy").read_text()
|
||||
assert script.splitlines()[1:] == [
|
||||
"# napalm-scheduled-job: netork-tidy tidy %tmp",
|
||||
"# schedule: */30 1-2 * * *",
|
||||
"find /tmp -mtime +7 -delete",
|
||||
]
|
||||
assert "systemctl daemon-reload" in host.log()
|
||||
assert "systemctl enable --now -- netork-tidy.timer" in host.log()
|
||||
|
||||
def test_reboot_is_a_boot_timer(self, root: Path):
|
||||
Host(root).put_scheduled_job({**SPEC, "schedule": "@reboot", "enabled": False})
|
||||
assert "OnBootSec=1min" in (root / "units/netork-tidy.timer").read_text()
|
||||
|
||||
def test_creating_one_that_is_there_and_changing_one_that_is_not(self, root: Path):
|
||||
host = Host(root)
|
||||
with pytest.raises(ScheduledJobChanged):
|
||||
host.put_scheduled_job({**SPEC, "name": "backup"})
|
||||
with pytest.raises(LookupError):
|
||||
host.put_scheduled_job(SPEC, digest="x")
|
||||
with pytest.raises(ScheduledJobChanged):
|
||||
host.put_scheduled_job({**SPEC, "name": "backup"}, digest="x")
|
||||
|
||||
def test_an_unknown_user_is_refused(self, root: Path):
|
||||
with pytest.raises(ScheduledJobRefused, match="no user"):
|
||||
Host(root).put_scheduled_job({**SPEC, "run_as": "no-such-user-here"})
|
||||
|
||||
def test_a_timer_deleted(self, root: Path):
|
||||
host = Host(root)
|
||||
backup = _jobs(host)["job:backup"]
|
||||
host.delete_scheduled_job("job:backup", digest=backup["digest"])
|
||||
assert not (root / "scripts/backup").exists()
|
||||
assert "systemctl disable --now -- backup.timer" in host.log()
|
||||
|
||||
def test_only_an_applications_job_is_deleted(self, root: Path):
|
||||
host = Host(root)
|
||||
apt = _jobs(host)["timer:apt-daily.timer"]
|
||||
with pytest.raises(ScheduledJobRefused):
|
||||
host.delete_scheduled_job(apt["key"], digest=apt["digest"])
|
||||
|
||||
def test_a_cron_d_file_without_systemd(self, root: Path):
|
||||
host = Host(root, systemd=False)
|
||||
host.put_scheduled_job(SPEC)
|
||||
assert (root / "etc/cron.d/netork-tidy").read_text() == (
|
||||
"# napalm-scheduled-job: netork-tidy tidy %tmp\n*/30 1-2 * * * root find /tmp -mtime +7 -delete\n"
|
||||
)
|
||||
job = _jobs(host)["job:netork-tidy"]
|
||||
assert job["managed_by"] == "application" and job["kind"] == "cron"
|
||||
host.delete_scheduled_job("job:netork-tidy", digest=job["digest"])
|
||||
assert not (root / "etc/cron.d/netork-tidy").exists()
|
||||
|
||||
def test_marked_lines_in_the_users_crontab_on_a_busybox_host(self, root: Path):
|
||||
host = Host(root, systemd=False, own_cron_d=False)
|
||||
host.put_scheduled_job({**SPEC, "run_as": "www-data"})
|
||||
text = (root / "spool/www-data").read_text()
|
||||
assert text.endswith(
|
||||
"# napalm-scheduled-job: netork-tidy tidy %tmp\n*/30 1-2 * * * find /tmp -mtime +7 -delete\n"
|
||||
)
|
||||
job = _jobs(host)["job:netork-tidy"]
|
||||
host.put_scheduled_job(
|
||||
{**SPEC, "run_as": "www-data", "enabled": False}, digest=job["digest"]
|
||||
)
|
||||
assert "#napalm-disabled: */30" in (root / "spool/www-data").read_text()
|
||||
job = _jobs(host)["job:netork-tidy"]
|
||||
host.put_scheduled_job({**SPEC, "run_as": "nobody"}, digest=job["digest"])
|
||||
assert "netork-tidy" not in (root / "spool/www-data").read_text()
|
||||
assert "netork-tidy" in (root / "spool/nobody").read_text()
|
||||
job = _jobs(host)["job:netork-tidy"]
|
||||
host.delete_scheduled_job("job:netork-tidy", digest=job["digest"])
|
||||
assert (root / "spool/nobody").read_text() == ""
|
||||
|
||||
def test_a_cron_d_that_is_not_there_is_the_crontab(self, root: Path):
|
||||
host = Host(root, systemd=False)
|
||||
host.OWN_CRON_D_DIR = str(root / "etc/no-cron.d") # type: ignore[misc]
|
||||
host.put_scheduled_job(SPEC)
|
||||
assert "netork-tidy" in (root / "spool/root").read_text()
|
||||
|
||||
def test_a_cron_line_holds_one_line(self, root: Path):
|
||||
with pytest.raises(ScheduledJobRefused, match="one line"):
|
||||
Host(root, systemd=False).put_scheduled_job({**SPEC, "command": "a\nb"})
|
||||
|
||||
def test_a_calendar_systemd_refuses(self, root: Path):
|
||||
_tool(root, "systemd-analyze", "#!/bin/sh\necho 'Failed to parse' >&2\nexit 1\n")
|
||||
with pytest.raises(ScheduledJobRefused, match="systemd does not take"):
|
||||
Host(root).put_scheduled_job(SPEC)
|
||||
|
||||
def test_a_failed_install_says_so(self, root: Path):
|
||||
_tool(root, "install", "#!/bin/sh\necho 'read-only file system' >&2\nexit 1\n")
|
||||
with pytest.raises(RuntimeError, match="read-only"):
|
||||
Host(root).put_scheduled_job(SPEC)
|
||||
@@ -14,6 +14,7 @@ import pytest
|
||||
|
||||
from napalm_device_types.channel import CommandResult
|
||||
from napalm_device_types.ssh_server import (
|
||||
DROPIN_DIR,
|
||||
DROPIN_PATH,
|
||||
MAIN_CONFIG,
|
||||
OpenSshServerMixin,
|
||||
@@ -401,6 +402,32 @@ def test_mixin_falls_back_to_sshd_config_without_an_include():
|
||||
assert host._effective()["mac"] == ["hmac-sha2-256"]
|
||||
|
||||
|
||||
class GeneratedConfigHost(FakeHost):
|
||||
"""A platform that writes sshd_config from its own database (OpenMediaVault)."""
|
||||
|
||||
SSHD_CONFIG_GENERATED = True
|
||||
|
||||
|
||||
def test_generated_sshd_config_still_gets_the_dropin():
|
||||
host = GeneratedConfigHost()
|
||||
result = host.harden_ssh_server({"mac": ["hmac-sha1"]})
|
||||
assert result.applied
|
||||
assert result.path == DROPIN_PATH
|
||||
assert host.files[MAIN_CONFIG] == "Include /etc/ssh/sshd_config.d/*.conf\nPort 22\n"
|
||||
|
||||
|
||||
def test_generated_sshd_config_without_an_include_is_left_alone():
|
||||
"""The platform would write its own file over the change on its next apply."""
|
||||
host = GeneratedConfigHost(main="Port 22\nMACs hmac-sha1,hmac-sha2-256\n")
|
||||
result = host.harden_ssh_server({"mac": ["hmac-sha1"]})
|
||||
assert not result.applied and not result.rolled_back
|
||||
assert result.path is None
|
||||
assert host.files[MAIN_CONFIG] == "Port 22\nMACs hmac-sha1,hmac-sha2-256\n"
|
||||
assert list(host.files) == [MAIN_CONFIG]
|
||||
assert host.reloads == 0
|
||||
assert DROPIN_DIR in result.message
|
||||
|
||||
|
||||
def test_mixin_restores_when_sshd_rejects_the_config():
|
||||
host = FakeHost()
|
||||
host.syntax_ok = False
|
||||
|
||||
@@ -4,6 +4,7 @@ import pytest
|
||||
|
||||
from napalm_device_types import (
|
||||
AccessPointDriver,
|
||||
BackupServerDriver,
|
||||
FirewallDriver,
|
||||
HypervisorDriver,
|
||||
ManagementControllerDriver,
|
||||
@@ -15,6 +16,7 @@ from napalm_device_types import (
|
||||
|
||||
BASE_CLASSES = [
|
||||
(AccessPointDriver, "Access Point"),
|
||||
(BackupServerDriver, "Backup Server"),
|
||||
(FirewallDriver, "Firewall"),
|
||||
(HypervisorDriver, "Hypervisor"),
|
||||
(ManagementControllerDriver, "Management Controller"),
|
||||
|
||||
Reference in new issue
Block a user