Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
4e40f95bc1 | ||
|
|
70e1c9b218 | ||
|
|
a70501ff60 | ||
|
|
c6d119eb83 | ||
|
|
8e9d21c368 | ||
|
|
25f28297e4 | ||
|
|
847c902193 | ||
|
|
ffbb194300 | ||
|
|
edef6cf07c | ||
|
|
df476a12f2 | ||
|
|
834cb2e0ea | ||
|
|
f573fa53d2 | ||
|
|
fdf2f89115 | ||
|
|
d63abff467 | ||
|
|
9d6a1a65ed | ||
|
|
95e3aa1e26 | ||
|
|
a0901e3277 | ||
|
|
060e210130 | ||
|
|
31973d2d5f | ||
|
|
abda0973e7 | ||
|
|
4eeaaef236 | ||
|
|
3490d8b62a | ||
|
|
2542081c6f | ||
|
|
4427be35cc | ||
|
|
ad9ec0fb4d | ||
|
|
8dd7e8ab9a | ||
|
|
935fad2a2d | ||
|
|
f337ae7fb8 | ||
|
|
e2aebeadeb | ||
|
|
36cf117b95 | ||
|
|
02d461036b | ||
|
|
63039ce758 | ||
|
|
516fb18c0e | ||
|
|
8d32e70cd7 | ||
|
|
51e8c849bb | ||
|
|
6aadcc3d91 | ||
|
|
e98d4763f3 | ||
|
|
24f4be3b7e | ||
|
|
f4354f498e | ||
|
|
73771aba00 | ||
|
|
c9571d59f1 | ||
|
|
cf862af026 | ||
|
|
83ea13f88e | ||
|
|
4938056d20 | ||
|
|
9c3be46d8a | ||
|
|
a6289cb24a | ||
|
|
46c1c9dc4e | ||
|
|
bc17862a44 | ||
|
|
9f6ffa8a25 | ||
|
|
4150570a94 | ||
|
|
a809e6c02f | ||
|
|
5b894913e1 | ||
|
|
00aa5efef0 | ||
|
|
97cb8779c5 | ||
|
|
47751d12d6 | ||
|
|
3f59023f46 | ||
|
|
666be091e3 | ||
|
|
25cc264306 | ||
|
|
86959b7599 | ||
|
|
8664accbc4 | ||
|
|
449a6b13ca | ||
|
|
c71ddd7fe6 | ||
|
|
3b6c5f9585 | ||
|
|
1c71fe6896 | ||
|
|
9159106837 | ||
|
|
28ff14589b | ||
|
|
5fffb3b886 | ||
|
|
2846824a21 | ||
|
|
0545d1d814 | ||
|
|
49c65cbd5d | ||
|
|
888a7229dc | ||
|
|
0f0064c2d7 | ||
|
|
cb90fa9835 | ||
|
|
551ed0a29d | ||
|
|
0dc1a598e2 | ||
|
|
131346c228 | ||
|
|
a5b7b540c6 | ||
|
|
874c6c5a92 | ||
|
|
145c2c668b | ||
|
|
1cef3385e8 | ||
|
|
fbd3cf0129 | ||
|
|
e400c4359c | ||
|
|
4d3574cd60 | ||
|
|
c7ea15dbb4 | ||
|
|
01fe680456 |
No files matched your search
+219
@@ -7,6 +7,225 @@ recorded here; their history is the git log.
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [4.27.0] - 2026-11-03
|
||||
|
||||
### Added
|
||||
- `ScheduledJobsMixin`: what a host runs on its own schedule as one shape,
|
||||
`ScheduledJobDict` — cron lines, run-parts scripts, anacron, systemd timers, `periodic`,
|
||||
a platform's own jobs, Windows tasks — with who owns each definition (`managed_by`) and
|
||||
what the driver can do to it (`actions`). `get_scheduled_jobs` reads them with the host's
|
||||
time zone; `manage_scheduled_job` enables, disables or runs one now; `put_scheduled_job`
|
||||
and `delete_scheduled_job` create, change and delete a job an application writes. Every
|
||||
write names the `digest` the caller saw (`job_digest`) and raises `ScheduledJobChanged`
|
||||
when the host's definition differs; `ScheduledJobRefused` for a job that cannot take a
|
||||
verb (NetOrk/netork#1933, #86).
|
||||
- `PosixScheduledJobsMixin`: the implementation over the command channel. One framed report
|
||||
as root reads `/etc/crontab`, `/etc/cron.d`, the user crontabs, the run-parts directories,
|
||||
`/etc/anacrontab` and every systemd timer with the unit it starts, and which package owns
|
||||
each definition. A cron line is switched off behind `#napalm-disabled: `; an
|
||||
application's job is a timer where systemd runs, else a file in `/etc/cron.d` or marked
|
||||
lines in the user's crontab.
|
||||
- `napalm_device_types.cron`: cron expressions as Vixie cron reads them, their next runs in
|
||||
a time zone, and as systemd `OnCalendar` lines that fire on exactly the same minutes.
|
||||
|
||||
### Removed
|
||||
- `OSDriver.get_cron_jobs` and `CronJobDict`. No application called them; the readers in
|
||||
napalm-linux and napalm-bsd go with them. `get_scheduled_jobs` replaces them.
|
||||
|
||||
## [4.26.0] - 2026-11-03
|
||||
|
||||
### Added
|
||||
- `OpenSshServerMixin.SSHD_CONFIG_GENERATED`: a driver whose platform writes sshd_config
|
||||
from its own database, but includes `/etc/ssh/sshd_config.d` (OpenMediaVault), sets it
|
||||
and keeps `harden_ssh_server`. The hardening then goes only into the drop-in; a
|
||||
sshd_config without that include is left alone and the result says why, since the
|
||||
platform would write over a change to it on its next apply (#88, NetOrk/netork#1950).
|
||||
|
||||
## [4.25.0] - 2026-11-02
|
||||
|
||||
### Added
|
||||
- `BackupServerDriver` (`ROLE = "backup_server"`), a role base for backup servers such as
|
||||
Proxmox Backup Server (#82). It declares `get_backup_datastores`, `get_backup_groups`
|
||||
(each group with its newest snapshot), `get_backup_snapshots`, `get_backup_jobs`
|
||||
(garbage collection, verify, prune, sync and tape in one list) and `get_backup_remotes`,
|
||||
and the actions `start_garbage_collection`, `run_backup_job`, `verify_backup_snapshot`,
|
||||
`set_backup_snapshot_protected` and `delete_backup_snapshot`, which return a task to
|
||||
follow with `get_backup_task`. The shapes live in `napalm_device_types.backups`. A
|
||||
server that manages its own disks lists `StorageDriver` as its second role.
|
||||
- `ClusterStorageDict.backup_server` (optional): which backup server, datastore and
|
||||
namespace a hypervisor's backup storage writes to, with the certificate fingerprint it
|
||||
pins, so an application can tie a server's backup groups to a cluster's guests.
|
||||
|
||||
## [4.24.0] - 2026-11-02
|
||||
|
||||
### Added
|
||||
- `parse_lldpctl_keyvalue(output)` and `LLDPCTL_KEYVALUE_COMMAND`: lldpd's neighbours
|
||||
(`lldpctl -f keyvalue`) as one `LldpNeighborRow` per neighbour, for every driver of a
|
||||
host that runs lldpd. Several neighbours on one port (a phone and the PC behind it, an
|
||||
unmanaged switch) stay apart, an interface name with dots (`eth0.1`) stays whole, the
|
||||
chassis and port ids are read whatever their type (MAC, address, interface name, local
|
||||
id), with management addresses and capabilities. A failed lldpctl (lldpd not running,
|
||||
lldpctl missing) raises instead of reading as no neighbours.
|
||||
- `lldp_neighbors(rows)` and `lldp_neighbors_detail(rows, interface="")`: NAPALM's
|
||||
`get_lldp_neighbors` and `get_lldp_neighbors_detail` from those rows, so a driver no
|
||||
longer builds the two shapes itself. A neighbour that names its chassis by a MAC
|
||||
address carries it in `get_lldp_neighbors` as `mac`, in lower case
|
||||
(`LldpNeighborDict`), so an application can match it by MAC.
|
||||
|
||||
### Fixed
|
||||
- `DNF_SECURITY_COMMAND` reads the cached metadata only (`dnf --cacheonly`), as
|
||||
`APT_UPGRADABLE_COMMAND` reads apt's local index (#76). With an expired cache dnf
|
||||
downloaded the repositories' metadata first, and on an Oracle Linux host on OCI that
|
||||
outlasted netOrk's whole poll budget. Refreshing the index stays a step of its own
|
||||
(`dnf makecache`). Without a cache dnf fails at once, which already reads as
|
||||
"security unknown".
|
||||
|
||||
## [4.23.0] - 2026-11-02
|
||||
|
||||
### Changed
|
||||
- Python 3.12 is the floor (`requires-python = ">=3.12"`): netOrk runs every driver
|
||||
on 3.12, so 3.10 and 3.11 are no longer tested (NAPALM/driver-standard#6).
|
||||
- The floor on napalm is 5.1: older releases import `pkg_resources`, which a Python
|
||||
3.12 venv no longer has.
|
||||
- `requests` is a declared dependency (floor 2.25.0), for `tls_certificates`; it came
|
||||
with napalm before.
|
||||
- The SNMP health profile shapes (`SnmpHealthProfile`, `SnmpValueSpec`,
|
||||
`SnmpMemorySpec`, `SnmpInterfacesSpec`) are defined in
|
||||
`napalm_device_types.snmp_health_profile`. The package and `models` still export
|
||||
them, so an existing import keeps working.
|
||||
|
||||
### Added
|
||||
- `HypervisorDriver.get_vm_migration_preflight(name)`, `migrate_vm(name, target_node,
|
||||
online=True, target_storage=None, timeout=3600)` and `set_node_maintenance(node,
|
||||
enabled)`: which nodes of its cluster a guest could move to and what blocks it
|
||||
(`MigrationPreflightDict`: `allowed_nodes`, `refused_nodes` with the platform's
|
||||
reason, `local_disks` the migration copies, `local_resources` that block it), the
|
||||
move itself, blocking until the platform's task ended and raising with the
|
||||
platform's own cause, and a node's HA maintenance mode. Declared only; netOrk
|
||||
probes each with `hasattr`.
|
||||
- `get_clusters()` may report HA, backups and quorum votes: `ClusterDict.ha_resources`
|
||||
(`HAResourceDict`: requested and current state, node, PVE 8 group, PVE 9 rules),
|
||||
`backup_jobs` (`BackupJobDict`: schedule, storage, mode, the guests a job covers)
|
||||
and `backups` (`BackupDict`: per guest and storage the newest backup, its size
|
||||
and how many there are), and `ClusterNodeDict.votes`. All `NotRequired`, so
|
||||
drivers written before stay valid; None means could not be read, `[]` read and
|
||||
empty.
|
||||
- `napalm_device_types.tls_certificates`: TLS certificates by trust on first use, the
|
||||
counterpart of `ssh_host_keys` for a driver's HTTPS API. A driver takes the
|
||||
fingerprint to expect from `optional_args["tls_fingerprints"]` (by port, the SHA-256
|
||||
of the DER certificate, `SHA256:AB:CD:…`) and refuses any other certificate with
|
||||
`TlsCertificateMismatch` (a `ConnectionException` naming both fingerprints) before a
|
||||
password is sent; no CA is consulted, so a self-signed certificate verifies. Without
|
||||
a fingerprint the first certificate is accepted. Every certificate seen goes to
|
||||
`optional_args["tls_certificate_observer"]`. `CertificateCheck.verify_server()`
|
||||
checks the certificate in a handshake of its own, `pin(session)` makes every later
|
||||
connection of a `requests` session present the same one, `refusing()` turns a refusal
|
||||
there into the mismatch, `observe()` reports it once the login stands
|
||||
(NAPALM/napalm-hpe-aruba-procurve#17).
|
||||
- `LldpAgentMixin.configure_lldp_agent(interfaces)` (`napalm_device_types.fixes`): run
|
||||
an LLDP agent on the interfaces the caller names. The template installs the agent
|
||||
(`ensure_package(LLDP_AGENT_PACKAGE)`), has the driver write the interface list
|
||||
(`_write_lldp_agent_config`), then enables and restarts the service
|
||||
(`manage_service(LLDP_AGENT_SERVICE, …)`); both names default to `lldpd`. No
|
||||
interface is a ValueError before anything changes. It is the action for what LLDP
|
||||
getters used to do on their own (NAPALM/napalm-openwrt#8).
|
||||
- `SnmpValueSpec.unknown`: a driver whose device's agent answers a value wrongly says
|
||||
so in its `SNMP_HEALTH_PROFILE` (`"cpu": {"unknown": True}`), and the application
|
||||
reports no value for it -- not the standard reading, not 0. OpenBSD's net-snmp,
|
||||
which reports the CPU as never idle, is the case (NAPALM/napalm-bsd#7).
|
||||
- `VMProvisionIncomplete` (`napalm_device_types.provisioning`): what
|
||||
`create_vm_from_cloud_init` raises when it fails after the VM was created. A
|
||||
`RuntimeError` with the failure as its cause and the half-built VM's `vmid` and
|
||||
`node`, so the caller can remove it with `destroy_vm` (NAPALM/napalm-proxmox#28).
|
||||
- `HypervisorDriver.get_clusters()` and its shapes `ClusterDict`, `ClusterNodeDict`,
|
||||
`ClusterStorageDict`, `ClusterNetworkDict`: every cluster the connection can see,
|
||||
with its nodes (offline ones included, `local` on the node the connection is),
|
||||
their CPU and memory, and the storages and networks with the nodes that see each.
|
||||
A standalone host is a cluster of one with no name; a manager reports each cluster
|
||||
and standalone host it runs. Declared only (NetOrk/netork#484, #1821).
|
||||
- `create_vm_from_cloud_init(node=…)`: the cluster node to create the VM on. None
|
||||
leaves it to the driver; a driver that can only create on its own node raises
|
||||
ValueError for another (NetOrk/netork#1822).
|
||||
- `LocalAccountsMixin.add_service_account(name, public_key)`: netOrk's own account,
|
||||
created through the account the driver logs in with today. It takes a key and no
|
||||
password, and a rule of its own in `85-service-<name>` lets it run everything as
|
||||
root without a password (`NOPASSWD: ALL`, checked with `visudo` and `sudo -l`). An
|
||||
account that is there is opened again and keeps its other keys; root is refused
|
||||
(NetOrk/netork#1817). With `uid=`, a new account gets that uid once no other account
|
||||
has it; one that is there keeps its own. A driver supplies
|
||||
`_account_create_with_uid`. An account that is there and cannot log in (`nologin`,
|
||||
`false`) is another service's, such as a netOrk satellite's `netork`, and is refused
|
||||
before anything changes.
|
||||
- `LocalAccountsMixin` (`napalm_device_types.local_accounts`): personal login accounts
|
||||
on a host. `get_local_accounts(names)` reads them back (groups, lock, sudo, key
|
||||
files), `add_local_account(name, public_key, password_hash=, admin=)` creates or
|
||||
reopens one, `lock_local_account(name)` takes its keys and sudo away and refuses
|
||||
every login while keeping its home. Every command root runs is one fixed command,
|
||||
the key files are read and written as the account itself, and the sudo rule is a
|
||||
drop-in of its own, checked with `visudo` and `sudo -l`. A driver supplies the
|
||||
commands that create, open and lock an account and take it out of a group, and may
|
||||
replace where the keys are kept (`_account_key_files`, `_account_read_keys`,
|
||||
`_account_add_key`, `_account_remove_keys`; default `~/.ssh/authorized_keys`)
|
||||
(NetOrk/netork#1793).
|
||||
- `OpenSshServerMixin.exempt_from_ssh_penalties`: puts the address the host sees
|
||||
the managing client at (`$SSH_CONNECTION`) into `PerSourcePenaltyExemptList`,
|
||||
in `/etc/ssh/sshd_config.d/00-netork-penalty-exempt.conf`, next to the host's
|
||||
own exemptions. Only where sshd has per-source penalties (OpenSSH 9.8+; an older
|
||||
one refuses the directive) and sshd_config includes the drop-in directory;
|
||||
checked with `sshd -t` and `sshd -T`, then a new session, and taken back on any
|
||||
failure. Helpers: `parse_penalty_exemptions`, `client_address`, `is_exempt`
|
||||
(NetOrk/netork#1800).
|
||||
- `retry_refused` and `refused_before_banner` (`napalm_device_types.ssh_host_keys`):
|
||||
a connection sshd closes before its banner -- what OpenSSH 9.8+ does to a source
|
||||
it has just penalised -- is tried again after 1, 3, 7 and 15 s.
|
||||
`HostKeyCheck.connect` does so itself (NetOrk/netork#1800).
|
||||
- `ARP_AGE_UNKNOWN` (`napalm_device_types.arp`, `-1.0`): the `age` a driver reports
|
||||
in `get_arp_table` and `get_ipv6_neighbors_table` when the device does not say how
|
||||
old an entry is. `0.0` means "confirmed just now" and is no stand-in for it
|
||||
(NetOrk/netork#1327).
|
||||
- `NetworkTargetDict.uplinks` (optional): the physical NICs a bridge leaves the
|
||||
host through, one group per way out -- a bond's slaves together, a single NIC
|
||||
alone (`[["eno1", "eno2"]]`). A missing key means unknown, an empty list a
|
||||
bridge without uplink. NetOrk follows their LLDP neighbours to the switch port
|
||||
and its untagged VLAN; one slave's neighbour speaks for its bond
|
||||
(NetOrk/netork#1325).
|
||||
- `HostShutdownMixin.shutdown_host()`, on `DeviceTypeDriver`: shut the device itself
|
||||
down cleanly through its own means; declared, so `hasattr` says whether a driver
|
||||
can. No forced variant (NetOrk/netork#1636).
|
||||
- `VmStopRequestMixin.begin_stop_vm(name)`: ask for a VM's clean shutdown and return
|
||||
at once, for callers that wait on a timer instead of in `stop_vm`. No role demands
|
||||
it (NetOrk/netork#1636).
|
||||
- `SudoHistoryMixin` (`napalm_device_types.sudo_history`): the sudo password an earlier
|
||||
driver typed into an interactive shell (`echo '<password>' | sudo -S -p "" …`) and
|
||||
left in the login user's shell history, for a driver to mix in (NetOrk/netork#1090).
|
||||
The driver says what its lines carry (`SUDO_HISTORY_MARKER`) and which files its
|
||||
partial sudo left to the login user (`SUDO_ROOT_FILES`), and supplies `run_command`.
|
||||
- `get_history_sudo_lines()` counts the lines per history file (`~/.bash_history`,
|
||||
`~/.zsh_history`, `~/.ash_history`, `~/.sh_history`) with `grep -c` as the login
|
||||
user. No line is read back; a file it cannot read raises.
|
||||
- `remove_history_sudo_lines()` deletes exactly those lines as the login user (`sed
|
||||
-i`, its backup overwritten with `shred -u`), and as root gives each of
|
||||
`SUDO_ROOT_FILES` back to `root:root` where it is a regular file, not a symbolic
|
||||
link, owned by the login user. It reports every file's lines before and after,
|
||||
how many were removed and which files are root's again. Nothing goes to stdin.
|
||||
|
||||
### Fixed
|
||||
- `AptProxyMixin` knows that apt's https sources take the http proxy too: apt's https
|
||||
method asks `Acquire::https::Proxy` and then `Acquire::http::Proxy`, for the host and
|
||||
then for all. An `00proxy` with the http line alone sent download.docker.com and
|
||||
every other https source to apt-cacher-ng, which answers `403 CONNECT denied` without
|
||||
a `PassThroughPattern`, and the whole upgrade failed (NetOrk/netork#1811).
|
||||
- The report carries `https_proxy`, `https_per_host` and `https_settings`;
|
||||
`https_proxy_for(report, host)` is the proxy apt takes for one https host.
|
||||
- `check_https_source()` fetches one https source's `InRelease` the way apt does;
|
||||
None where no https source goes through a proxy.
|
||||
- `configure_package_proxy` gains an `https` step: one https source through the new
|
||||
proxy. Where the proxy refuses it, `00proxy` also says `Acquire::https::Proxy
|
||||
"DIRECT";`, later https lines to the same proxy are commented out, and the source
|
||||
is fetched direct. An HTTP error from the mirror counts as let through.
|
||||
- `apt_fetch_command` takes https URLs; `direct` sets the host's https setting too.
|
||||
- `comment_out_command(..., scheme="https")`.
|
||||
|
||||
## [4.22.0] - 2026-10-23
|
||||
|
||||
### Added
|
||||
|
||||
@@ -47,6 +47,18 @@ UUID and NIC MACs -- what ties the BMC to the host it manages. Its sensors come
|
||||
NAPALM's `get_environment`, with `cpu` and `memory` empty, since a BMC does not know the
|
||||
host OS's load; it has no `HealthMetricsMixin` (SNMP) and no command channel.
|
||||
|
||||
A backup server (Proxmox Backup Server) is a role of its own too, `BackupServerDriver`
|
||||
(`backup_server`). Its datastores, the backup groups in them (each with its newest
|
||||
snapshot), its jobs (garbage collection, verify, prune, sync, tape) and its remotes have
|
||||
no counterpart in `StorageDriver`, so they are not bent into pools, volumes and
|
||||
replication jobs. A server that also manages its own disks and pools lists
|
||||
`StorageDriver` second: `class ProxmoxBackupDriver(BackupServerDriver, StorageDriver, ...)`.
|
||||
Its actions (garbage collection, running a job, verifying, protecting or deleting a
|
||||
snapshot) start a task and return its id at once; `get_backup_task` follows it. On the
|
||||
hypervisor side, a cluster storage that writes to a backup server says which one
|
||||
(`ClusterStorageDict.backup_server`: server, port, datastore, namespace, fingerprint), so
|
||||
an application can tie the server's groups to the cluster's guests.
|
||||
|
||||
### Role bases declare; they never implement
|
||||
|
||||
**A role base must not contain a single runtime method — not even a
|
||||
@@ -85,20 +97,34 @@ base; a mixin listed after it loses the MRO to NAPALM's stub. For the same reaso
|
||||
`hasattr` says nothing about NAPALM's getters, and the OS role's account listing is
|
||||
`get_os_users`, not `get_users` -- NAPALM's `get_users` has a different shape.
|
||||
|
||||
### NAPALM's getters: an unknown ARP age is `-1.0`
|
||||
|
||||
NAPALM requires an `age` on every entry of `get_arp_table` and
|
||||
`get_ipv6_neighbors_table`: seconds since the device last confirmed that neighbour.
|
||||
Many devices do not report one (`ip neigh`, `/proc/net/arp`, a router's host list).
|
||||
A driver that cannot read the age reports `ARP_AGE_UNKNOWN` (`-1.0`, as NAPALM's IOS
|
||||
driver does), never `0.0`: `0.0` means "confirmed just now", and an application that
|
||||
ranks sightings by freshness takes it at its word (NetOrk/netork#1327). An expiry
|
||||
countdown (FreeBSD's `expires in 1180 seconds`) is not an age either. A driver whose
|
||||
device does report the age keeps reporting it.
|
||||
|
||||
## Function classes: what a device *can do*
|
||||
|
||||
Behaviour shared across roles lives in a function class, exactly once, and a role base
|
||||
is a thin bundle over them — `PackageManagementMixin`, `HealthMetricsMixin`,
|
||||
`ServiceControlMixin`, `UpdateMixin`, `NatVpnMixin`, `MacAclMixin`, `FirewallRuleMixin`,
|
||||
`DhcpServerMixin`, `PingSweepMixin`, `ConfigLifecycleMixin`, `InterfaceFilterMixin`.
|
||||
`HostRebootMixin` (`reboot_host`) is mixed into `DeviceTypeDriver` itself, since any
|
||||
device may be restartable; like the others it only declares.
|
||||
`HostRebootMixin` (`reboot_host`) and `HostShutdownMixin` (`shutdown_host`) are mixed
|
||||
into `DeviceTypeDriver` itself, since any device may be restartable or able to shut itself
|
||||
down; like the others they only declare. `VmStopRequestMixin` (`begin_stop_vm`) asks for a
|
||||
VM's clean shutdown without waiting for it — a hypervisor driver mixes it in when it can.
|
||||
|
||||
**Fixes are templates** (`napalm_device_types.fixes`, since 4.13). A change the
|
||||
application wants made on a device runs as fixed steps the driver implements, with values
|
||||
the caller supplies: `SnmpAgentMixin.configure_snmp_agent(settings)` (install, configure,
|
||||
firewall, start), `EnsurePackageMixin.ensure_package(name)`,
|
||||
`PackageProxyMixin.configure_package_proxy(url)` and
|
||||
firewall, start), `LldpAgentMixin.configure_lldp_agent(interfaces)` (install, write the
|
||||
interfaces, enable and restart; the driver writes only the interfaces),
|
||||
`EnsurePackageMixin.ensure_package(name)`, `PackageProxyMixin.configure_package_proxy(url)` and
|
||||
`ContainerEngineMixin.grant_container_engine_access(user)`. A driver whose device takes the
|
||||
change in one call overrides the template method. A driver mixes in the fixes its device
|
||||
can take; none is on a role base. They replace `run_device_action`, which is deprecated.
|
||||
@@ -119,6 +145,9 @@ class MySwitchDriver(SwitchDriver):
|
||||
A value is `oids` (the first that answers wins) or a `table` folded by `first`, `average`
|
||||
or `max`, with an optional `pattern` for an agent that answers with text. What the profile
|
||||
does not name, the application reads from the standard UCD-MIB and IF-MIB.
|
||||
A value the agent gets wrong is `{"unknown": True}`: the application then reports no value
|
||||
for it, rather than the standard reading or a default (OpenBSD's net-snmp reports its CPU
|
||||
as never idle).
|
||||
|
||||
`KernelFactsMixin` (`get_kernel_facts`) is the exception that is mixed in by a driver
|
||||
rather than by a role base: what a Linux kernel has built and loaded is read the same way
|
||||
@@ -162,6 +191,16 @@ without `ss` is read with `netstat -lntup` (OpenWrt's busybox, old net-tools); o
|
||||
cgroup names the procd service (`/services/<name>/<instance>`). A host with neither raises
|
||||
`ListeningSocketsUnavailable`.
|
||||
|
||||
LLDP neighbours are rows: `LldpNeighborRow` is one neighbour as its local port heard it
|
||||
(chassis and port id with their type, name, description, management addresses,
|
||||
capabilities in NAPALM's names), and `lldp_neighbors(rows)` / `lldp_neighbors_detail(rows,
|
||||
interface)` are NAPALM's two getters projected from them, a port with two neighbours
|
||||
keeping both; a neighbour named by its chassis MAC carries it as `mac` (lower case). A driver for a host that runs lldpd sends `LLDPCTL_KEYVALUE_COMMAND` and
|
||||
hands its output to `parse_lldpctl_keyvalue`: one row per neighbour block, an interface
|
||||
name with dots (`eth0.1`) kept whole, ids read under the type lldpd prints them with
|
||||
(`chassis.mac`, `port.local`, ...). When lldpctl fails (lldpd not running, no `lldpctl`)
|
||||
the parse raises instead of reading as no neighbours.
|
||||
|
||||
`HostFirewallMixin` (`get_host_firewall`) reads a Linux host's own netfilter rules the same
|
||||
way: `nft -j list ruleset`, with the tables iptables-nft manages taken from `iptables-save`
|
||||
instead, because nft shows their extensions (`-m conntrack`, `-m set`) only as an opaque `xt`.
|
||||
@@ -239,6 +278,16 @@ A driver mixes it in and supplies `run_command`; a platform that regenerates ssh
|
||||
itself (an appliance with its own database) must not, or opts out with
|
||||
`harden_ssh_server = None`.
|
||||
|
||||
`SudoHistoryMixin` (`get_history_sudo_lines`, `remove_history_sudo_lines`) deals with what
|
||||
a driver left behind when it typed `echo '<password>' | sudo -S -p "" …` into an interactive
|
||||
shell: that line, password and all, in the login user's shell history, and files its partial
|
||||
sudo wrote as the login user. The driver names the marker its lines carry
|
||||
(`SUDO_HISTORY_MARKER`) and those files (`SUDO_ROOT_FILES`). The getter counts the lines per
|
||||
history file with `grep -c`, as the login user; the fix deletes them with `sed -i` and
|
||||
overwrites the backup with `shred -u`, then gives the files back to root (regular files the
|
||||
login user owns, never a symbolic link). The password is never read, sent or written. A
|
||||
driver mixes it in and supplies `run_command`.
|
||||
|
||||
**Update readers raise when they cannot read.** `get_available_updates` returns an empty
|
||||
list only when nothing is pending; netOrk keeps "pending since" per package, and an empty
|
||||
list for "don't know" would reset it.
|
||||
@@ -252,6 +301,20 @@ trip (`list-unit-files` plus one `systemctl show` over every loaded unit) instea
|
||||
`is-enabled` and a `show` per unit. A host without systemd raises `SystemdUnavailable`,
|
||||
a `NotImplementedError`, so a driver can fall back to another init system.
|
||||
|
||||
`ScheduledJobsMixin` (`get_scheduled_jobs`, `manage_scheduled_job`, `put_scheduled_job`,
|
||||
`delete_scheduled_job`) is what a host runs on its own schedule — cron lines, run-parts
|
||||
scripts, anacron, systemd timers, `periodic`, a platform's own jobs, Windows tasks — as one
|
||||
shape, `ScheduledJobDict`. Every job says who owns its definition (`managed_by`: package,
|
||||
admin, platform, application) and what the driver can do to it (`actions`): every job a
|
||||
driver can switch takes `enable`, `disable` and `run`; only a job an application wrote
|
||||
through `put_scheduled_job` takes `edit` and `delete`. Every write names the `digest` the
|
||||
caller saw and raises `ScheduledJobChanged` when the host's definition is no longer that.
|
||||
`PosixScheduledJobsMixin` implements it over `run_command`: one framed report as root,
|
||||
a cron line switched off behind `#napalm-disabled: `, a timer by `systemctl`, an
|
||||
application's job as a timer where systemd runs, else in `/etc/cron.d` or the user's
|
||||
crontab. A driver sets the paths its host keeps jobs in. `napalm_device_types.cron` parses
|
||||
cron expressions, gives their next runs and writes them as `OnCalendar` lines.
|
||||
|
||||
### Access channels: why there is no container model here
|
||||
|
||||
`CommandChannelMixin` declares a driver's public channel to its host:
|
||||
@@ -305,6 +368,26 @@ caller's; `ssh_host_keys.HostKeyCheck` is how every SSH driver verifies against
|
||||
A caller that passes neither key gets first use on every connection, as before; a driver
|
||||
that ignores both still connects. Old and new versions on either side keep working.
|
||||
|
||||
### TLS certificates: trust on first use
|
||||
|
||||
The same for a driver's HTTPS API. Devices ship self-signed certificates, so a CA check
|
||||
fails on all of them and turning verification off trusts whatever answers;
|
||||
`tls_certificates.CertificateCheck` pins the certificate by its fingerprint instead
|
||||
(NAPALM/napalm-hpe-aruba-procurve#17). No CA is consulted.
|
||||
|
||||
- **In:** `optional_args["tls_fingerprints"]` = `{"443": "SHA256:AB:CD:…"}`, by port: the
|
||||
SHA-256 of the DER certificate as `openssl x509 -fingerprint -sha256` prints it (prefix,
|
||||
colons and case optional). With one, any other certificate is refused *before* a request
|
||||
carries credentials and the connection raises `TlsCertificateMismatch` (a
|
||||
`ConnectionException` naming both fingerprints). Without one, the first certificate is
|
||||
accepted. A driver does not fall back to another transport on a mismatch.
|
||||
- **Out:** `optional_args["tls_certificate_observer"]`, a callable, gets one dict per
|
||||
connection: `{"host", "port", "seen": {"fingerprint"}, "expected", "accepted"}`.
|
||||
- **requests:** `check = CertificateCheck(optional_args, host, port)`;
|
||||
`check.verify_server(timeout)` before the login, `check.pin(session)` so every later
|
||||
connection of the session must present that certificate, the login inside
|
||||
`with check.refusing():`, then `check.observe()`.
|
||||
|
||||
## Design principle: generic vs. device-specific logic
|
||||
|
||||
When adding behavior to a device-type base class, split it along one line: **would
|
||||
@@ -388,7 +471,7 @@ detection vs. presentation there.
|
||||
pip install napalm-device-types
|
||||
```
|
||||
|
||||
Requires Python ≥ 3.10 (tested up to 3.13) and NAPALM ≥ 4.0.
|
||||
Requires Python ≥ 3.12 (tested up to 3.13) and NAPALM ≥ 4.0.
|
||||
|
||||
## Available base classes
|
||||
|
||||
@@ -404,6 +487,7 @@ Requires Python ≥ 3.10 (tested up to 3.13) and NAPALM ≥ 4.0.
|
||||
| `PhoneDriver` | IP telephony endpoints | Yealink |
|
||||
| `MediaDriver` | Speakers, streamers, media renderers | Sonos, Chromecast |
|
||||
| `ManagementControllerDriver` | Server baseboard management controllers | Supermicro BMC, Dell iDRAC, HPE iLO, Lenovo XCC |
|
||||
| `BackupServerDriver` | Backup servers: datastores, backup groups, GC/verify/prune/sync/tape jobs | Proxmox Backup Server |
|
||||
|
||||
Mixins that come with the classes above: `PingSweepMixin` (subnet sweeps, in
|
||||
every driver through `DeviceTypeDriver`) and `DhcpServerMixin` (static DHCP
|
||||
@@ -501,6 +585,14 @@ class ProxmoxDriver(HypervisorDriver):
|
||||
|
||||
def create_vm_snapshot(self, name, snapshot, description="", include_memory=False): ...
|
||||
|
||||
def get_network_targets(self):
|
||||
# return List[NetworkTargetDict]: bridges and vnets a new VM's NIC may
|
||||
# attach to; a bridge may name its physical NICs in `uplinks`, one group
|
||||
# per way out: [["eno1", "eno2"]] for a bond, [["eno1"]] for a single NIC,
|
||||
# no internal or VLAN ports -- leave the key out when they cannot be
|
||||
# determined
|
||||
...
|
||||
|
||||
def get_vm_cpu_types(self):
|
||||
# optional — return List[VMCpuTypeDict]: the CPU models a new VM may get
|
||||
# on this node, each with its cpuinfo flags and whether the node can run
|
||||
@@ -516,7 +608,10 @@ any other. A QEMU-based driver takes `provisioning.QEMU_GUEST_AGENTS`
|
||||
(Linux, FreeBSD, OpenBSD). `provisioning.network_config()` writes the
|
||||
network-config v2 every cloud-init flavour reads (FreeBSD's nuageinit reads
|
||||
no other), and `provisioning.split_compression()` says how a packed image is
|
||||
unpacked; both are the same for every hypervisor.
|
||||
unpacked; both are the same for every hypervisor. A creation that fails once
|
||||
the VM exists raises `VMProvisionIncomplete`, a `RuntimeError` that names the
|
||||
half-built VM (`vmid`, `node`) for the caller to `destroy_vm`, with the failure
|
||||
as its cause.
|
||||
|
||||
### OS / Linux
|
||||
|
||||
@@ -540,10 +635,6 @@ class LinuxDriver(OSDriver):
|
||||
def get_processes(self):
|
||||
# return List[ProcessDict]
|
||||
...
|
||||
|
||||
def get_cron_jobs(self):
|
||||
# return List[CronJobDict]
|
||||
...
|
||||
```
|
||||
|
||||
### Storage / NAS
|
||||
|
||||
@@ -0,0 +1,244 @@
|
||||
# Structure
|
||||
|
||||
Status: agreed plan. Nothing here is implemented yet; the migration at the end says in
|
||||
which order it will be.
|
||||
|
||||
## Principle
|
||||
|
||||
**napalm-device-types is vendor-, OS-, protocol- and product-agnostic.** It holds the
|
||||
contracts every driver implements and the generic logic that works unchanged for every
|
||||
driver of a role. It knows no shell, no paramiko, no package manager, no container CLI,
|
||||
no SNMP OID, no cloud-init and no application that uses the drivers.
|
||||
|
||||
That rules out three kinds of code that are here today:
|
||||
|
||||
- **How a kind of host is read.** Shell commands and their parsers for POSIX hosts move to
|
||||
a new package, **napalm-unix**, which napalm-linux and napalm-bsd depend on.
|
||||
- **What an application does with a device.** Polling a device over SNMP, sweeping a range
|
||||
with pings, scoring discovery evidence, choosing what a provisioned guest installs, and
|
||||
deciding what counts as a problem belong to the application, not to the drivers.
|
||||
- **Product naming.** Files, markers and messages a driver leaves on a device carry the
|
||||
driver's name or none, never the name of the application that called it.
|
||||
|
||||
```
|
||||
napalm-device-types roles, capabilities, templates, models, reconcilers <- every driver
|
||||
^
|
||||
napalm-unix POSIX host readers, SSH exec transport, privilege <- napalm-linux, napalm-bsd
|
||||
^
|
||||
napalm-linux, napalm-bsd connection, platform getters, platform steps
|
||||
|
||||
application SNMP polling, ping sweeps, discovery scoring, guest
|
||||
profiles, warnings, and the values every fix is given
|
||||
```
|
||||
|
||||
## What stays in napalm-device-types
|
||||
|
||||
| Kind | Rule | Modules |
|
||||
|---|---|---|
|
||||
| **Role** | Declares only, under `TYPE_CHECKING`. No runtime method. | `access_point`, `switch`, `firewall`, `hypervisor`, `os`, `storage`, `phone`, `media`, `residential_gateway`, `management_controller`, `backup_server` |
|
||||
| **Capability** | Declares only. | `packages`, `services`, `updates`, `nat_vpn`, `mac_acl`, `config_lifecycle`, `interface_filter`, `host_reboot` |
|
||||
| **Template** | A concrete method that runs a fixed sequence of abstract steps. A driver implements the steps, or overrides the whole method when its device does it in one go. All values come from the caller. | the fix routines below |
|
||||
| **Contract types** | Interfaces and data shapes other packages implement or fill. | `CommandChannelMixin`, `CommandResult`, `ByteStream`; the declaration of `ContainerEngineMixin`; the reader mixins' public method declarations; the SNMP profile type; `models` |
|
||||
| **Reconciler** | Concrete and vendor-neutral, without I/O of its own. | `firewall_rules` and `dhcp` diff/apply, `_matching`, `lag` |
|
||||
| **Introspection** | Concrete, without I/O. | `roles`; `base` (`FingerprintRule`, `PortSpec`, the fingerprint attributes) |
|
||||
| **Driver facts** | Class attributes that describe the device or how it is reached, not what an application does with it. | `USES_SSH`, `FULL_UPGRADE`, `platform`, `SNMP_HEALTH_PROFILE` |
|
||||
|
||||
### Fix routines are templates
|
||||
|
||||
A change the application wants made on a device, for example "run an SNMP agent that
|
||||
answers us", is one template per purpose. The template fixes the order of the steps; the
|
||||
driver knows how each step is done on its device; the caller supplies every value.
|
||||
|
||||
```python
|
||||
class SnmpAgentMixin:
|
||||
def configure_snmp_agent(self, settings: SnmpAgentSettingsDict) -> FixResultDict:
|
||||
steps = [
|
||||
self._install_snmp_agent,
|
||||
self._write_snmp_agent_config,
|
||||
self._enable_snmp_agent,
|
||||
self._verify_snmp_agent,
|
||||
]
|
||||
... # runs them in order, stops at the first failure
|
||||
|
||||
# declared only; napalm-unix implements them for net-snmp
|
||||
def _install_snmp_agent(self) -> FixStepDict: ...
|
||||
def _write_snmp_agent_config(self, settings: SnmpAgentSettingsDict) -> FixStepDict: ...
|
||||
def _enable_snmp_agent(self) -> FixStepDict: ...
|
||||
def _verify_snmp_agent(self, settings: SnmpAgentSettingsDict) -> FixStepDict: ...
|
||||
```
|
||||
|
||||
napalm-unix implements the four steps for net-snmp, for Linux and the BSDs alike. A REST
|
||||
driver such as OPNsense overrides `configure_snmp_agent` as a whole, because its device
|
||||
takes the whole configuration in one call. The community, the allowed sources, contact
|
||||
and location are the caller's.
|
||||
|
||||
The same shape replaces every action in today's `run_device_action`:
|
||||
|
||||
| Today | Template |
|
||||
|---|---|
|
||||
| `fix_snmp` | `configure_snmp_agent(settings)` |
|
||||
| `fix_apt_proxy` | `configure_package_proxy(url)` |
|
||||
| `fix_docker_permissions` | `grant_container_engine_access(user)` |
|
||||
| `install_lldpd`, `install_auc` | `ensure_package(name)` |
|
||||
|
||||
The mixin is mixed in by the driver, or by the platform package, not by a role base. A
|
||||
driver that inherits a template it cannot run on its device, as a NAS driver built on the
|
||||
Linux driver inherits the package proxy, opts out by setting the method to `None`. So a
|
||||
caller asks `callable(getattr(driver, "configure_snmp_agent", None))`, never `hasattr`.
|
||||
|
||||
The caller verifies the result from its own side where it can; the SNMP agent is asked
|
||||
by the application's SNMP client, not by a probe the driver hand-builds.
|
||||
|
||||
### SNMP: drivers declare data, the application polls
|
||||
|
||||
The drivers do not collect over SNMP. A driver whose device answers the standard UCD-MIB
|
||||
and IF-MIB declares nothing. A driver whose device publishes the same values under
|
||||
proprietary OIDs declares them as data, in a class attribute of a type defined here:
|
||||
|
||||
```python
|
||||
class ProcurveDriver(SwitchDriver):
|
||||
SNMP_HEALTH_PROFILE: ClassVar[SnmpHealthProfile] = {
|
||||
"uptime": {"oids": ["1.3.6.1.4.1.11.2.14.11.1.2.4.1.5.1"], "format": "ticks"},
|
||||
"cpu": {"oids": ["1.3.6.1.4.1.11.2.14.11.5.1.9.6.1.0", "1.3.6.1.4.1.11.2.3.7.8.6.1.0"]},
|
||||
"memory": {"total": "...", "free": "...", "unit": "bytes"},
|
||||
}
|
||||
```
|
||||
|
||||
One OID per value is not enough for the drivers there are. The profile type has to say:
|
||||
|
||||
- **Fallbacks:** an ordered list of OIDs; the first that answers wins (ProCurve's CPU).
|
||||
- **Tables:** an OID to walk and how to fold its rows: first, average or maximum (TP-Link
|
||||
reports CPU and memory per stack unit).
|
||||
- **Text values:** a pattern that takes the number out of a string (Netgear reports CPU
|
||||
as `"5 Secs ( 3.2%)"`).
|
||||
- **Memory:** total with used or free, or a percentage only, and the unit.
|
||||
- **Uptime:** which OID, standard ones included (`sysUpTime` where a device has no
|
||||
`hrSystemUptime`), and its format.
|
||||
- **Interfaces:** a pattern of names to skip, and whether the TX-error counter counts
|
||||
drops. Today the second is `True` on every firewall, because OPNsense counts drops
|
||||
there; it becomes OPNsense's profile entry.
|
||||
|
||||
Polling, the standard OIDs, the counter arithmetic (64-bit counters first, 32-bit as the
|
||||
fallback) and the defaults are the application's.
|
||||
|
||||
### Warnings are facts
|
||||
|
||||
A driver reports facts: the proxy a host is configured with, a DSL line's SNR, the
|
||||
firmware version it runs and the one it is offered. It does not compare them with a
|
||||
setting of the application or with a threshold, and it does not name a warning.
|
||||
|
||||
The same goes for lookups a device does not make itself: what firmware a vendor
|
||||
currently publishes is the application's catalogue, not a getter.
|
||||
|
||||
### Getters read, and say when they cannot
|
||||
|
||||
- A getter changes nothing on the device. It does not start a daemon, rewrite its
|
||||
configuration or install a package on the way to an answer.
|
||||
- A read that fails raises. An empty list means "there is none", never "could not ask";
|
||||
a sync that deletes and re-adds on top of an empty answer would otherwise duplicate
|
||||
everything.
|
||||
- A write reports failure from the device's answer or exit status, never `success: True`
|
||||
by default.
|
||||
|
||||
### The application uses public methods only
|
||||
|
||||
The application never calls a driver's private methods (`_get`, `_post`,
|
||||
`_send_command`, `_action_*`). What it needs from a device becomes a declared method of
|
||||
the driver.
|
||||
|
||||
## What moves to napalm-unix
|
||||
|
||||
| Module | Knows |
|
||||
|---|---|
|
||||
| `host_status` | apt, dnf, dpkg, systemd timers, `df` |
|
||||
| `kernel` | `/proc`, `/lib/modules`, gzip, base64 |
|
||||
| `listening` | `ss`, `netstat`, `/proc/<pid>/cgroup` |
|
||||
| `systemd` | `systemctl` |
|
||||
| `virtual_ips` | vip-manager, keepalived, awk |
|
||||
| `system_identity` | SMBIOS via sysfs and `dmidecode` |
|
||||
| `package_updates` | apt and dnf output |
|
||||
| `ssh_server` | OpenSSH (`sshd -T`, `sshd -t`, `sshd_config`); which algorithms to drop stays the caller's |
|
||||
| `terminal` | ANSI escape codes in shell output |
|
||||
| `channel`: `run_on_transport`, `open_stream_on_transport`, `ParamikoExecStream` | paramiko exec channels |
|
||||
| `container_engine`: `ContainerEngineConnection`, engine probing | the Docker CLI, `system dial-stdio` |
|
||||
|
||||
It also takes what napalm-linux and napalm-bsd each carry today:
|
||||
|
||||
- **Privilege:** one `sudo -S` / `sudo -n` / root layer, including the rule that a
|
||||
privileged command never gets stdin while a sudo password is set, and one way to write a
|
||||
file as root (stage with `mktemp`, then `install`). Two copies of this have already
|
||||
drifted apart.
|
||||
- **Parsers:** `/etc/passwd` with `/etc/group`, crontab lines, `ps` with `lstart`.
|
||||
- **Fix steps:** net-snmp, the package proxy, container engine group membership.
|
||||
|
||||
The FreeBSD part of `host_status` (`freebsd-version -k` against `-r`) moves to napalm-bsd.
|
||||
|
||||
In napalm-unix every reader follows one pattern:
|
||||
|
||||
- **Shape:** `FOO_COMMAND`, `parse_foo(output)`, and `FooMixin.get_foo()`.
|
||||
- **Reaching the host:** the mixin uses `run_command(command, privileged=...)` from
|
||||
`CommandChannelMixin`. The six per-reader `_run_*` hooks, with four different signatures,
|
||||
go away.
|
||||
- **Framing:** one frame helper writes and parses the `BEGIN`/`END` markers for every reader.
|
||||
- **Errors:** one exception for "cannot read" and one for "not on this host". A read that
|
||||
fails or is cut short raises; it never reads as "nothing there".
|
||||
- **Names on the host:** temporary files, markers and drop-ins carry `napalm` or the
|
||||
driver's name. A drop-in written under an older name is still recognised and replaced.
|
||||
|
||||
Layout: `napalm_unix/{posix,linux,freebsd,openssh,transport}/`.
|
||||
|
||||
## What moves to the application
|
||||
|
||||
| Today | Why it is not a driver's |
|
||||
|---|---|
|
||||
| `health_metrics`, `_ucd_metrics` | Polling over SNMP is something the application does to a device; the driver is not even connected while it runs. |
|
||||
| `ping_sweep` | Looping over addresses, capping the targets and reporting progress is a scan. A device with a faster batch keeps it as a capability of its own. |
|
||||
| `rules_max_weight` | Scoring is how one application ranks drivers. The rules themselves stay with the driver. |
|
||||
| `provisioning` (`QEMU_GUEST_AGENTS`, cloud-init network-config, decompressors) | Which agent a guest gets and how its network is configured is a guest profile. The hypervisor driver is handed the packages and commands. |
|
||||
| `TYPE_LABEL`, `REBOOT_SETTLE_SECONDS` | A display name and a polling delay. |
|
||||
| firmware catalogues (napalm-netgear's `netgear_plus`) | What a vendor publishes is not something the device reports. |
|
||||
| `run_device_action` | Replaced by the templates above. |
|
||||
|
||||
`OSDriver.ROLE` stops defaulting to `"linux"`. Each OS driver states its own role.
|
||||
|
||||
## Rules, enforced by tests in this package
|
||||
|
||||
1. Roles and capabilities own no runtime method; a template runs only declared steps.
|
||||
2. No module imports `paramiko`, `socket`, `subprocess` or `asyncio`, and none defines a
|
||||
`*_COMMAND`.
|
||||
3. No module, string or default names an application.
|
||||
4. Role methods return TypedDicts, not `Any`.
|
||||
|
||||
Each rule is added with the step that makes it pass.
|
||||
|
||||
## Migration
|
||||
|
||||
This package cannot keep re-exports pointing at napalm-unix: the contract package would
|
||||
then depend on its implementation. So a moved name is first deprecated here and then
|
||||
removed, and nothing breaks in between.
|
||||
|
||||
| Step | Change |
|
||||
|---|---|
|
||||
| 1 | The application moves to 4.9 and the drivers that need it; napalm-bsd stops passing stdin to a privileged command while a sudo password is set. |
|
||||
| 2 | This document; the application's architecture docs and this README follow it. |
|
||||
| 3 | `SnmpHealthProfile`. The application polls on its own; the four switch and phone drivers declare their OIDs as data. `HealthMetricsMixin` is deprecated. |
|
||||
| 4 | The fix templates. napalm-linux and napalm-bsd implement the steps; the application calls the templates with its values. `run_device_action` is deprecated. The other drivers follow, one issue each. |
|
||||
| 5 | Ping sweep, scoring, provisioning, `TYPE_LABEL` and `REBOOT_SETTLE_SECONDS` move to the application and are deprecated here. Drivers report facts instead of warnings. |
|
||||
| 6 | napalm-unix, with the moved code and its tests, the unified hooks and the frame helper. napalm-linux and napalm-bsd import from it; the moved names are deprecated here. |
|
||||
| 7 | 5.0 removes everything deprecated. It also splits `models.py` by area, types the remaining `Dict[str, Any]` role returns, declares what drivers already offer undeclared (`get_networks`, `search_packages`, `first_boot_pending`, the base-system update entry), and makes `install_package` declare the result drivers return. |
|
||||
| 8 | The README shrinks to the concepts; the reader details move with the readers. |
|
||||
|
||||
## Proposals this replaces
|
||||
|
||||
Several open issues proposed adding shared driver code here. Under this structure:
|
||||
|
||||
| Issue | Goes to |
|
||||
|---|---|
|
||||
| #8 netmiko session | not here: a CLI-driver helper package, or left in the drivers |
|
||||
| #9 config mode for CLI switches | not here: a CLI-driver helper package, or left in the drivers |
|
||||
| #10 LLDP rows | the projection of rows into NAPALM's two shapes stays here; the `lldpctl` parser goes to napalm-unix |
|
||||
| #11 CLI text helpers | not here; the natural port sort, which `lag` already has, may become public |
|
||||
| #12 ping | the reply builder stays here; the BusyBox and iputils parser goes to napalm-unix |
|
||||
| #13 Linux userland | napalm-unix; the snmpd part becomes the SNMP template's steps |
|
||||
| #14 `run_device_action` | replaced by the fix templates |
|
||||
| #22 hooks onto `run_command` | step 6 |
|
||||
@@ -22,6 +22,7 @@ driver do X".
|
||||
Role bases -- what a device *is*:
|
||||
|
||||
* :class:`~napalm_device_types.access_point.AccessPointDriver`
|
||||
* :class:`~napalm_device_types.backup_server.BackupServerDriver`
|
||||
* :class:`~napalm_device_types.switch.SwitchDriver`
|
||||
* :class:`~napalm_device_types.firewall.FirewallDriver`
|
||||
* :class:`~napalm_device_types.hypervisor.HypervisorDriver`
|
||||
@@ -44,9 +45,11 @@ instead of being restated on every role that happens to need it:
|
||||
* :class:`~napalm_device_types.health_metrics.HealthMetricsMixin`
|
||||
* :class:`~napalm_device_types.host_status.HostStatusMixin`
|
||||
* :class:`~napalm_device_types.host_reboot.HostRebootMixin`
|
||||
* :class:`~napalm_device_types.host_shutdown.HostShutdownMixin`
|
||||
* :class:`~napalm_device_types.interface_filter.InterfaceFilterMixin`
|
||||
* :class:`~napalm_device_types.kernel.KernelFactsMixin`
|
||||
* :class:`~napalm_device_types.listening.ListeningSocketsMixin`
|
||||
* :class:`~napalm_device_types.local_accounts.LocalAccountsMixin`
|
||||
* :class:`~napalm_device_types.host_firewall.HostFirewallMixin`
|
||||
* :class:`~napalm_device_types.mac_acl.MacAclMixin`
|
||||
* :class:`~napalm_device_types.nat_vpn.NatVpnMixin`
|
||||
@@ -54,10 +57,14 @@ instead of being restated on every role that happens to need it:
|
||||
* :class:`~napalm_device_types.ping_sweep.PingSweepMixin`
|
||||
* :class:`~napalm_device_types.service_clusters.ServiceClustersMixin`
|
||||
* :class:`~napalm_device_types.service_packages.ServicePackagesMixin`
|
||||
* :class:`~napalm_device_types.sudo_history.SudoHistoryMixin`
|
||||
* :class:`~napalm_device_types.scheduled_jobs.ScheduledJobsMixin`
|
||||
* :class:`~napalm_device_types.scheduled_jobs_posix.PosixScheduledJobsMixin`
|
||||
* :class:`~napalm_device_types.services.ServiceControlMixin`
|
||||
* :class:`~napalm_device_types.systemd.SystemdServicesMixin`
|
||||
* :class:`~napalm_device_types.updates.UpdateMixin`
|
||||
* :class:`~napalm_device_types.virtual_ips.VirtualIpsMixin`
|
||||
* :class:`~napalm_device_types.vm_stop_request.VmStopRequestMixin`
|
||||
|
||||
Introspection -- :func:`~napalm_device_types.roles.roles_of`,
|
||||
:func:`~napalm_device_types.roles.role_keys_of` and
|
||||
@@ -68,10 +75,28 @@ from napalm_device_types.access_point import AccessPointDriver
|
||||
from napalm_device_types.apt_proxy import (
|
||||
APT_PROXY_COMMAND,
|
||||
AptProxyMixin,
|
||||
https_proxy_for,
|
||||
parse_apt_fetch,
|
||||
parse_apt_proxy,
|
||||
same_proxy_url,
|
||||
)
|
||||
from napalm_device_types.arp import ARP_AGE_UNKNOWN
|
||||
from napalm_device_types.backup_server import BackupServerDriver
|
||||
from napalm_device_types.backups import (
|
||||
BACKUP_JOB_KINDS,
|
||||
BackupDatastoreDict,
|
||||
BackupGroupDict,
|
||||
BackupJobKind,
|
||||
BackupRemoteDict,
|
||||
BackupRunState,
|
||||
BackupServerJobDict,
|
||||
BackupServerRefDict,
|
||||
BackupSnapshotDict,
|
||||
BackupSnapshotRefDict,
|
||||
BackupTaskDict,
|
||||
BackupTaskStatusDict,
|
||||
BackupVerifyState,
|
||||
)
|
||||
from napalm_device_types.base import DeviceTypeDriver, FingerprintRule, PortSpec, rules_max_weight
|
||||
from napalm_device_types.channel import (
|
||||
ByteStream,
|
||||
@@ -81,6 +106,15 @@ from napalm_device_types.channel import (
|
||||
open_stream_on_transport,
|
||||
run_on_transport,
|
||||
)
|
||||
from napalm_device_types.clusters import (
|
||||
BackupDict,
|
||||
BackupJobDict,
|
||||
ClusterDict,
|
||||
ClusterNetworkDict,
|
||||
ClusterNodeDict,
|
||||
ClusterStorageDict,
|
||||
HAResourceDict,
|
||||
)
|
||||
from napalm_device_types.config_lifecycle import ConfigLifecycleMixin
|
||||
from napalm_device_types.config_sync import ConfigSyncMixin
|
||||
from napalm_device_types.container_engine import (
|
||||
@@ -95,6 +129,7 @@ from napalm_device_types.firewall_rules import FirewallRuleMixin
|
||||
from napalm_device_types.firmware import FirmwareOfferMixin
|
||||
from napalm_device_types.fixes import (
|
||||
EnsurePackageMixin,
|
||||
LldpAgentMixin,
|
||||
PackageProxyMixin,
|
||||
SnmpAgentMixin,
|
||||
run_fix_steps,
|
||||
@@ -106,6 +141,7 @@ from napalm_device_types.host_firewall import (
|
||||
parse_host_firewall,
|
||||
)
|
||||
from napalm_device_types.host_reboot import HostRebootMixin
|
||||
from napalm_device_types.host_shutdown import HostShutdownMixin
|
||||
from napalm_device_types.host_status import HOST_STATUS_COMMAND, HostStatusMixin, parse_host_status
|
||||
from napalm_device_types.hypervisor import HypervisorDriver
|
||||
from napalm_device_types.interface_filter import InterfaceFilterMixin
|
||||
@@ -118,6 +154,18 @@ from napalm_device_types.listening import (
|
||||
ListeningSocketsUnavailable,
|
||||
parse_listening_sockets,
|
||||
)
|
||||
from napalm_device_types.lldp import (
|
||||
LldpNeighborDict,
|
||||
LldpNeighborRow,
|
||||
lldp_neighbors,
|
||||
lldp_neighbors_detail,
|
||||
)
|
||||
from napalm_device_types.lldpctl import LLDPCTL_KEYVALUE_COMMAND, parse_lldpctl_keyvalue
|
||||
from napalm_device_types.local_accounts import (
|
||||
LocalAccountChangeDict,
|
||||
LocalAccountDict,
|
||||
LocalAccountsMixin,
|
||||
)
|
||||
from napalm_device_types.mac_acl import MacAclMixin
|
||||
from napalm_device_types.management_controller import ManagementControllerDriver
|
||||
from napalm_device_types.media import MediaDriver
|
||||
@@ -142,10 +190,6 @@ from napalm_device_types.models import (
|
||||
PowerAction,
|
||||
PowerStateDict,
|
||||
SnmpAgentSettingsDict,
|
||||
SnmpHealthProfile,
|
||||
SnmpInterfacesSpec,
|
||||
SnmpMemorySpec,
|
||||
SnmpValueSpec,
|
||||
UpdateProgressDict,
|
||||
)
|
||||
from napalm_device_types.nat_vpn import NatVpnMixin
|
||||
@@ -160,8 +204,20 @@ from napalm_device_types.package_updates import (
|
||||
from napalm_device_types.packages import PackageManagementMixin
|
||||
from napalm_device_types.phone import PhoneDriver
|
||||
from napalm_device_types.ping_sweep import PingSweepMixin, driver_supports_ping
|
||||
from napalm_device_types.provisioning import VMProvisionIncomplete
|
||||
from napalm_device_types.residential_gateway import ResidentialGatewayDriver
|
||||
from napalm_device_types.roles import primary_role_of, role_keys_of, roles_of
|
||||
from napalm_device_types.scheduled_jobs import (
|
||||
ScheduledJobChanged,
|
||||
ScheduledJobDict,
|
||||
ScheduledJobRefused,
|
||||
ScheduledJobResultDict,
|
||||
ScheduledJobsMixin,
|
||||
ScheduledJobSpecDict,
|
||||
ScheduledJobsReadingDict,
|
||||
job_digest,
|
||||
)
|
||||
from napalm_device_types.scheduled_jobs_posix import PosixScheduledJobsMixin
|
||||
from napalm_device_types.service_clusters import (
|
||||
SERVICE_CLUSTERS_COMMAND,
|
||||
ServiceClustersMixin,
|
||||
@@ -174,6 +230,12 @@ from napalm_device_types.service_packages import (
|
||||
parse_service_packages,
|
||||
)
|
||||
from napalm_device_types.services import ServiceControlMixin
|
||||
from napalm_device_types.snmp_health_profile import (
|
||||
SnmpHealthProfile,
|
||||
SnmpInterfacesSpec,
|
||||
SnmpMemorySpec,
|
||||
SnmpValueSpec,
|
||||
)
|
||||
from napalm_device_types.ssh_host_keys import (
|
||||
KNOWN_KEYS_ARG as SSH_HOST_KEYS_ARG,
|
||||
)
|
||||
@@ -194,6 +256,12 @@ from napalm_device_types.ssh_server import (
|
||||
plan_ssh_hardening,
|
||||
)
|
||||
from napalm_device_types.storage import StorageDriver
|
||||
from napalm_device_types.sudo_history import (
|
||||
HistoryCleanupDict,
|
||||
HistorySudoLinesDict,
|
||||
SudoHistoryCleanupDict,
|
||||
SudoHistoryMixin,
|
||||
)
|
||||
from napalm_device_types.switch import SwitchDriver
|
||||
from napalm_device_types.system_identity import (
|
||||
SYSTEM_IDENTITY_COMMAND,
|
||||
@@ -214,8 +282,31 @@ from napalm_device_types.virtual_ips import (
|
||||
VirtualIpsMixin,
|
||||
parse_virtual_ips,
|
||||
)
|
||||
from napalm_device_types.vm_migration import MigrationPreflightDict
|
||||
from napalm_device_types.vm_stop_request import VmStopRequestMixin
|
||||
|
||||
__all__ = [
|
||||
"BACKUP_JOB_KINDS",
|
||||
"BackupDatastoreDict",
|
||||
"BackupGroupDict",
|
||||
"BackupJobKind",
|
||||
"BackupRemoteDict",
|
||||
"BackupRunState",
|
||||
"BackupServerJobDict",
|
||||
"BackupServerRefDict",
|
||||
"BackupSnapshotDict",
|
||||
"BackupSnapshotRefDict",
|
||||
"BackupTaskDict",
|
||||
"BackupTaskStatusDict",
|
||||
"BackupVerifyState",
|
||||
"BackupDict",
|
||||
"BackupJobDict",
|
||||
"ClusterDict",
|
||||
"ClusterNetworkDict",
|
||||
"ClusterNodeDict",
|
||||
"ClusterStorageDict",
|
||||
"HAResourceDict",
|
||||
"MigrationPreflightDict",
|
||||
"HypervisorHostDict",
|
||||
"DnsForwardDict",
|
||||
"DnsForwardMixin",
|
||||
@@ -231,9 +322,11 @@ __all__ = [
|
||||
"FixResultDict",
|
||||
"FixStepDict",
|
||||
"FixStepOutcomeDict",
|
||||
"LldpAgentMixin",
|
||||
"PackageProxyMixin",
|
||||
"APT_PROXY_COMMAND",
|
||||
"AptProxyMixin",
|
||||
"https_proxy_for",
|
||||
"parse_apt_fetch",
|
||||
"parse_apt_proxy",
|
||||
"same_proxy_url",
|
||||
@@ -245,6 +338,7 @@ __all__ = [
|
||||
"SnmpMemorySpec",
|
||||
"SnmpValueSpec",
|
||||
"AccessPointDriver",
|
||||
"BackupServerDriver",
|
||||
"ByteStream",
|
||||
"CommandChannelMixin",
|
||||
"CommandResult",
|
||||
@@ -269,10 +363,12 @@ __all__ = [
|
||||
"HOST_STATUS_COMMAND",
|
||||
"HostStatusMixin",
|
||||
"HostRebootMixin",
|
||||
"HostShutdownMixin",
|
||||
"HypervisorDriver",
|
||||
"InterfaceFilterMixin",
|
||||
"MacAclMixin",
|
||||
"ManagementControllerDriver",
|
||||
"ARP_AGE_UNKNOWN",
|
||||
"POWER_ACTIONS",
|
||||
"PowerAction",
|
||||
"PowerStateDict",
|
||||
@@ -303,11 +399,29 @@ __all__ = [
|
||||
"ListeningSocketsMixin",
|
||||
"ListeningSocketsUnavailable",
|
||||
"parse_listening_sockets",
|
||||
"LLDPCTL_KEYVALUE_COMMAND",
|
||||
"LldpNeighborDict",
|
||||
"LldpNeighborRow",
|
||||
"lldp_neighbors",
|
||||
"lldp_neighbors_detail",
|
||||
"parse_lldpctl_keyvalue",
|
||||
"LocalAccountsMixin",
|
||||
"LocalAccountDict",
|
||||
"LocalAccountChangeDict",
|
||||
"PhoneDriver",
|
||||
"PingSweepMixin",
|
||||
"PortSpec",
|
||||
"ResidentialGatewayDriver",
|
||||
"ServiceControlMixin",
|
||||
"PosixScheduledJobsMixin",
|
||||
"ScheduledJobChanged",
|
||||
"ScheduledJobDict",
|
||||
"ScheduledJobRefused",
|
||||
"ScheduledJobResultDict",
|
||||
"ScheduledJobSpecDict",
|
||||
"ScheduledJobsMixin",
|
||||
"ScheduledJobsReadingDict",
|
||||
"job_digest",
|
||||
"StorageDriver",
|
||||
"SwitchDriver",
|
||||
"SYSTEMD_SERVICES_COMMAND",
|
||||
@@ -317,6 +431,8 @@ __all__ = [
|
||||
"UpdateMixin",
|
||||
"VIRTUAL_IPS_COMMAND",
|
||||
"VirtualIpsMixin",
|
||||
"VmStopRequestMixin",
|
||||
"VMProvisionIncomplete",
|
||||
"SERVICE_CLUSTERS_COMMAND",
|
||||
"ServiceClustersMixin",
|
||||
"parse_service_clusters",
|
||||
@@ -330,6 +446,10 @@ __all__ = [
|
||||
"SshHardeningPlan",
|
||||
"SshHardeningResult",
|
||||
"plan_ssh_hardening",
|
||||
"SudoHistoryMixin",
|
||||
"HistorySudoLinesDict",
|
||||
"HistoryCleanupDict",
|
||||
"SudoHistoryCleanupDict",
|
||||
"SSH_HOST_KEYS_ARG",
|
||||
"SSH_HOST_KEY_OBSERVER_ARG",
|
||||
"HostKeyCheck",
|
||||
|
||||
@@ -0,0 +1,222 @@
|
||||
"""Crontabs and anacrontabs as text: the jobs in them, and changing one line of them.
|
||||
|
||||
Pure: no I/O. :mod:`napalm_device_types.scheduled_jobs_posix` reads the files and
|
||||
writes them back; this module says what is in them and what the new text is.
|
||||
|
||||
**A line's key survives switching it off.** A job switched off is the same line
|
||||
behind the marker :data:`DISABLED_MARKER`; the key is built from the line without
|
||||
it, so the job keeps its key and the marker is all that changes. Two equal lines
|
||||
in one file get ``:2``, ``:3`` after the first.
|
||||
|
||||
**A job an application wrote** is the line after ``# napalm-scheduled-job: <name>``
|
||||
and is keyed ``job:<name>``, whatever its schedule.
|
||||
|
||||
**Everything else is left as it was**: comments, blank lines, variables, lines
|
||||
cron would not take.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import re
|
||||
from dataclasses import dataclass, field
|
||||
from shlex import quote
|
||||
|
||||
DISABLED_MARKER = "#napalm-disabled: "
|
||||
JOB_MARKER = "# napalm-scheduled-job: "
|
||||
|
||||
_VARIABLE = re.compile(r"^([A-Za-z_][A-Za-z0-9_]*)\s*=\s*(.*)$")
|
||||
_ANACRON_PERIOD = re.compile(r"^(\d+|@(daily|weekly|monthly|yearly|annually))$")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CronLine:
|
||||
"""One job in a crontab: where it is, what it says, and what was set before it."""
|
||||
|
||||
index: int # the line's number in the text, from 0
|
||||
key: str
|
||||
schedule: str
|
||||
run_as: str
|
||||
command: str
|
||||
enabled: bool
|
||||
env: dict[str, str] = field(default_factory=dict)
|
||||
name: str = "" # the name an application gave it
|
||||
description: str = ""
|
||||
|
||||
|
||||
def _unquote(value: str) -> str:
|
||||
value = value.strip()
|
||||
if len(value) >= 2 and value[0] == value[-1] and value[0] in "'\"":
|
||||
return value[1:-1]
|
||||
return value
|
||||
|
||||
|
||||
def _digest(text: str) -> str:
|
||||
return hashlib.sha256(" ".join(text.split()).encode()).hexdigest()[:8]
|
||||
|
||||
|
||||
def _split(content: str, *, system: bool, user: str) -> tuple[str, str, str] | None:
|
||||
"""(schedule, run_as, command) of a cron line; None for one cron would not take."""
|
||||
fields = 1 if content.startswith("@") else 5
|
||||
owner = 1 if system else 0
|
||||
parts = content.split(None, fields + owner)
|
||||
if len(parts) < fields + owner + 1:
|
||||
return None
|
||||
schedule = " ".join(parts[:fields])
|
||||
run_as = parts[fields] if system else user
|
||||
return schedule, run_as, parts[-1]
|
||||
|
||||
|
||||
def _anacron_split(content: str) -> tuple[str, str, str] | None:
|
||||
"""(period, job identifier, command) of an anacrontab line."""
|
||||
parts = content.split(None, 3)
|
||||
if len(parts) < 4 or not _ANACRON_PERIOD.match(parts[0]) or not parts[1].isdigit():
|
||||
return None
|
||||
return parts[0], parts[2], parts[3]
|
||||
|
||||
|
||||
@dataclass
|
||||
class _Scan:
|
||||
"""What is set while a text is read line by line."""
|
||||
|
||||
env: dict[str, str] = field(default_factory=dict)
|
||||
seen: dict[str, int] = field(default_factory=dict)
|
||||
name: str = ""
|
||||
description: str = ""
|
||||
|
||||
def key(self, base: str) -> str:
|
||||
n = self.seen.get(base, 0) + 1
|
||||
self.seen[base] = n
|
||||
return base if n == 1 else f"{base}:{n}"
|
||||
|
||||
def take_name(self) -> tuple[str, str]:
|
||||
named = self.name, self.description
|
||||
self.name, self.description = "", ""
|
||||
return named
|
||||
|
||||
|
||||
def _content(raw: str) -> tuple[str, bool] | None:
|
||||
"""The job text of *raw* and whether it is on; None for a comment or a blank."""
|
||||
line = raw.strip()
|
||||
if line.startswith(DISABLED_MARKER.strip()):
|
||||
return line[len(DISABLED_MARKER.strip()) :].strip(), False
|
||||
if not line or line.startswith("#"):
|
||||
return None
|
||||
return line, True
|
||||
|
||||
|
||||
def _remember(scan: _Scan, raw: str) -> bool:
|
||||
"""Take a variable or an application's marker from *raw*; whether it was one."""
|
||||
line = raw.strip()
|
||||
if line.startswith(JOB_MARKER.strip()):
|
||||
name, _, description = line[len(JOB_MARKER.strip()) :].strip().partition(" ")
|
||||
scan.name, scan.description = name, description.strip()
|
||||
return True
|
||||
variable = _VARIABLE.match(line)
|
||||
if variable:
|
||||
scan.env[variable[1]] = _unquote(variable[2])
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def parse_crontab(path: str, text: str, *, system: bool, user: str = "") -> list[CronLine]:
|
||||
"""The jobs of the crontab *text* at *path*.
|
||||
|
||||
*system*: a sixth field names the user (``/etc/crontab``, ``/etc/cron.d``);
|
||||
otherwise every job runs as *user*.
|
||||
"""
|
||||
scan, jobs = _Scan(), []
|
||||
for index, raw in enumerate(text.splitlines()):
|
||||
if _remember(scan, raw):
|
||||
continue
|
||||
found = _content(raw)
|
||||
split = _split(found[0], system=system, user=user) if found else None
|
||||
if found is None or split is None:
|
||||
continue
|
||||
content, enabled = found
|
||||
name, description = scan.take_name()
|
||||
key = f"job:{name}" if name else scan.key(f"cron:{path}:{_digest(content)}")
|
||||
schedule, run_as, command = split
|
||||
jobs.append(
|
||||
CronLine(
|
||||
index, key, schedule, run_as, command, enabled, dict(scan.env), name, description
|
||||
)
|
||||
)
|
||||
return jobs
|
||||
|
||||
|
||||
def parse_anacrontab(text: str) -> list[CronLine]:
|
||||
"""The jobs of an anacrontab: period, delay, identifier, command; all run as root."""
|
||||
scan, jobs = _Scan(), []
|
||||
for index, raw in enumerate(text.splitlines()):
|
||||
if _remember(scan, raw):
|
||||
continue
|
||||
found = _content(raw)
|
||||
split = _anacron_split(found[0]) if found else None
|
||||
if found is None or split is None:
|
||||
continue
|
||||
period, identifier, command = split
|
||||
key = scan.key(f"anacron:{identifier}")
|
||||
jobs.append(CronLine(index, key, period, "root", command, found[1], dict(scan.env)))
|
||||
return jobs
|
||||
|
||||
|
||||
def _replace(text: str, index: int, lines: list[str]) -> str:
|
||||
rows = text.splitlines()
|
||||
rows[index : index + 1] = lines
|
||||
return "\n".join(rows) + "\n"
|
||||
|
||||
|
||||
def switched(text: str, line: CronLine, *, enabled: bool) -> str:
|
||||
"""*text* with *line* switched on or off: the marker added or taken away, nothing else."""
|
||||
raw = text.splitlines()[line.index]
|
||||
found = _content(raw)
|
||||
if found is None: # pragma: no cover - a CronLine always points at a job
|
||||
raise ValueError(f"line {line.index} is no job")
|
||||
content = found[0]
|
||||
return _replace(text, line.index, [content if enabled else DISABLED_MARKER + content])
|
||||
|
||||
|
||||
def marked_lines(name: str, description: str, line: str, *, enabled: bool) -> list[str]:
|
||||
"""An application's job as crontab lines: its marker, then the job, maybe switched off."""
|
||||
marker = f"{JOB_MARKER}{name} {description}".rstrip()
|
||||
return [marker, line if enabled else DISABLED_MARKER + line]
|
||||
|
||||
|
||||
def with_job(text: str, jobs: list[CronLine], name: str, lines: list[str]) -> str:
|
||||
"""*text* with the application's job *name* as *lines*: replaced, or added at the end."""
|
||||
current = next((j for j in jobs if j.name == name), None)
|
||||
if current is None:
|
||||
body = text if not text or text.endswith("\n") else text + "\n"
|
||||
return body + "\n".join(lines) + "\n"
|
||||
rows = text.splitlines()
|
||||
rows[current.index - 1 : current.index + 1] = lines
|
||||
return "\n".join(rows) + "\n"
|
||||
|
||||
|
||||
def without_job(text: str, jobs: list[CronLine], name: str) -> str:
|
||||
"""*text* without the application's job *name* and its marker."""
|
||||
current = next(j for j in jobs if j.name == name)
|
||||
rows = text.splitlines()
|
||||
del rows[current.index - 1 : current.index + 1]
|
||||
return "\n".join(rows) + ("\n" if rows else "")
|
||||
|
||||
|
||||
def _unescape(part: str) -> str:
|
||||
return part.replace("\\%", "%")
|
||||
|
||||
|
||||
def run_script(command: str, env: dict[str, str]) -> str:
|
||||
"""*command* as ``sh -c`` takes it, the way cron would run it.
|
||||
|
||||
The crontab's variables are exported first. The first ``%`` that is not
|
||||
escaped ends the command; what follows is its stdin, with every further
|
||||
``%`` a newline, and ``\\%`` is a ``%``.
|
||||
"""
|
||||
parts = re.split(r"(?<!\\)%", command)
|
||||
exports = "".join(f"export {k}={quote(v)}; " for k, v in env.items() if k != "MAILTO")
|
||||
head = _unescape(parts[0])
|
||||
if len(parts) == 1:
|
||||
return exports + head
|
||||
stdin = "\n".join(_unescape(p) for p in parts[1:])
|
||||
return f"{exports}printf '%s\\n' {quote(stdin)} | {{ {head}\n}}"
|
||||
@@ -8,12 +8,9 @@ name a single entry, and guessing would overwrite one the caller never meant.
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable, Iterable
|
||||
from typing import TypeVar
|
||||
|
||||
T = TypeVar("T")
|
||||
|
||||
|
||||
def index_live(entries: Iterable[T], key: Callable[[T], str]) -> tuple[dict[str, T], set[str]]:
|
||||
def index_live[T](entries: Iterable[T], key: Callable[[T], str]) -> tuple[dict[str, T], set[str]]:
|
||||
"""Live entries by key, and the keys more than one of them holds.
|
||||
|
||||
Entries without a key are left out: they are not the caller's to match.
|
||||
@@ -32,7 +29,9 @@ def index_live(entries: Iterable[T], key: Callable[[T], str]) -> tuple[dict[str,
|
||||
return by_key, shared
|
||||
|
||||
|
||||
def check_desired(entries: list[T], key: Callable[[T], str], shared: set[str], what: str) -> None:
|
||||
def check_desired[T](
|
||||
entries: list[T], key: Callable[[T], str], shared: set[str], what: str
|
||||
) -> None:
|
||||
"""``ValueError`` for a desired entry with no key, a key given twice, or a
|
||||
key that names more than one live entry."""
|
||||
seen: set[str] = set()
|
||||
|
||||
@@ -0,0 +1,417 @@
|
||||
"""The scheduled-jobs report of a POSIX host: the command that prints it, and its parse.
|
||||
|
||||
:mod:`napalm_device_types.scheduled_jobs_posix` sends :func:`reading_command` and
|
||||
hands what came back to :func:`parse_reading`. Pure: no I/O.
|
||||
|
||||
**One framed report, as root.** Every crontab, cron.d file, anacrontab and job
|
||||
script is printed behind a header line and with ``|`` before each of its lines,
|
||||
so no line of a file can pass for a header; run-parts scripts by their execute
|
||||
bit and checksum; systemd's timers and the units they start by ``systemctl show``;
|
||||
then the time zone and which package owns each definition file. A report whose
|
||||
end marker is missing raises: a list cut short must never read as jobs that went
|
||||
away.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import UTC, datetime
|
||||
from shlex import quote
|
||||
|
||||
from napalm_device_types._crontab import JOB_MARKER, CronLine, parse_anacrontab, parse_crontab
|
||||
from napalm_device_types.scheduled_jobs import DELETE, EDIT, job_digest
|
||||
from napalm_device_types.terminal import strip_terminal_codes
|
||||
|
||||
_BEGIN = "SJ_BEGIN"
|
||||
_END = "SJ_END"
|
||||
SCHEDULE_HEADER = "# schedule: "
|
||||
|
||||
TIMER_PROPERTIES = (
|
||||
"Id,Unit,Description,TimersCalendar,TimersMonotonic,NextElapseUSecRealtime,"
|
||||
"LastTriggerUSec,UnitFileState,ActiveState,FragmentPath"
|
||||
)
|
||||
UNIT_PROPERTIES = "Id,ExecStart,User,Result,FragmentPath"
|
||||
|
||||
#: The names run-parts runs (Debian's ``--lsbsysinit``-less default).
|
||||
_RUN_PARTS_NAME = re.compile(r"^[A-Za-z0-9_-]+$")
|
||||
_SECTION = re.compile(r"^\[([a-z-]+)(?: (.*))?\]$")
|
||||
|
||||
#: Reads definition paths on stdin, prints ``<package>: <path>`` for each a package owns.
|
||||
OWNERS_SH = (
|
||||
"if command -v dpkg >/dev/null 2>&1; then xargs dpkg -S 2>/dev/null; "
|
||||
"elif command -v rpm >/dev/null 2>&1; then while read -r p; do "
|
||||
'o=$(rpm -qf --qf \'%{NAME}\' -- "$p" 2>/dev/null) && printf \'%s: %s\\n\' "$o" "$p"; done; '
|
||||
"elif command -v apk >/dev/null 2>&1; then while read -r p; do "
|
||||
'o=$(apk info -qW -- "$p" 2>/dev/null) && [ -n "$o" ] && printf \'%s: %s\\n\' "$o" "$p"; done; '
|
||||
"else cat >/dev/null; fi"
|
||||
)
|
||||
|
||||
|
||||
#: Prints the host's time zone: an IANA name, or a path ending in ``zoneinfo/<name>``.
|
||||
TIME_ZONE_SH = "cat /etc/timezone 2>/dev/null || readlink /etc/localtime 2>/dev/null"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Paths:
|
||||
"""Where a host keeps its scheduled jobs."""
|
||||
|
||||
system_files: tuple[str, ...]
|
||||
cron_d_dirs: tuple[str, ...]
|
||||
spool_dirs: tuple[str, ...]
|
||||
run_parts: tuple[tuple[str, str], ...] # (period, directory)
|
||||
anacrontab: str
|
||||
script_dir: str
|
||||
systemd_marker: str = "/run/systemd/system"
|
||||
owners_sh: str = OWNERS_SH
|
||||
time_zone_sh: str = TIME_ZONE_SH
|
||||
|
||||
|
||||
@dataclass
|
||||
class Reading:
|
||||
"""What one report said, beyond the jobs: what writes need to know."""
|
||||
|
||||
jobs: list[dict[str, object]] = field(default_factory=list)
|
||||
time_zone: str | None = None
|
||||
systemd: bool = False
|
||||
#: per key: the unit a timer starts, the crontab line a cron job is
|
||||
units: dict[str, str] = field(default_factory=dict)
|
||||
lines: dict[str, CronLine] = field(default_factory=dict)
|
||||
|
||||
|
||||
def _q(path: str) -> str:
|
||||
return quote(path)
|
||||
|
||||
|
||||
def _files(section: str, pattern: str) -> str:
|
||||
return (
|
||||
f'for f in {pattern}; do [ -f "$f" ] || continue; printf \'[%s %s]\\n\' {section} "$f"; '
|
||||
# GNU sed keeps a missing last newline; without one of its own the next
|
||||
# header would run on into the file's last line.
|
||||
'sed \'s/^/|/\' "$f"; [ -n "$(tail -c 1 "$f")" ] && echo; P="$P\n$f"; done; '
|
||||
)
|
||||
|
||||
|
||||
def _run_parts(period: str, directory: str) -> str:
|
||||
return (
|
||||
f'for f in {_q(directory)}/*; do [ -f "$f" ] || continue; '
|
||||
'[ -x "$f" ] && x=1 || x=0; c=$(cksum < "$f" | cut -d" " -f1); '
|
||||
f'printf \'[run-parts %s %s %s %s]\\n\' {period} "$x" "$c" "$f"; P="$P\n$f"; done; '
|
||||
)
|
||||
|
||||
|
||||
def _timers(paths: Paths) -> str:
|
||||
show = f"systemctl show --no-pager -p {TIMER_PROPERTIES}"
|
||||
units = f"systemctl show --no-pager -p {UNIT_PROPERTIES}"
|
||||
return (
|
||||
f"if [ -d {_q(paths.systemd_marker)} ]; then echo '[systemd]'; "
|
||||
"t=$(systemctl list-unit-files --type=timer --no-legend --no-pager 2>/dev/null "
|
||||
"| awk '$1 !~ /@\\./ {print $1}'); "
|
||||
'if [ -n "$t" ]; then '
|
||||
f"s=$({show} --timestamp=unix $t 2>/dev/null || {show} $t 2>/dev/null); "
|
||||
"echo '[timers]'; printf '%s\\n' \"$s\" | sed 's/^/|/'; "
|
||||
"u=$(printf '%s\\n' \"$s\" | sed -n 's/^Unit=//p'); "
|
||||
f"echo '[units]'; {units} $u 2>/dev/null | sed 's/^/|/'; "
|
||||
"P=\"$P\n$(printf '%s\\n' \"$s\" | sed -n 's/^FragmentPath=//p')\"; fi; fi; "
|
||||
)
|
||||
|
||||
|
||||
def reading_command(paths: Paths) -> str:
|
||||
"""One POSIX ``sh`` script that prints every scheduled job of the host; run it as root."""
|
||||
parts = ["printf '%s%s\\n' SJ_ BEGIN; P=''; "]
|
||||
parts += [_files("system", _q(f)) for f in paths.system_files]
|
||||
parts += [_files("system", f"{_q(d)}/*") for d in paths.cron_d_dirs]
|
||||
parts += [_files("user", f"{_q(d)}/*") for d in paths.spool_dirs]
|
||||
parts.append(_files("anacron", _q(paths.anacrontab)))
|
||||
parts += [_run_parts(period, d) for period, d in paths.run_parts]
|
||||
parts.append(_files("script", f"{_q(paths.script_dir)}/*"))
|
||||
parts.append(_timers(paths))
|
||||
parts.append(
|
||||
f"echo '[tz]'; {{ {paths.time_zone_sh}; }}; "
|
||||
f"echo '[owners]'; printf '%s\\n' \"$P\" | sed '/^$/d' | {{ {paths.owners_sh}; }}; "
|
||||
"printf '%s%s\\n' SJ_ END"
|
||||
)
|
||||
return "".join(parts)
|
||||
|
||||
|
||||
# -- parse ------------------------------------------------------------------------------------
|
||||
|
||||
|
||||
@dataclass
|
||||
class _Section:
|
||||
name: str
|
||||
args: str
|
||||
lines: list[str] = field(default_factory=list)
|
||||
|
||||
|
||||
def _sections(output: str) -> list[_Section]:
|
||||
text = strip_terminal_codes(output)
|
||||
begin, end = text.rfind(_BEGIN), text.rfind(_END)
|
||||
if begin < 0 or end < begin:
|
||||
raise ValueError("the scheduled-jobs report is cut short (no end marker)")
|
||||
sections: list[_Section] = []
|
||||
for line in text[begin + len(_BEGIN) : end].splitlines():
|
||||
header = _SECTION.match(line)
|
||||
if header:
|
||||
sections.append(_Section(header[1], header[2] or ""))
|
||||
elif sections:
|
||||
sections[-1].lines.append(line[1:] if line.startswith("|") else line)
|
||||
return sections
|
||||
|
||||
|
||||
def _owned(lines: list[str]) -> set[str]:
|
||||
owned: set[str] = set()
|
||||
for line in lines:
|
||||
package, sep, path = line.partition(": ")
|
||||
if sep and path.startswith("/") and not package.startswith("diversion"):
|
||||
owned.add(path.strip())
|
||||
return owned
|
||||
|
||||
|
||||
def _time_zone(lines: list[str]) -> str | None:
|
||||
value = next((ln.strip() for ln in lines if ln.strip()), "")
|
||||
if "zoneinfo/" in value:
|
||||
value = value.split("zoneinfo/", 1)[1]
|
||||
return value or None
|
||||
|
||||
|
||||
def _blocks(lines: list[str]) -> list[dict[str, list[str]]]:
|
||||
blocks: list[dict[str, list[str]]] = [{}]
|
||||
for line in lines:
|
||||
if not line.strip():
|
||||
blocks.append({})
|
||||
continue
|
||||
key, sep, value = line.partition("=")
|
||||
if sep:
|
||||
blocks[-1].setdefault(key, []).append(value)
|
||||
return [b for b in blocks if b.get("Id")]
|
||||
|
||||
|
||||
def _one(block: dict[str, list[str]], key: str) -> str:
|
||||
return (block.get(key) or [""])[0].strip()
|
||||
|
||||
|
||||
def _timestamp(value: str) -> str | None:
|
||||
"""systemd's ``@<seconds>`` (or a UTC time) as ISO 8601; None when it never was."""
|
||||
value = value.strip()
|
||||
if value.startswith("@") and value[1:].isdigit() and int(value[1:]) > 0:
|
||||
return datetime.fromtimestamp(int(value[1:]), UTC).isoformat()
|
||||
match = re.search(r"(\d{4}-\d\d-\d\d \d\d:\d\d:\d\d) UTC$", value)
|
||||
if match:
|
||||
return datetime.strptime(match[1], "%Y-%m-%d %H:%M:%S").replace(tzinfo=UTC).isoformat()
|
||||
return None
|
||||
|
||||
|
||||
def _timer_entries(block: dict[str, list[str]], key: str) -> list[str]:
|
||||
"""``{ OnCalendar=daily ; next_elapse=… }`` → ``OnCalendar=daily``, one per entry.
|
||||
|
||||
``systemctl show`` names a monotonic setting by its property (``OnUnitActiveUSec``);
|
||||
it is given as the unit file says it (``OnUnitActiveSec``).
|
||||
"""
|
||||
entries = (v.strip(" {}").split(" ; ")[0].strip() for v in block.get(key, []) if v.strip())
|
||||
return [re.sub(r"^(On\w+)USec=", r"\1Sec=", e) for e in entries]
|
||||
|
||||
|
||||
def _argv(exec_start: list[str]) -> str:
|
||||
commands = []
|
||||
for value in exec_start:
|
||||
match = re.search(r"argv\[\]=(.*?) ; ", value)
|
||||
if match:
|
||||
commands.append(match[1].strip())
|
||||
return "; ".join(commands)
|
||||
|
||||
|
||||
@dataclass
|
||||
class _Script:
|
||||
name: str
|
||||
description: str
|
||||
schedule: str
|
||||
command: str
|
||||
|
||||
|
||||
def _script(path: str, lines: list[str]) -> _Script | None:
|
||||
"""An application's job script: its marker, schedule and command; None for any other file."""
|
||||
marker = next((ln for ln in lines if ln.startswith(JOB_MARKER)), None)
|
||||
schedule = next((ln for ln in lines if ln.startswith(SCHEDULE_HEADER)), None)
|
||||
if marker is None or schedule is None:
|
||||
return None
|
||||
name, _, description = marker[len(JOB_MARKER) :].partition(" ")
|
||||
body = [ln for ln in lines if ln not in (marker, schedule) and not ln.startswith("#!")]
|
||||
return _Script(
|
||||
name, description.strip(), schedule[len(SCHEDULE_HEADER) :], "\n".join(body).strip()
|
||||
)
|
||||
|
||||
|
||||
def _job(
|
||||
*,
|
||||
key: str,
|
||||
kind: str,
|
||||
source: str,
|
||||
run_as: str,
|
||||
schedule: str,
|
||||
schedule_kind: str,
|
||||
command: str,
|
||||
enabled: bool,
|
||||
managed_by: str,
|
||||
actions: list[str],
|
||||
content: str = "",
|
||||
**extra: object,
|
||||
) -> dict[str, object]:
|
||||
job: dict[str, object] = {
|
||||
"key": key,
|
||||
"kind": kind,
|
||||
"source": source,
|
||||
"run_as": run_as,
|
||||
"schedule": schedule,
|
||||
"schedule_kind": schedule_kind,
|
||||
"command": command,
|
||||
"enabled": enabled,
|
||||
"managed_by": managed_by,
|
||||
"actions": actions,
|
||||
}
|
||||
job["digest"] = job_digest(job, content=content)
|
||||
job.update({k: v for k, v in extra.items() if v not in (None, "")})
|
||||
return job
|
||||
|
||||
|
||||
_SWITCH = ["enable", "disable", "run"]
|
||||
_OWN = [*_SWITCH, EDIT, DELETE]
|
||||
|
||||
|
||||
def _cron_jobs(section: _Section, owned: set[str], reading: Reading) -> None:
|
||||
path = section.args
|
||||
text = "\n".join(section.lines)
|
||||
if section.name == "anacron":
|
||||
found = parse_anacrontab(text)
|
||||
else:
|
||||
user = path.rsplit("/", 1)[-1]
|
||||
found = parse_crontab(path, text, system=section.name == "system", user=user)
|
||||
kind = "anacron" if section.name == "anacron" else "cron"
|
||||
for line in found:
|
||||
own = bool(line.name)
|
||||
reading.lines[line.key] = line
|
||||
reading.jobs.append(
|
||||
_job(
|
||||
key=line.key,
|
||||
kind=kind,
|
||||
source=path,
|
||||
run_as=line.run_as,
|
||||
schedule=line.schedule,
|
||||
schedule_kind="period" if kind == "anacron" else "cron",
|
||||
command=line.command,
|
||||
enabled=line.enabled,
|
||||
managed_by="application" if own else ("package" if path in owned else "admin"),
|
||||
actions=_OWN if own else _SWITCH,
|
||||
name=line.name,
|
||||
description=line.description,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _run_parts_job(section: _Section, owned: set[str], reading: Reading) -> None:
|
||||
period, executable, checksum, path = section.args.split(" ", 3)
|
||||
if not _RUN_PARTS_NAME.match(path.rsplit("/", 1)[-1]):
|
||||
return
|
||||
reading.jobs.append(
|
||||
_job(
|
||||
key=f"run-parts:{path}",
|
||||
kind="run_parts",
|
||||
source=path,
|
||||
run_as="root",
|
||||
schedule=period,
|
||||
schedule_kind="period",
|
||||
command=path,
|
||||
enabled=executable == "1",
|
||||
managed_by="package" if path in owned else "admin",
|
||||
actions=_SWITCH,
|
||||
content=checksum,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _timer_job(
|
||||
timer: dict[str, list[str]],
|
||||
unit: dict[str, list[str]],
|
||||
scripts: dict[str, _Script],
|
||||
owned: set[str],
|
||||
) -> dict[str, object]:
|
||||
fragment = _one(timer, "FragmentPath")
|
||||
calendar = _timer_entries(timer, "TimersCalendar")
|
||||
schedule = "; ".join(calendar + _timer_entries(timer, "TimersMonotonic"))
|
||||
command = _argv(unit.get("ExecStart", []))
|
||||
script = scripts.get(command.split(" ", 1)[0])
|
||||
last_run = _timestamp(_one(timer, "LastTriggerUSec"))
|
||||
switchable = _one(timer, "UnitFileState") not in ("static", "generated", "transient")
|
||||
common = {
|
||||
"kind": "systemd_timer",
|
||||
"source": fragment,
|
||||
"run_as": _one(unit, "User") or "root",
|
||||
"enabled": _one(timer, "ActiveState") == "active",
|
||||
"last_run": last_run,
|
||||
"last_result": _one(unit, "Result") or None if last_run else None,
|
||||
"next_run": _timestamp(_one(timer, "NextElapseUSecRealtime")),
|
||||
}
|
||||
if script is not None:
|
||||
return _job(
|
||||
key=f"job:{script.name}",
|
||||
schedule=script.schedule,
|
||||
schedule_kind="cron",
|
||||
command=script.command,
|
||||
managed_by="application",
|
||||
actions=_OWN,
|
||||
name=script.name,
|
||||
description=script.description,
|
||||
**common, # type: ignore[arg-type]
|
||||
)
|
||||
packaged = fragment in owned or fragment.startswith(("/usr/lib/", "/lib/"))
|
||||
return _job(
|
||||
key=f"timer:{_one(timer, 'Id')}",
|
||||
schedule=schedule,
|
||||
schedule_kind="calendar" if calendar else "monotonic",
|
||||
command=command,
|
||||
managed_by="package" if packaged else "admin",
|
||||
actions=_SWITCH if switchable else ["run"],
|
||||
description=_one(timer, "Description"),
|
||||
**common, # type: ignore[arg-type]
|
||||
)
|
||||
|
||||
|
||||
def _timer_jobs(
|
||||
sections: dict[str, _Section], scripts: dict[str, _Script], owned: set[str], reading: Reading
|
||||
) -> None:
|
||||
timers = _blocks(sections["timers"].lines) if "timers" in sections else []
|
||||
units = (
|
||||
{_one(b, "Id"): b for b in _blocks(sections["units"].lines)} if "units" in sections else {}
|
||||
)
|
||||
for timer in timers:
|
||||
unit = units.get(_one(timer, "Unit"), {})
|
||||
job = _timer_job(timer, unit, scripts, owned)
|
||||
reading.units[str(job["key"])] = _one(timer, "Unit")
|
||||
reading.jobs.append(job)
|
||||
|
||||
|
||||
def parse_reading(output: str) -> Reading:
|
||||
"""The jobs, time zone and write context of a report from :func:`reading_command`.
|
||||
|
||||
:raises ValueError: when the report is cut short.
|
||||
"""
|
||||
sections = _sections(output)
|
||||
named = {s.name: s for s in sections if s.name in ("timers", "units", "tz", "owners")}
|
||||
owned = _owned(named["owners"].lines) if "owners" in named else set()
|
||||
reading = Reading(
|
||||
time_zone=_time_zone(named["tz"].lines) if "tz" in named else None,
|
||||
systemd=any(s.name == "systemd" for s in sections),
|
||||
)
|
||||
for section in sections:
|
||||
if section.name in ("system", "user", "anacron"):
|
||||
_cron_jobs(section, owned, reading)
|
||||
elif section.name == "run-parts":
|
||||
_run_parts_job(section, owned, reading)
|
||||
_timer_jobs(named, _scripts(sections), owned, reading)
|
||||
return reading
|
||||
|
||||
|
||||
def _scripts(sections: list[_Section]) -> dict[str, _Script]:
|
||||
"""An application's job scripts, by path."""
|
||||
scripts = ((s.args, _script(s.args, s.lines)) for s in sections if s.name == "script")
|
||||
return {path: script for path, script in scripts if script is not None}
|
||||
@@ -12,12 +12,18 @@ Debian installer writes its proxy into ``apt.conf``, Ubuntu's into
|
||||
(NetOrk/netork#1397). ``DIRECT`` for the proxy, or for one host
|
||||
(``Acquire::http::Proxy::<host>``), means no proxy there.
|
||||
|
||||
**https** goes through the http proxy too. apt's https method asks
|
||||
``Acquire::https::Proxy`` first and ``Acquire::http::Proxy`` after it, for one
|
||||
host and then for all (NetOrk/netork#1811). An http proxy alone therefore takes
|
||||
every https source as well, and a cache that does not tunnel them answers 403.
|
||||
|
||||
**Where it is set** is every line of apt's files that sets the http proxy, in the
|
||||
order apt reads them. A line of the form ``Acquire::http::Proxy "…";`` can be
|
||||
commented out; a setting inside a block (``Acquire::http { Proxy "…"; };``) is
|
||||
named but left alone. apt skips a file in ``apt.conf.d`` whose name has other
|
||||
characters than letters, digits, ``_``, ``-`` and ``.``, or an extension other
|
||||
than ``.conf`` (``00proxy.bak``, ``*.dpkg-old``); so is it here.
|
||||
order apt reads them; the https proxy's lines are named apart. A line of the form
|
||||
``Acquire::http::Proxy "…";`` can be commented out; a setting inside a block
|
||||
(``Acquire::http { Proxy "…"; };``) is named but left alone. apt skips a file in
|
||||
``apt.conf.d`` whose name has other characters than letters, digits, ``_``, ``-``
|
||||
and ``.``, or an extension other than ``.conf`` (``00proxy.bak``, ``*.dpkg-old``);
|
||||
so is it here.
|
||||
|
||||
**Whether a proxy answers** is a fetch with apt's own downloader,
|
||||
``apt-helper download-file``, which is on every apt host and needs no root:
|
||||
@@ -29,13 +35,15 @@ page is the right one, the caller says (``expect``).
|
||||
out every other line that sets a different one, reads the result back from
|
||||
``apt-config``, and fetches the ``InRelease`` of one http source through it. The
|
||||
last step is what proves it: apt reached a mirror through the proxy it is now set
|
||||
to. The source chosen is one apt does not fetch ``DIRECT``.
|
||||
to. The source chosen is one apt does not fetch ``DIRECT``. Then it fetches one
|
||||
https source the same way; where the proxy refuses it, https goes ``DIRECT``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import shlex
|
||||
from collections.abc import Callable
|
||||
from typing import TYPE_CHECKING
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
@@ -47,7 +55,7 @@ from napalm_device_types.terminal import strip_terminal_codes
|
||||
|
||||
|
||||
class AptProxySettingDict(TypedDict):
|
||||
"""One line of apt's configuration that sets the http proxy (``AptProxyMixin``)."""
|
||||
"""One line of apt's configuration that sets a proxy (``AptProxyMixin``)."""
|
||||
|
||||
file: str
|
||||
line: int # 1-based
|
||||
@@ -68,6 +76,14 @@ class AptProxyReportDict(TypedDict):
|
||||
auto_detect: str | None
|
||||
#: Every line that sets it, in the order apt reads them; the last one wins.
|
||||
settings: list[AptProxySettingDict]
|
||||
#: The proxy apt takes for https: ``Acquire::https::Proxy``, else the http one;
|
||||
#: None for none or ``DIRECT``.
|
||||
https_proxy: str | None
|
||||
#: The per-host proxies https takes: ``Acquire::https::Proxy::<host>``, else
|
||||
#: ``Acquire::http::Proxy::<host>``.
|
||||
https_per_host: dict[str, str]
|
||||
#: Every line that sets ``Acquire::https::Proxy``, in the order apt reads them.
|
||||
https_settings: list[AptProxySettingDict]
|
||||
#: The base URIs of the host's sources over http and over https.
|
||||
http_sources: list[str]
|
||||
https_sources: list[str]
|
||||
@@ -85,6 +101,15 @@ class ProxyFetchDict(TypedDict):
|
||||
error: str
|
||||
|
||||
|
||||
class HttpsSourceFetchDict(ProxyFetchDict):
|
||||
"""One https source fetched the way apt fetches it (``check_https_source``)."""
|
||||
|
||||
#: What was fetched: the source's ``InRelease``.
|
||||
url: str
|
||||
#: The proxy apt took for it.
|
||||
proxy: str
|
||||
|
||||
|
||||
_BEGIN = "APROXY_BEGIN"
|
||||
_END = "APROXY_END"
|
||||
_APT_HELPER = "/usr/lib/apt/apt-helper"
|
||||
@@ -98,7 +123,7 @@ PROXY_URL = re.compile(r"^https?://[A-Za-z0-9.\-\[\]:@%_~/]+$")
|
||||
#: them early.
|
||||
APT_PROXY_COMMAND = (
|
||||
"( export LC_ALL=C; printf '%s%s\\n' APROXY_ BEGIN; "
|
||||
"echo '[dump]'; apt-config dump 2>/dev/null | grep -i '^Acquire::http::Proxy'; "
|
||||
"echo '[dump]'; apt-config dump 2>/dev/null | grep -i -E '^Acquire::https?::Proxy'; "
|
||||
'eval "$(apt-config shell ap_parts Dir::Etc::parts/d ap_main Dir::Etc::main/f 2>/dev/null)"; '
|
||||
"echo '[parts]'; for f in \"${ap_parts:-/etc/apt/apt.conf.d/}\"*; do "
|
||||
'[ -f "$f" ] && grep -Hin proxy "$f"; done; '
|
||||
@@ -108,12 +133,15 @@ APT_PROXY_COMMAND = (
|
||||
"printf '%s%s\\n' APROXY_ END ) 2>/dev/null | cat"
|
||||
)
|
||||
|
||||
_DUMP = re.compile(r'^Acquire::http::Proxy(::\S+|-Auto-Detect)?\s+"([^"]*)"', re.I)
|
||||
_DUMP = re.compile(r'^Acquire::(https?)::Proxy(::\S+|-Auto-Detect)?\s+"([^"]*)"', re.I)
|
||||
_GREP = re.compile(r"^(/[^:]+):(\d+):(.*)$")
|
||||
_FLAT = re.compile(r'^\s*Acquire::http::Proxy\s+"([^"]*)"\s*;', re.I)
|
||||
_NOT_HTTP_PROXY = re.compile(
|
||||
r"^\s*(//|#)|Acquire::(https|ftp|socks)\b|::Proxy::|Proxy-Auto-Detect", re.I
|
||||
)
|
||||
_FLAT_HTTPS = re.compile(r'^\s*Acquire::https::Proxy\s+"([^"]*)"\s*;', re.I)
|
||||
#: A line that sets the https proxy inside a block, all on the line itself.
|
||||
_HTTPS_BLOCK = re.compile(r'^\s*Acquire::https\s*\{[^}]*\bProxy\s+"([^"]*)"', re.I)
|
||||
_IN_BLOCK = re.compile(r'\bProxy\s+"([^"]*)"', re.I)
|
||||
#: The names apt reads in ``apt.conf.d``: no extension, or ``.conf``.
|
||||
_PART_NAME = re.compile(r"^(?:[A-Za-z0-9_\-]+|[A-Za-z0-9_.\-]+\.conf)$")
|
||||
@@ -138,13 +166,22 @@ def _sections(output: str) -> dict[str, list[str]]:
|
||||
return sections
|
||||
|
||||
|
||||
def _setting(line: str, main: bool) -> AptProxySettingDict | None:
|
||||
def _grep_line(line: str, main: bool) -> tuple[str, int, str] | None:
|
||||
"""File, line number and text of one line grep printed; None for a file apt skips."""
|
||||
found = _GREP.match(line)
|
||||
if not found:
|
||||
return None
|
||||
path, number, text = found.groups()
|
||||
if not main and not _PART_NAME.match(path.rsplit("/", 1)[-1]):
|
||||
return None
|
||||
return path, int(number), text
|
||||
|
||||
|
||||
def _setting(line: str, main: bool) -> AptProxySettingDict | None:
|
||||
found = _grep_line(line, main)
|
||||
if not found:
|
||||
return None
|
||||
path, number, text = found
|
||||
flat = _FLAT.match(text)
|
||||
if flat:
|
||||
return {"file": path, "line": int(number), "value": flat.group(1), "editable": True}
|
||||
@@ -156,22 +193,81 @@ def _setting(line: str, main: bool) -> AptProxySettingDict | None:
|
||||
return {"file": path, "line": int(number), "value": nested.group(1), "editable": False}
|
||||
|
||||
|
||||
def _dump(lines: list[str]) -> tuple[str | None, dict[str, str], str | None]:
|
||||
"""The proxy, the per-host proxies and the auto-detect program in ``apt-config dump``."""
|
||||
proxy: str | None = None
|
||||
per_host: dict[str, str] = {}
|
||||
auto_detect: str | None = None
|
||||
def _https_setting(line: str, main: bool) -> AptProxySettingDict | None:
|
||||
found = _grep_line(line, main)
|
||||
if not found:
|
||||
return None
|
||||
path, number, text = found
|
||||
flat = _FLAT_HTTPS.match(text)
|
||||
if flat:
|
||||
return {"file": path, "line": number, "value": flat.group(1), "editable": True}
|
||||
block = _HTTPS_BLOCK.match(text)
|
||||
if block:
|
||||
return {"file": path, "line": number, "value": block.group(1), "editable": False}
|
||||
return None
|
||||
|
||||
|
||||
class _Dump(TypedDict):
|
||||
proxy: str | None
|
||||
per_host: dict[str, str]
|
||||
auto_detect: str | None
|
||||
#: ``Acquire::https::Proxy`` as written; None when unset or empty.
|
||||
https: str | None
|
||||
https_per_host: dict[str, str]
|
||||
|
||||
|
||||
def _dump(lines: list[str]) -> _Dump:
|
||||
"""The proxies, the per-host proxies and the auto-detect program in ``apt-config dump``."""
|
||||
dump: _Dump = {
|
||||
"proxy": None,
|
||||
"per_host": {},
|
||||
"auto_detect": None,
|
||||
"https": None,
|
||||
"https_per_host": {},
|
||||
}
|
||||
for found in (_DUMP.match(line.strip()) for line in lines):
|
||||
if not found:
|
||||
continue
|
||||
suffix, value = found.groups()
|
||||
if not suffix:
|
||||
proxy = None if value.lower() in _NO_PROXY else value
|
||||
elif suffix.lower() == "-auto-detect":
|
||||
auto_detect = value or None
|
||||
else:
|
||||
per_host[suffix[2:]] = value
|
||||
return proxy, per_host, auto_detect
|
||||
if found:
|
||||
_take(dump, *found.groups())
|
||||
return dump
|
||||
|
||||
|
||||
def _take(dump: _Dump, scheme: str, suffix: str | None, value: str) -> None:
|
||||
"""One line of the dump into *dump*; a later one of the same key wins, as in apt."""
|
||||
https = scheme.lower() == "https"
|
||||
if not suffix:
|
||||
_take_for_all(dump, https, value)
|
||||
elif suffix.lower() == "-auto-detect":
|
||||
if not https:
|
||||
dump["auto_detect"] = value or None
|
||||
elif value:
|
||||
dump["https_per_host" if https else "per_host"][suffix[2:]] = value
|
||||
|
||||
|
||||
def _take_for_all(dump: _Dump, https: bool, value: str) -> None:
|
||||
"""The proxy for every host: https keeps an empty value as unset, apt's fallback."""
|
||||
if https:
|
||||
dump["https"] = value or None
|
||||
else:
|
||||
dump["proxy"] = None if value.lower() in _NO_PROXY else value
|
||||
|
||||
|
||||
def _https_proxy(dump: _Dump) -> str | None:
|
||||
"""apt's https method takes its own setting and falls back to the http one."""
|
||||
if dump["https"] is None:
|
||||
return dump["proxy"]
|
||||
return None if dump["https"].lower() in _NO_PROXY else dump["https"]
|
||||
|
||||
|
||||
def https_proxy_for(report: AptProxyReportDict, host: str) -> str | None:
|
||||
"""The proxy apt takes for an https source on *host*; None for ``DIRECT`` or none.
|
||||
|
||||
The host's own setting first, https before http, then the one for all hosts.
|
||||
"""
|
||||
for per_host in (report["https_per_host"], report["per_host"]):
|
||||
value = next((v for h, v in per_host.items() if h.lower() == host.lower()), None)
|
||||
if value:
|
||||
return None if value.lower() in _NO_PROXY else value
|
||||
return report["https_proxy"]
|
||||
|
||||
|
||||
def parse_apt_proxy(output: str) -> AptProxyReportDict:
|
||||
@@ -180,24 +276,36 @@ def parse_apt_proxy(output: str) -> AptProxyReportDict:
|
||||
:raises ValueError: if the output carried no intact report.
|
||||
"""
|
||||
sections = _sections(output)
|
||||
proxy, per_host, auto_detect = _dump(sections.get("dump", []))
|
||||
settings = [
|
||||
s
|
||||
dump = _dump(sections.get("dump", []))
|
||||
lines = [
|
||||
(main, line)
|
||||
for main, key in ((False, "parts"), (True, "main"))
|
||||
for s in (_setting(line, main) for line in sections.get(key, []))
|
||||
if s
|
||||
for line in sections.get(key, [])
|
||||
]
|
||||
sources = [line.strip() for line in sections.get("sources", [])]
|
||||
return {
|
||||
"proxy": proxy,
|
||||
"per_host": per_host,
|
||||
"auto_detect": auto_detect,
|
||||
"settings": settings,
|
||||
"http_sources": [s for s in sources if s.startswith("http://")],
|
||||
"https_sources": [s for s in sources if s.startswith("https://")],
|
||||
"proxy": dump["proxy"],
|
||||
"per_host": dump["per_host"],
|
||||
"auto_detect": dump["auto_detect"],
|
||||
"settings": _settings(lines, _setting),
|
||||
"https_proxy": _https_proxy(dump),
|
||||
"https_per_host": {**dump["per_host"], **dump["https_per_host"]},
|
||||
"https_settings": _settings(lines, _https_setting),
|
||||
"http_sources": _of_scheme(sources, "http"),
|
||||
"https_sources": _of_scheme(sources, "https"),
|
||||
}
|
||||
|
||||
|
||||
def _settings(
|
||||
lines: list[tuple[bool, str]], parse: Callable[[str, bool], AptProxySettingDict | None]
|
||||
) -> list[AptProxySettingDict]:
|
||||
return [s for s in (parse(line, main) for main, line in lines) if s]
|
||||
|
||||
|
||||
def _of_scheme(sources: list[str], scheme: str) -> list[str]:
|
||||
return [s for s in sources if s.startswith(f"{scheme}://")]
|
||||
|
||||
|
||||
_DEFAULT_PORT = {"http": 80, "https": 443}
|
||||
|
||||
|
||||
@@ -225,14 +333,17 @@ def apt_fetch_command(url: str, *, direct: bool = False, expect: str = "") -> st
|
||||
*direct* goes past any proxy apt is set to, to *url*'s host itself; without it
|
||||
the fetch takes apt's own way. *expect* is text the page should contain.
|
||||
|
||||
:raises ValueError: if *url* is not an http URL a shell word can carry.
|
||||
:raises ValueError: if *url* is not an http(s) URL a shell word can carry.
|
||||
"""
|
||||
if not PROXY_URL.match(url) or not url.startswith("http://"):
|
||||
if not PROXY_URL.match(url):
|
||||
raise ValueError(f"not an http URL: {url!r}")
|
||||
options = ["-o Acquire::Retries=0", "-o Acquire::http::Timeout=10"]
|
||||
host = urlsplit(url).hostname or ""
|
||||
if direct and host and ":" not in host:
|
||||
# *host* comes from a URL PROXY_URL let through: no quote in it.
|
||||
# *host* comes from a URL PROXY_URL let through: no quote in it. For https
|
||||
# the host's https setting comes first, so both are set.
|
||||
if url.startswith("https://"):
|
||||
options.append(f"-o 'Acquire::https::Proxy::{host}=DIRECT'")
|
||||
options.append(f"-o 'Acquire::http::Proxy::{host}=DIRECT'")
|
||||
match = f"echo '[match]'; grep -c -F -- {shlex.quote(expect)} \"$t\"; " if expect else ""
|
||||
return (
|
||||
@@ -272,13 +383,16 @@ def parse_apt_fetch(output: str) -> ProxyFetchDict:
|
||||
return {"reachable": answered, "expected": False if answered else None, "error": error}
|
||||
|
||||
|
||||
#: sed's edit of one line: comment it out, if it still sets the http proxy.
|
||||
_COMMENT_OUT = "-e '{line}{{/^[[:space:]]*Acquire::http::Proxy[[:space:]]/Is|^|// |}}'"
|
||||
#: sed's edit of one line: comment it out, if it still sets the proxy of *scheme*.
|
||||
_COMMENT_OUT = "-e '{line}{{/^[[:space:]]*Acquire::{scheme}::Proxy[[:space:]]/Is|^|// |}}'"
|
||||
_SCHEMES = ("http", "https")
|
||||
|
||||
|
||||
def comment_out_command(path: str, lines: list[int]) -> str:
|
||||
"""The root command that comments out *lines* of *path* where they set the http proxy."""
|
||||
edits = " ".join(_COMMENT_OUT.format(line=int(n)) for n in lines)
|
||||
def comment_out_command(path: str, lines: list[int], scheme: str = "http") -> str:
|
||||
"""The root command that comments out *lines* of *path* where they set *scheme*'s proxy."""
|
||||
if scheme not in _SCHEMES:
|
||||
raise ValueError(f"not a scheme apt proxies: {scheme!r}")
|
||||
edits = " ".join(_COMMENT_OUT.format(line=int(n), scheme=scheme) for n in lines)
|
||||
return f"sed -i {edits} {shlex.quote(path)}"
|
||||
|
||||
|
||||
@@ -333,6 +447,8 @@ class AptProxyMixin(PackageProxyMixin):
|
||||
* per_host ({host: proxy or "DIRECT"})
|
||||
* auto_detect (string or None): ``Acquire::http::Proxy-Auto-Detect``
|
||||
* settings (list of {file, line, value, editable}), in apt's reading order
|
||||
* https_proxy (string or None): the proxy https takes, the http one if unset
|
||||
* https_per_host ({host: proxy or "DIRECT"}), https_settings (as settings)
|
||||
* http_sources, https_sources (lists of the sources' base URIs)
|
||||
|
||||
:raises ValueError: if the host's output carried no intact report.
|
||||
@@ -360,8 +476,30 @@ class AptProxyMixin(PackageProxyMixin):
|
||||
self._run_apt_command(apt_fetch_command(url, direct=True, expect=expect))
|
||||
)
|
||||
|
||||
def check_https_source(self) -> HttpsSourceFetchDict | None:
|
||||
"""Fetch one https source's ``InRelease`` the way apt does, through its proxy.
|
||||
|
||||
None when no https source goes through a proxy: there are none, or https
|
||||
goes ``DIRECT``.
|
||||
|
||||
:raises ValueError: if the host's output carried no intact report.
|
||||
:raises NotImplementedError: on a host without apt.
|
||||
"""
|
||||
if not self._is_apt_host():
|
||||
raise NotImplementedError("not an apt host")
|
||||
report = self._report()
|
||||
found = _https_source(report, lambda proxy: proxy is not None)
|
||||
if found is None:
|
||||
return None
|
||||
target, proxy = found
|
||||
fetched = parse_apt_fetch(self._run_apt_command(apt_fetch_command(target)))
|
||||
return {**fetched, "url": target, "proxy": proxy}
|
||||
|
||||
def configure_package_proxy(self, url: str) -> FixResultDict:
|
||||
"""apt fetches through *url*, and is shown to; ``""`` removes :attr:`APT_PROXY_FILE`."""
|
||||
"""apt fetches through *url*, and is shown to; ``""`` removes :attr:`APT_PROXY_FILE`.
|
||||
|
||||
https goes through *url* too, unless *url* refuses it: then ``DIRECT``.
|
||||
"""
|
||||
if not url:
|
||||
return run_fix_steps([("configure", self._remove_package_proxy)])
|
||||
return run_fix_steps(
|
||||
@@ -370,6 +508,7 @@ class AptProxyMixin(PackageProxyMixin):
|
||||
("conflicts", lambda: self._comment_out_other_proxies(url)),
|
||||
("verify", lambda: self._verify_package_proxy(url)),
|
||||
("fetch", lambda: self._fetch_through_package_proxy(url)),
|
||||
("https", lambda: self._https_through_package_proxy(url)),
|
||||
]
|
||||
)
|
||||
|
||||
@@ -383,12 +522,15 @@ class AptProxyMixin(PackageProxyMixin):
|
||||
f"rm -f {shlex.quote(self.APT_PROXY_FILE)}", "proxy removed"
|
||||
)
|
||||
|
||||
def _write_package_proxy(self, url: str) -> FixStepOutcomeDict:
|
||||
def _write_package_proxy(self, url: str, https_direct: bool = False) -> FixStepOutcomeDict:
|
||||
if not self._is_apt_host():
|
||||
return self._not_apt()
|
||||
if not PROXY_URL.match(url):
|
||||
raise ValueError(f"not a proxy URL: {url!r}")
|
||||
return self._write_apt_file(self.APT_PROXY_FILE, f'Acquire::http::Proxy "{url}";\n')
|
||||
content = f'Acquire::http::Proxy "{url}";\n'
|
||||
if https_direct:
|
||||
content += 'Acquire::https::Proxy "DIRECT";\n'
|
||||
return self._write_apt_file(self.APT_PROXY_FILE, content)
|
||||
|
||||
def _report(self) -> AptProxyReportDict:
|
||||
# The steps after "configure" run only once it passed, on an apt host.
|
||||
@@ -442,16 +584,81 @@ class AptProxyMixin(PackageProxyMixin):
|
||||
return {"ok": True, "output": f"fetched {target} through {url}"}
|
||||
return {"ok": False, "output": f"{target}: {fetched['error']}"}
|
||||
|
||||
def _https_through_package_proxy(self, url: str) -> FixStepOutcomeDict:
|
||||
"""An https source through *url*; where *url* refuses it, https goes ``DIRECT``."""
|
||||
found = _https_source(self._report(), lambda proxy: same_proxy_url(proxy, url))
|
||||
if found is None:
|
||||
return {"ok": True, "output": f"no https source goes through {url}; not tried"}
|
||||
target = found[0]
|
||||
fetched = parse_apt_fetch(self._run_apt_command(apt_fetch_command(target)))
|
||||
# An HTTP error came from the mirror, through the tunnel: the cache let it by.
|
||||
if fetched["reachable"]:
|
||||
return {"ok": True, "output": f"fetched {target} through {url}"}
|
||||
refused = f"{url} refused https ({fetched['error']})"
|
||||
outcome = self._https_direct(url)
|
||||
if not outcome["ok"]:
|
||||
return {"ok": False, "output": f"{refused}; {outcome['output']}"}
|
||||
direct = parse_apt_fetch(self._run_apt_command(apt_fetch_command(target)))
|
||||
if not direct["reachable"]:
|
||||
return {"ok": False, "output": f"{refused}; direct too: {target}: {direct['error']}"}
|
||||
named = f"; {outcome['output']}" if outcome["output"] else ""
|
||||
return {
|
||||
"ok": True,
|
||||
"output": f"{refused}{named}; https sources go direct now: fetched {target}",
|
||||
}
|
||||
|
||||
def _https_direct(self, url: str) -> FixStepOutcomeDict:
|
||||
"""Write https ``DIRECT`` beside *url*, and comment out the later lines to *url*."""
|
||||
written = self._write_package_proxy(url, https_direct=True)
|
||||
if not written["ok"]:
|
||||
return written
|
||||
later = self._later_https_lines(self._report(), url)
|
||||
for path, numbers in _editable_lines(later).items():
|
||||
outcome = self._run_apt_root_command(
|
||||
comment_out_command(path, numbers, scheme="https"), f"commented out in {path}"
|
||||
)
|
||||
if not outcome["ok"]:
|
||||
return outcome
|
||||
report = self._report()
|
||||
if same_proxy_url(report["https_proxy"], url):
|
||||
still = [f"{s['file']}:{s['line']}" for s in self._later_https_lines(report, url)]
|
||||
where = f", set in {', '.join(still)}" if still else ""
|
||||
return {"ok": False, "output": f"https still goes through {url}{where}"}
|
||||
return {"ok": True, "output": "; ".join(map(_described, later))}
|
||||
|
||||
def _later_https_lines(self, report: AptProxyReportDict, url: str) -> list[AptProxySettingDict]:
|
||||
"""The lines outside :attr:`APT_PROXY_FILE` that send https to *url*."""
|
||||
return [
|
||||
s
|
||||
for s in report["https_settings"]
|
||||
if s["file"] != self.APT_PROXY_FILE and same_proxy_url(s["value"], url)
|
||||
]
|
||||
|
||||
|
||||
def _https_source(
|
||||
report: AptProxyReportDict, through: Callable[[str | None], bool]
|
||||
) -> tuple[str, str] | None:
|
||||
"""The ``InRelease`` of the first https source whose proxy *through* accepts, and that proxy."""
|
||||
for source in report["https_sources"]:
|
||||
if "/dists/" not in source:
|
||||
continue
|
||||
proxy = https_proxy_for(report, urlsplit(source).hostname or "")
|
||||
if through(proxy):
|
||||
return source.rstrip("/") + "/InRelease", proxy or ""
|
||||
return None
|
||||
|
||||
|
||||
__all__ = [
|
||||
"APT_PROXY_COMMAND",
|
||||
"AptProxyMixin",
|
||||
"AptProxyReportDict",
|
||||
"AptProxySettingDict",
|
||||
"HttpsSourceFetchDict",
|
||||
"ProxyFetchDict",
|
||||
"PROXY_URL",
|
||||
"apt_fetch_command",
|
||||
"comment_out_command",
|
||||
"https_proxy_for",
|
||||
"parse_apt_fetch",
|
||||
"parse_apt_proxy",
|
||||
"same_proxy_url",
|
||||
|
||||
@@ -0,0 +1,13 @@
|
||||
"""The ``age`` of an entry in NAPALM's neighbour tables when the device does not say.
|
||||
|
||||
NAPALM requires an ``age`` on every entry of ``get_arp_table`` and
|
||||
``get_ipv6_neighbors_table``: seconds since the device last confirmed that
|
||||
neighbour. Many devices do not report one. ``0.0`` means "confirmed just now"
|
||||
and must never stand in for "not reported": a caller ranking sightings by
|
||||
freshness takes it at its word (NetOrk/netork#1327). An expiry countdown is not
|
||||
an age either. The value is NAPALM's own; its IOS driver reports ``-1.0``.
|
||||
"""
|
||||
|
||||
from typing import Final
|
||||
|
||||
ARP_AGE_UNKNOWN: Final[float] = -1.0
|
||||
@@ -0,0 +1,238 @@
|
||||
"""
|
||||
Abstract base class for backup server drivers.
|
||||
|
||||
Usage::
|
||||
|
||||
from napalm_device_types import BackupServerDriver, StorageDriver
|
||||
|
||||
class ProxmoxBackupDriver(BackupServerDriver, StorageDriver):
|
||||
def get_backup_datastores(self):
|
||||
...
|
||||
"""
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from napalm_device_types.base import DeviceTypeDriver
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - for type checkers only
|
||||
from napalm_device_types.backups import (
|
||||
BackupDatastoreDict,
|
||||
BackupGroupDict,
|
||||
BackupJobKind,
|
||||
BackupRemoteDict,
|
||||
BackupServerJobDict,
|
||||
BackupSnapshotDict,
|
||||
BackupSnapshotRefDict,
|
||||
BackupTaskDict,
|
||||
BackupTaskStatusDict,
|
||||
)
|
||||
|
||||
|
||||
class BackupServerDriver(DeviceTypeDriver):
|
||||
"""
|
||||
Abstract intermediate driver for backup servers (e.g. Proxmox Backup
|
||||
Server): the device hypervisors and hosts write their backups to.
|
||||
|
||||
A backup server keeps its backups in datastores, one group per source (a
|
||||
VM, a container, a host) and one snapshot per backup run, and keeps them
|
||||
healthy with scheduled jobs: garbage collection, verify, prune, sync with
|
||||
other servers and tape. None of that is a storage pool, a volume or a
|
||||
replication job, so it is not modelled through ``StorageDriver``. A server
|
||||
that also manages its own disks and pools lists ``StorageDriver`` as a
|
||||
second role, after this one.
|
||||
|
||||
The actions start a task on the server and return at once with its id;
|
||||
:meth:`get_backup_task` follows it.
|
||||
"""
|
||||
|
||||
#: Stable key netOrk exposes as ``device_class``. The order in which a
|
||||
#: driver lists its role bases is the ranking; see
|
||||
#: :func:`napalm_device_types.roles.primary_role_of`.
|
||||
ROLE: str = "backup_server"
|
||||
TYPE_LABEL: str = "Backup Server"
|
||||
|
||||
if TYPE_CHECKING:
|
||||
# ------------------------------------------------------------------
|
||||
# Read
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def get_backup_datastores(self) -> list[BackupDatastoreDict]:
|
||||
"""
|
||||
Returns the server's datastores with their usage.
|
||||
|
||||
Example::
|
||||
|
||||
[
|
||||
{
|
||||
"name": "store1",
|
||||
"path": "/mnt/datastore/store1",
|
||||
"backend": "local",
|
||||
"total": 4000787030016,
|
||||
"used": 1529008357376,
|
||||
"available": 2471778672640,
|
||||
"estimated_full": 1798761600,
|
||||
"maintenance": None,
|
||||
"comment": None,
|
||||
},
|
||||
]
|
||||
|
||||
:raises Exception: when the server could not be read; never an
|
||||
empty list for a failed read.
|
||||
"""
|
||||
...
|
||||
|
||||
def get_backup_groups(self, datastore: str = "") -> list[BackupGroupDict]:
|
||||
"""
|
||||
Returns the backup groups of every namespace of *datastore*, or of
|
||||
every datastore when it is empty, each with its newest snapshot
|
||||
only. :meth:`get_backup_snapshots` lists all of a group's.
|
||||
|
||||
Example::
|
||||
|
||||
[
|
||||
{
|
||||
"datastore": "store1",
|
||||
"namespace": "",
|
||||
"type": "vm",
|
||||
"id": "101",
|
||||
"owner": "backup@pbs",
|
||||
"count": 14,
|
||||
"newest": {
|
||||
"time": 1793660400,
|
||||
"size": 34359738368,
|
||||
"verify": "ok",
|
||||
"protected": False,
|
||||
"comment": "web-01",
|
||||
},
|
||||
"oldest_time": 1792537200,
|
||||
"comment": None,
|
||||
},
|
||||
]
|
||||
|
||||
:raises Exception: when the server could not be read.
|
||||
"""
|
||||
...
|
||||
|
||||
def get_backup_snapshots(
|
||||
self, datastore: str, namespace: str, group_type: str, group_id: str
|
||||
) -> list[BackupSnapshotDict]:
|
||||
"""
|
||||
Returns every snapshot of one group, newest first.
|
||||
|
||||
:param namespace: ``""`` for the datastore's root.
|
||||
:raises ValueError: when the datastore or the group does not exist.
|
||||
"""
|
||||
...
|
||||
|
||||
def get_backup_jobs(self) -> list[BackupServerJobDict]:
|
||||
"""
|
||||
Returns the server's scheduled jobs of every kind, garbage
|
||||
collection included (one per datastore, ``id`` = the datastore).
|
||||
|
||||
Example::
|
||||
|
||||
[
|
||||
{
|
||||
"id": "store1",
|
||||
"kind": "gc",
|
||||
"datastore": "store1",
|
||||
"namespace": None,
|
||||
"schedule": "daily",
|
||||
"enabled": True,
|
||||
"last_run": 1793664000,
|
||||
"last_state": "ok",
|
||||
"last_error": None,
|
||||
"next_run": 1793750400,
|
||||
"remote": None,
|
||||
"direction": None,
|
||||
"comment": None,
|
||||
},
|
||||
{
|
||||
"id": "offsite",
|
||||
"kind": "sync",
|
||||
"datastore": "store1",
|
||||
"namespace": "",
|
||||
"schedule": "sat 03:00",
|
||||
"enabled": True,
|
||||
"last_run": 1793502000,
|
||||
"last_state": "error",
|
||||
"last_error": "connection refused",
|
||||
"next_run": 1794106800,
|
||||
"remote": "pbs-offsite",
|
||||
"direction": "push",
|
||||
"comment": None,
|
||||
},
|
||||
]
|
||||
|
||||
:raises Exception: when the server could not be read.
|
||||
"""
|
||||
...
|
||||
|
||||
def get_backup_remotes(self) -> list[BackupRemoteDict]:
|
||||
"""
|
||||
Returns the other backup servers this one syncs with, without their
|
||||
secrets.
|
||||
|
||||
:raises Exception: when the server could not be read.
|
||||
"""
|
||||
...
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Actions
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def start_garbage_collection(self, datastore: str) -> BackupTaskDict:
|
||||
"""
|
||||
Starts garbage collection on *datastore*.
|
||||
|
||||
:raises ValueError: when the datastore does not exist.
|
||||
:raises RuntimeError: when the server refuses (e.g. one is running).
|
||||
"""
|
||||
...
|
||||
|
||||
def run_backup_job(self, kind: BackupJobKind, job_id: str) -> BackupTaskDict:
|
||||
"""
|
||||
Runs a scheduled job now, outside its schedule.
|
||||
|
||||
:param kind: a :data:`~napalm_device_types.backups.BackupJobKind`;
|
||||
``"gc"`` runs the datastore named *job_id*'s garbage collection.
|
||||
:raises ValueError: when there is no such job.
|
||||
:raises RuntimeError: when the server refuses.
|
||||
"""
|
||||
...
|
||||
|
||||
def verify_backup_snapshot(self, snapshot: BackupSnapshotRefDict) -> BackupTaskDict:
|
||||
"""
|
||||
Verifies one snapshot now.
|
||||
|
||||
:raises ValueError: when the snapshot does not exist.
|
||||
"""
|
||||
...
|
||||
|
||||
def set_backup_snapshot_protected(
|
||||
self, snapshot: BackupSnapshotRefDict, protected: bool
|
||||
) -> None:
|
||||
"""
|
||||
Protects a snapshot from pruning and deletion, or lifts that.
|
||||
|
||||
:raises ValueError: when the snapshot does not exist.
|
||||
"""
|
||||
...
|
||||
|
||||
def delete_backup_snapshot(self, snapshot: BackupSnapshotRefDict) -> None:
|
||||
"""
|
||||
Deletes one snapshot. The data is gone once garbage collection has
|
||||
run; there is no undo.
|
||||
|
||||
:raises ValueError: when the snapshot does not exist.
|
||||
:raises RuntimeError: when it is protected.
|
||||
"""
|
||||
...
|
||||
|
||||
def get_backup_task(self, task: str) -> BackupTaskStatusDict:
|
||||
"""
|
||||
Returns the state and the log so far of a task an action started.
|
||||
|
||||
:raises ValueError: when the server does not know the task.
|
||||
"""
|
||||
...
|
||||
@@ -0,0 +1,119 @@
|
||||
"""The shapes of a backup server (``BackupServerDriver``).
|
||||
|
||||
Apart from ``models.py``, which is over the file size limit. A value the
|
||||
platform does not report is None; it is never guessed.
|
||||
"""
|
||||
|
||||
from typing import Literal, TypedDict, get_args
|
||||
|
||||
#: What a backup server's jobs do. Garbage collection is one of them: it has a
|
||||
#: schedule and a last run like the others.
|
||||
BackupJobKind = Literal["gc", "verify", "prune", "sync", "tape"]
|
||||
BACKUP_JOB_KINDS: tuple[str, ...] = get_args(BackupJobKind)
|
||||
|
||||
#: How a job's or a task's run ended; "running" while it has not.
|
||||
BackupRunState = Literal["ok", "warning", "error", "running"]
|
||||
|
||||
#: The outcome of the last verification of a snapshot.
|
||||
BackupVerifyState = Literal["ok", "failed"]
|
||||
|
||||
|
||||
class BackupDatastoreDict(TypedDict):
|
||||
"""A datastore: where a backup server keeps its backups."""
|
||||
|
||||
name: str
|
||||
path: str | None # local path; for an object-store backend the local cache, if any
|
||||
backend: str # "local", "s3", "removable", ... as the platform names it
|
||||
total: int | None # bytes
|
||||
used: int | None # bytes
|
||||
available: int | None # bytes
|
||||
estimated_full: int | None # unix seconds the platform expects it to be full
|
||||
maintenance: str | None # "read-only", "offline", ...; None when in service
|
||||
comment: str | None
|
||||
|
||||
|
||||
class BackupSnapshotDict(TypedDict):
|
||||
"""One backup of a group: the state of its source at one time."""
|
||||
|
||||
time: int # unix seconds the backup was taken
|
||||
size: int | None # bytes
|
||||
verify: BackupVerifyState | None # None: never verified
|
||||
protected: bool # exempt from pruning and deletion
|
||||
comment: str | None
|
||||
|
||||
|
||||
class BackupGroupDict(TypedDict):
|
||||
"""The backups of one source (a VM, a container, a host) in one namespace."""
|
||||
|
||||
datastore: str
|
||||
namespace: str # "" for the datastore's root
|
||||
type: str # "vm", "ct", "host"
|
||||
id: str # the guest's id ("101") or the host's name
|
||||
owner: str | None # the account that writes the group's backups
|
||||
count: int # snapshots in the group
|
||||
newest: BackupSnapshotDict | None # None for a group without a snapshot
|
||||
oldest_time: int | None # unix seconds of the oldest snapshot
|
||||
comment: str | None
|
||||
|
||||
|
||||
class BackupSnapshotRefDict(TypedDict):
|
||||
"""Names one snapshot, for an action on it."""
|
||||
|
||||
datastore: str
|
||||
namespace: str # "" for the datastore's root
|
||||
group_type: str # BackupGroupDict.type
|
||||
group_id: str # BackupGroupDict.id
|
||||
time: int # BackupSnapshotDict.time
|
||||
|
||||
|
||||
class BackupServerJobDict(TypedDict):
|
||||
"""A scheduled job of a backup server."""
|
||||
|
||||
id: str # for "gc", the datastore's name
|
||||
kind: BackupJobKind
|
||||
datastore: str | None
|
||||
namespace: str | None
|
||||
schedule: str | None # as the platform writes it: "daily", "sat 02:00", "*:0/30"
|
||||
enabled: bool
|
||||
last_run: int | None # unix seconds the last run ended (or started, while running)
|
||||
last_state: BackupRunState | None # None: never ran
|
||||
last_error: str | None
|
||||
next_run: int | None # unix seconds
|
||||
remote: str | None # a sync job's BackupRemoteDict.name
|
||||
direction: Literal["pull", "push"] | None # a sync job's
|
||||
comment: str | None
|
||||
|
||||
|
||||
class BackupRemoteDict(TypedDict):
|
||||
"""Another backup server this one syncs with. Never its secret."""
|
||||
|
||||
name: str
|
||||
host: str
|
||||
port: int | None
|
||||
fingerprint: str | None # of the remote's TLS certificate
|
||||
auth_id: str | None # the account the sync logs in as
|
||||
comment: str | None
|
||||
|
||||
|
||||
class BackupTaskDict(TypedDict):
|
||||
"""A task an action started on the server, to follow with ``get_backup_task``."""
|
||||
|
||||
task: str # the platform's task id
|
||||
|
||||
|
||||
class BackupTaskStatusDict(TypedDict):
|
||||
task: str
|
||||
running: bool
|
||||
state: BackupRunState | None # "running" while it runs
|
||||
error: str | None
|
||||
log: list[str] # the task's log lines so far
|
||||
|
||||
|
||||
class BackupServerRefDict(TypedDict):
|
||||
"""Which backup server and datastore a hypervisor's backup storage writes to."""
|
||||
|
||||
server: str # host name or address, as the hypervisor has it configured
|
||||
port: int | None
|
||||
datastore: str
|
||||
namespace: str | None
|
||||
fingerprint: str | None # of the server's TLS certificate, as the hypervisor pins it
|
||||
@@ -15,6 +15,7 @@ from napalm.base import NetworkDriver
|
||||
|
||||
from napalm_device_types.channel import CommandChannelMixin
|
||||
from napalm_device_types.host_reboot import HostRebootMixin
|
||||
from napalm_device_types.host_shutdown import HostShutdownMixin
|
||||
from napalm_device_types.ping_sweep import PingSweepMixin
|
||||
|
||||
|
||||
@@ -74,7 +75,13 @@ class PortSpec(NamedTuple):
|
||||
mandatory: bool = False
|
||||
|
||||
|
||||
class DeviceTypeDriver(PingSweepMixin, HostRebootMixin, CommandChannelMixin, NetworkDriver): # type: ignore[misc]
|
||||
class DeviceTypeDriver(
|
||||
PingSweepMixin,
|
||||
HostRebootMixin,
|
||||
HostShutdownMixin,
|
||||
CommandChannelMixin,
|
||||
NetworkDriver, # type: ignore[misc]
|
||||
):
|
||||
"""Common base for all netOrk device-type drivers.
|
||||
|
||||
Sits between napalm.base.NetworkDriver and the type-specific abstract
|
||||
|
||||
@@ -0,0 +1,104 @@
|
||||
"""The shapes of a hypervisor's clusters (``HypervisorDriver.get_clusters()``).
|
||||
|
||||
Apart from ``models.py``, which is over the file size limit.
|
||||
"""
|
||||
|
||||
from typing import NotRequired, TypedDict
|
||||
|
||||
from napalm_device_types.backups import BackupServerRefDict
|
||||
|
||||
|
||||
class ClusterNodeDict(TypedDict):
|
||||
"""One host of a cluster, as its cluster sees it. A value the platform does not
|
||||
report is None; it is never guessed."""
|
||||
|
||||
name: str # the node's name in its cluster, as get_vms() reports it in ``node``
|
||||
status: str # "online", "offline" or "unknown"
|
||||
local: bool # the node this connection talks to; False on a manager (vCenter)
|
||||
maintenance: bool | None # in maintenance mode
|
||||
cpu_threads: int | None # logical CPUs
|
||||
cpu_cores: int | None # physical cores
|
||||
cpu_model: str | None
|
||||
cpu_flags: list[str] | None # cpuinfo flags; None where the platform does not say
|
||||
cpu_usage: float | None # 0.0–1.0
|
||||
memory_total: int | None # bytes
|
||||
memory_used: int | None # bytes
|
||||
votes: NotRequired[int | None] # quorum votes of this node; None standalone/unreadable
|
||||
|
||||
|
||||
class ClusterStorageDict(TypedDict):
|
||||
"""A storage of a cluster and the nodes that see it."""
|
||||
|
||||
name: str # usable as create_vm_from_cloud_init(storage=...) on each of ``nodes``
|
||||
type: str # backend type: "dir", "lvmthin", "zfspool", "nfs", "rbd", "vmfs", ...
|
||||
shared: bool # the same storage on every node in ``nodes``, so a VM on it can move
|
||||
content: list[str] # what it holds: "images", "iso", "snippets", ...
|
||||
nodes: list[str] # the nodes it is available on
|
||||
total: int | None # bytes; None where the platform does not report it
|
||||
used: int | None # bytes
|
||||
# A backup storage that writes to a backup server: which one. Left out for every
|
||||
# other storage; None where the platform does not say.
|
||||
backup_server: NotRequired[BackupServerRefDict | None]
|
||||
|
||||
|
||||
class ClusterNetworkDict(TypedDict):
|
||||
"""A network a VM's NIC may attach to, and the nodes that carry it."""
|
||||
|
||||
name: str # usable as NICConfigDict.bridge on each of ``nodes``
|
||||
kind: str # "bridge", "vnet" or "portgroup", as in NetworkTargetDict
|
||||
vlan_aware: bool # True if a NICConfigDict.vlan_tag may be set on top of it
|
||||
fixed_vlan_tag: NotRequired[int | None] # vnet/portgroup: the VLAN baked into it
|
||||
nodes: list[str]
|
||||
|
||||
|
||||
class HAResourceDict(TypedDict):
|
||||
"""A guest the cluster's HA manager looks after."""
|
||||
|
||||
vmid: int
|
||||
kind: str # "qemu" or "lxc"
|
||||
requested: str # state HA is asked to keep: "started", "stopped", "disabled", "ignored"
|
||||
status: str | None # what the HA manager reports now: "started", "fence", "migrate", ...
|
||||
node: str | None # the node the HA manager places it on now
|
||||
group: NotRequired[str | None] # HA group (Proxmox VE 8)
|
||||
rules: list[str] # ids of the HA rules that name it (Proxmox VE 9); [] where there are none
|
||||
|
||||
|
||||
class BackupJobDict(TypedDict):
|
||||
"""A scheduled backup job of the cluster."""
|
||||
|
||||
id: str
|
||||
enabled: bool
|
||||
schedule: str | None # as the platform writes it: "sat 02:00", "daily", "mon..fri 21:00"
|
||||
storage: str | None
|
||||
mode: str | None # "snapshot", "suspend", "stop"
|
||||
all: bool # every guest, minus ``exclude``
|
||||
vmids: list[int] # the guests it names; a pool job: the pool's members now; [] with all
|
||||
exclude: list[int]
|
||||
pool: str | None
|
||||
node: str | None # the one node it runs on; None for every node
|
||||
|
||||
|
||||
class BackupDict(TypedDict):
|
||||
"""The backups of one guest on one storage."""
|
||||
|
||||
vmid: int
|
||||
storage: str
|
||||
time: int # unix seconds of the newest backup of this guest on this storage
|
||||
size: int | None # bytes of that newest backup
|
||||
count: int # backups of this guest on this storage
|
||||
|
||||
|
||||
class ClusterDict(TypedDict):
|
||||
"""A cluster the connection can see. A standalone host is a cluster of one,
|
||||
with no name."""
|
||||
|
||||
id: str | None # stable across connections where the platform has one; None otherwise
|
||||
name: str | None # the cluster's name; None for a standalone host
|
||||
quorate: bool | None # None where the platform has no quorum (standalone, vSphere)
|
||||
nodes: list[ClusterNodeDict]
|
||||
storages: list[ClusterStorageDict]
|
||||
networks: list[ClusterNetworkDict]
|
||||
# None: could not be read; []: read, nothing there.
|
||||
ha_resources: NotRequired[list[HAResourceDict] | None]
|
||||
backup_jobs: NotRequired[list[BackupJobDict] | None]
|
||||
backups: NotRequired[list[BackupDict] | None]
|
||||
@@ -0,0 +1,203 @@
|
||||
"""Cron expressions: what they mean, when they fire next, and as systemd ``OnCalendar``.
|
||||
|
||||
A scheduled job is given as a cron expression on every host, and a host with
|
||||
systemd runs it as a timer. So the expression has to mean the same in three
|
||||
places -- on the host's cron, in a preview of the next runs, and in the
|
||||
``OnCalendar`` lines a timer is written with -- and the meaning lives here once.
|
||||
|
||||
**What cron means is Vixie cron's.** Five fields (minute, hour, day of month,
|
||||
month, day of week), each ``*``, a value, a range ``a-b``, a step ``*/n`` or
|
||||
``a-b/n``, or a list of those; month and weekday names in any case; ``7`` is
|
||||
Sunday as well as ``0``; the ``@hourly`` … ``@yearly`` aliases. When both day
|
||||
fields are restricted -- neither starts with ``*`` -- a day matches if *either*
|
||||
does; otherwise both must (``*/2`` counts as starting with ``*``).
|
||||
|
||||
**``OnCalendar`` by expansion.** Every field is written as its values, so a range
|
||||
with a step needs no systemd syntax of its own, and a timer fires on exactly the
|
||||
minutes cron would. The *either* rule is two ``OnCalendar`` lines, which systemd
|
||||
ORs. ``@reboot`` has no calendar; a timer takes it as ``OnBootSec``.
|
||||
|
||||
**Next runs are in the host's time zone.** A local time the clock skips when
|
||||
summer time starts is left out; cron implementations disagree on it, and a
|
||||
preview that promised it would promise too much.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import date, datetime, timedelta
|
||||
from typing import NamedTuple
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
REBOOT = "@reboot"
|
||||
|
||||
ALIASES = {
|
||||
"@yearly": "0 0 1 1 *",
|
||||
"@annually": "0 0 1 1 *",
|
||||
"@monthly": "0 0 1 * *",
|
||||
"@weekly": "0 0 * * 0",
|
||||
"@daily": "0 0 * * *",
|
||||
"@midnight": "0 0 * * *",
|
||||
"@hourly": "0 * * * *",
|
||||
}
|
||||
|
||||
_MONTHS = ("jan", "feb", "mar", "apr", "may", "jun", "jul", "aug", "sep", "oct", "nov", "dec")
|
||||
_WEEKDAYS = ("sun", "mon", "tue", "wed", "thu", "fri", "sat")
|
||||
_SYSTEMD_WEEKDAYS = ("Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat")
|
||||
|
||||
#: (lowest, highest, names) per field; a name stands for its index plus the lowest.
|
||||
_FIELDS = (
|
||||
(0, 59, ()),
|
||||
(0, 23, ()),
|
||||
(1, 31, ()),
|
||||
(1, 12, _MONTHS),
|
||||
(0, 7, _WEEKDAYS),
|
||||
)
|
||||
|
||||
#: How far ahead a next run is looked for: a 29 February comes within eight years.
|
||||
_HORIZON_DAYS = 8 * 366
|
||||
|
||||
|
||||
class CronSchedule(NamedTuple):
|
||||
"""An expression's values per field, and whether a day matches on either day field."""
|
||||
|
||||
minutes: frozenset[int]
|
||||
hours: frozenset[int]
|
||||
days: frozenset[int]
|
||||
months: frozenset[int]
|
||||
weekdays: frozenset[int] # 0 = Sunday
|
||||
either_day: bool
|
||||
|
||||
|
||||
def is_reboot(expr: str) -> bool:
|
||||
"""Whether *expr* is ``@reboot``: once at boot, no calendar."""
|
||||
return expr.strip().lower() == REBOOT
|
||||
|
||||
|
||||
def _value(token: str, low: int, names: tuple[str, ...]) -> int:
|
||||
if token.isdigit():
|
||||
return int(token)
|
||||
if token.lower() in names:
|
||||
return names.index(token.lower()) + low
|
||||
raise ValueError(f"not a value: {token!r}")
|
||||
|
||||
|
||||
def _part(part: str, low: int, high: int, names: tuple[str, ...]) -> set[int]:
|
||||
span, _, step_text = part.partition("/")
|
||||
step = 1
|
||||
if step_text:
|
||||
if not step_text.isdigit() or int(step_text) == 0:
|
||||
raise ValueError(f"not a step: {part!r}")
|
||||
step = int(step_text)
|
||||
if span == "*":
|
||||
first, last = low, high
|
||||
elif "-" in span:
|
||||
a, _, b = span.partition("-")
|
||||
first, last = _value(a, low, names), _value(b, low, names)
|
||||
else:
|
||||
first = _value(span, low, names)
|
||||
last = high if step_text else first
|
||||
if not low <= first <= last <= high:
|
||||
raise ValueError(f"out of range {low}-{high}: {part!r}")
|
||||
return set(range(first, last + 1, step))
|
||||
|
||||
|
||||
def _field(text: str, low: int, high: int, names: tuple[str, ...]) -> frozenset[int]:
|
||||
values: set[int] = set()
|
||||
for part in text.split(","):
|
||||
if not part:
|
||||
raise ValueError(f"empty list item in {text!r}")
|
||||
values |= _part(part, low, high, names)
|
||||
return frozenset(values)
|
||||
|
||||
|
||||
def parse_cron(expr: str) -> CronSchedule:
|
||||
"""The values *expr* stands for.
|
||||
|
||||
:raises ValueError: for anything Vixie cron would not take, and for
|
||||
``@reboot``, which names no time.
|
||||
"""
|
||||
text = expr.strip()
|
||||
if is_reboot(text):
|
||||
raise ValueError("@reboot names no time; it runs once at boot")
|
||||
if text.startswith("@"):
|
||||
if text.lower() not in ALIASES:
|
||||
raise ValueError(f"unknown alias {text!r}")
|
||||
text = ALIASES[text.lower()]
|
||||
fields = text.split()
|
||||
if len(fields) != len(_FIELDS):
|
||||
raise ValueError(f"a cron expression has five fields, not {len(fields)}: {expr!r}")
|
||||
minutes, hours, days, months, weekdays = (
|
||||
_field(f, low, high, names) for f, (low, high, names) in zip(fields, _FIELDS, strict=True)
|
||||
)
|
||||
if 7 in weekdays:
|
||||
weekdays = (weekdays - {7}) | {0}
|
||||
either = not fields[2].startswith("*") and not fields[4].startswith("*")
|
||||
return CronSchedule(minutes, hours, days, months, frozenset(weekdays), either)
|
||||
|
||||
|
||||
def _day_matches(s: CronSchedule, day: date) -> bool:
|
||||
if day.month not in s.months:
|
||||
return False
|
||||
on_day = day.day in s.days
|
||||
on_weekday = (day.isoweekday() % 7) in s.weekdays
|
||||
return (on_day or on_weekday) if s.either_day else (on_day and on_weekday)
|
||||
|
||||
|
||||
def _local(day: date, hour: int, minute: int, tz: ZoneInfo) -> datetime | None:
|
||||
"""*day* at *hour*:*minute* in *tz*; None when the clock skips that time."""
|
||||
naive = datetime(day.year, day.month, day.day, hour, minute)
|
||||
aware = naive.replace(tzinfo=tz)
|
||||
if aware.astimezone(ZoneInfo("UTC")).astimezone(tz).replace(tzinfo=None) != naive:
|
||||
return None
|
||||
return aware
|
||||
|
||||
|
||||
def next_runs(expr: str, after: datetime, tz: ZoneInfo, count: int = 5) -> list[datetime]:
|
||||
"""The next *count* times *expr* fires after *after*, in *tz*.
|
||||
|
||||
``@reboot`` and a day that never comes (31 February) give none.
|
||||
|
||||
:raises ValueError: for an expression :func:`parse_cron` refuses.
|
||||
"""
|
||||
if is_reboot(expr):
|
||||
return []
|
||||
s = parse_cron(expr)
|
||||
start = after.astimezone(tz)
|
||||
runs: list[datetime] = []
|
||||
for offset in range(_HORIZON_DAYS):
|
||||
day = start.date() + timedelta(days=offset)
|
||||
if not _day_matches(s, day):
|
||||
continue
|
||||
for hour in sorted(s.hours):
|
||||
for minute in sorted(s.minutes):
|
||||
at = _local(day, hour, minute, tz)
|
||||
if at is None or at <= after:
|
||||
continue
|
||||
runs.append(at)
|
||||
if len(runs) == count:
|
||||
return runs
|
||||
return runs
|
||||
|
||||
|
||||
def _values(values: frozenset[int], low: int, high: int) -> str:
|
||||
if values == frozenset(range(low, high + 1)):
|
||||
return "*"
|
||||
return ",".join(f"{v:02d}" for v in sorted(values))
|
||||
|
||||
|
||||
def to_on_calendar(expr: str) -> list[str]:
|
||||
"""``OnCalendar`` lines that fire exactly when *expr* does.
|
||||
|
||||
:raises ValueError: for an expression :func:`parse_cron` refuses, ``@reboot``
|
||||
among them -- a timer takes that as ``OnBootSec``.
|
||||
"""
|
||||
s = parse_cron(expr)
|
||||
time = f"{_values(s.hours, 0, 23)}:{_values(s.minutes, 0, 59)}:00"
|
||||
months = _values(s.months, 1, 12)
|
||||
days = _values(s.days, 1, 31)
|
||||
weekdays = ",".join(_SYSTEMD_WEEKDAYS[d] for d in (1, 2, 3, 4, 5, 6, 0) if d in s.weekdays)
|
||||
every_weekday = s.weekdays == frozenset(range(7))
|
||||
if s.either_day:
|
||||
return [f"*-{months}-{days} {time}", f"{weekdays} *-{months}-* {time}"]
|
||||
prefix = "" if every_weekday else f"{weekdays} "
|
||||
return [f"{prefix}*-{months}-{days} {time}"]
|
||||
@@ -22,7 +22,7 @@ They replace ``run_device_action`` (docs/STRUCTURE.md):
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable, Sequence
|
||||
from typing import TYPE_CHECKING, Any
|
||||
from typing import TYPE_CHECKING, Any, ClassVar
|
||||
|
||||
from napalm_device_types.models import (
|
||||
FixResultDict,
|
||||
@@ -133,6 +133,66 @@ class EnsurePackageMixin:
|
||||
def install_package(self, name: str, version: str = "") -> None: ...
|
||||
|
||||
|
||||
class LldpAgentMixin:
|
||||
"""Run an LLDP agent on the device, on the interfaces the caller names.
|
||||
|
||||
The steps: install the agent, write the interfaces it runs on, enable it at boot
|
||||
and restart it so it reads them. Installing and starting are generic, over
|
||||
``ensure_package`` and ``manage_service`` with ``LLDP_AGENT_PACKAGE`` and
|
||||
``LLDP_AGENT_SERVICE``; the driver writes the configuration. Which interfaces is
|
||||
the caller's choice -- an agent on every interface also talks to the uplink of a
|
||||
WAN port. Whether neighbours appear is the caller's to read with
|
||||
``get_lldp_neighbors``, after the agent's first advertisement interval.
|
||||
"""
|
||||
|
||||
#: The package that brings the agent, as ``ensure_package`` takes it.
|
||||
LLDP_AGENT_PACKAGE: ClassVar[str] = "lldpd"
|
||||
#: The agent's service, as ``manage_service`` takes it.
|
||||
LLDP_AGENT_SERVICE: ClassVar[str] = "lldpd"
|
||||
|
||||
def configure_lldp_agent(self, interfaces: Sequence[str]) -> FixResultDict:
|
||||
"""Install, configure for *interfaces*, enable and restart the agent.
|
||||
|
||||
:raises ValueError: for no interface, before anything changes.
|
||||
"""
|
||||
names = [str(name) for name in interfaces]
|
||||
if not names:
|
||||
raise ValueError("name at least one interface for the LLDP agent")
|
||||
return run_fix_steps(
|
||||
[
|
||||
("install", self._install_lldp_agent),
|
||||
("configure", lambda: self._write_lldp_agent_config(names)),
|
||||
("start", self._start_lldp_agent),
|
||||
]
|
||||
)
|
||||
|
||||
def _install_lldp_agent(self) -> FixStepOutcomeDict:
|
||||
result = self.ensure_package(self.LLDP_AGENT_PACKAGE)
|
||||
return {"ok": bool(result["success"]), "output": str(result.get("output", ""))}
|
||||
|
||||
def _start_lldp_agent(self) -> FixStepOutcomeDict:
|
||||
service = self.LLDP_AGENT_SERVICE
|
||||
for action in ("enable", "restart"):
|
||||
result = self.manage_service(service, action)
|
||||
if not result.get("success"):
|
||||
output = str(result.get("output", "")).strip()
|
||||
return {"ok": False, "output": f"{action} {service}: {output}".rstrip(": ")}
|
||||
return {"ok": True, "output": f"{service} enabled and restarted"}
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def ensure_package(self, name: str) -> FixResultDict: ...
|
||||
|
||||
def manage_service(self, name: str, action: str) -> dict[str, Any]: ...
|
||||
|
||||
# The step a driver implements:
|
||||
|
||||
def _write_lldp_agent_config(self, interfaces: list[str]) -> FixStepOutcomeDict:
|
||||
"""Have the agent send and listen on exactly *interfaces* (names as the
|
||||
device's interface getters report them), replacing the list it had."""
|
||||
...
|
||||
|
||||
|
||||
class PackageProxyMixin:
|
||||
"""Point the device's package manager at a caching proxy."""
|
||||
|
||||
@@ -152,6 +212,7 @@ class PackageProxyMixin:
|
||||
__all__ = [
|
||||
"EnsurePackageMixin",
|
||||
"FixStep",
|
||||
"LldpAgentMixin",
|
||||
"PackageProxyMixin",
|
||||
"SnmpAgentMixin",
|
||||
"run_fix_steps",
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
|
||||
**Deprecated** (docs/STRUCTURE.md): drivers do not collect over SNMP. The
|
||||
application polls; a driver whose device publishes values under its own OIDs
|
||||
declares them as ``SNMP_HEALTH_PROFILE`` (:class:`~napalm_device_types.models.SnmpHealthProfile`).
|
||||
This mixin goes in 5.0.
|
||||
declares them as ``SNMP_HEALTH_PROFILE``
|
||||
(:class:`~napalm_device_types.snmp_health_profile.SnmpHealthProfile`). This mixin goes in 5.0.
|
||||
|
||||
The collection itself is generic: CPU, memory, uptime, load and per-interface
|
||||
counters come from the same standard OIDs on any UCD-MIB/IF-MIB capable device.
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
"""Shutting the device itself down cleanly.
|
||||
|
||||
Declared under ``if TYPE_CHECKING``: a contract, not a placeholder. Only a
|
||||
driver that can shut its device down through the device's own means defines
|
||||
``shutdown_host``, so ``hasattr(driver, "shutdown_host")`` tells a caller
|
||||
whether to offer it. There is no forced variant here, by design: cutting the
|
||||
power is the BMC's business, not a driver's.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
|
||||
class HostShutdownMixin:
|
||||
if TYPE_CHECKING:
|
||||
|
||||
def shutdown_host(self) -> None:
|
||||
"""
|
||||
Shuts down the device this driver is connected to, cleanly.
|
||||
|
||||
Returns once the device has accepted the request; the session is
|
||||
usually gone right after. Whether the device really went off is
|
||||
the caller's business, e.g. through its BMC.
|
||||
|
||||
A driver that manages other machines shuts down its *own* host,
|
||||
never one of them: a hypervisor node shuts down itself, and its
|
||||
guests are stopped the way the node stops them on shutdown.
|
||||
|
||||
:raises RuntimeError: If the device refuses the request.
|
||||
"""
|
||||
...
|
||||
@@ -13,6 +13,7 @@ Usage::
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from napalm_device_types.base import DeviceTypeDriver
|
||||
from napalm_device_types.clusters import ClusterDict
|
||||
from napalm_device_types.health_metrics import HealthMetricsMixin
|
||||
from napalm_device_types.models import (
|
||||
HypervisorHostDict,
|
||||
@@ -30,6 +31,7 @@ from napalm_device_types.models import (
|
||||
)
|
||||
from napalm_device_types.packages import PackageManagementMixin
|
||||
from napalm_device_types.provisioning import QEMU_GUEST_AGENTS, GuestAgents
|
||||
from napalm_device_types.vm_migration import MigrationPreflightDict
|
||||
|
||||
|
||||
class HypervisorDriver(PackageManagementMixin, HealthMetricsMixin, DeviceTypeDriver):
|
||||
@@ -464,21 +466,18 @@ class HypervisorDriver(PackageManagementMixin, HealthMetricsMixin, DeviceTypeDri
|
||||
storage: str | None = None,
|
||||
cpu_type: str | None = None,
|
||||
guest_os: str = "linux",
|
||||
node: str | None = None,
|
||||
download_timeout: int = 300,
|
||||
timeout: int = 180,
|
||||
) -> VMProvisionResultDict:
|
||||
"""
|
||||
Create a new virtual machine from a cloud image via Cloud-Init.
|
||||
|
||||
Downloads the cloud image (qcow2/raw) directly on the hypervisor if not
|
||||
already cached there, creates a new VM shell, imports the image as its
|
||||
root disk, configures virtual network interfaces, and injects Cloud-Init
|
||||
configuration via a storage snippet or similar mechanism. The resulting
|
||||
VM is left in a running state.
|
||||
|
||||
Implementations should cache downloaded images by URL/filename on the
|
||||
hypervisor so repeated provisioning from the same image does not
|
||||
re-download it every time.
|
||||
Downloads the cloud image (qcow2/raw) directly on the hypervisor unless it is
|
||||
cached there already (implementations cache by URL/filename), creates a new VM
|
||||
shell, imports the image as its root disk, configures virtual network interfaces,
|
||||
and injects Cloud-Init configuration via a storage snippet or similar mechanism.
|
||||
The resulting VM is left in a running state.
|
||||
|
||||
Args:
|
||||
name (string) - new VM display name
|
||||
@@ -497,10 +496,8 @@ class HypervisorDriver(PackageManagementMixin, HealthMetricsMixin, DeviceTypeDri
|
||||
image_checksum (string | None) - expected checksum of the downloaded
|
||||
image (e.g. "sha256:<hex>"). If given, verified after download;
|
||||
mismatch raises RuntimeError. If None, no verification is performed.
|
||||
ssh_public_keys (list[str] | None) - SSH public keys to inject into guest.
|
||||
If None or empty, no SSH key injection is performed.
|
||||
disk_resize_gb (int | None) - resize root disk to this size in GB.
|
||||
If None, disk remains the downloaded image's native size. Default None.
|
||||
ssh_public_keys (list[str] | None) - SSH public keys to inject (None: none).
|
||||
disk_resize_gb (int | None) - resize root disk to this size in GB (None: keep).
|
||||
storage (string | None) - name of the storage pool to place the root
|
||||
disk on (a name returned by ``get_image_storages()``). If None,
|
||||
the driver auto-detects the first enabled, node-available storage
|
||||
@@ -519,6 +516,9 @@ class HypervisorDriver(PackageManagementMixin, HealthMetricsMixin, DeviceTypeDri
|
||||
network-config itself (``provisioning.network_config``). A
|
||||
guest_os the driver does not list raises ValueError before
|
||||
anything is created. Default ``"linux"``.
|
||||
node (string | None) - a node ``name`` from ``get_clusters()`` to create
|
||||
the VM on; None leaves it to the driver. One that only creates on its
|
||||
own node raises ValueError for another, before creating anything.
|
||||
download_timeout (int) - maximum seconds to wait for the image download
|
||||
(skipped entirely if already cached on the hypervisor). Default 300.
|
||||
timeout (int) - maximum seconds to wait for the remaining provisioning
|
||||
@@ -531,6 +531,8 @@ class HypervisorDriver(PackageManagementMixin, HealthMetricsMixin, DeviceTypeDri
|
||||
Raises:
|
||||
RuntimeError - if provisioning fails (download failure, checksum
|
||||
mismatch, storage unavailable, invalid config, timeout, etc.)
|
||||
VMProvisionIncomplete - a RuntimeError for a failure after the VM was created,
|
||||
with that VM's ``vmid`` and ``node`` for ``destroy_vm`` and the failure as cause.
|
||||
"""
|
||||
...
|
||||
|
||||
@@ -613,7 +615,9 @@ class HypervisorDriver(PackageManagementMixin, HealthMetricsMixin, DeviceTypeDri
|
||||
List[NetworkTargetDict] - each entry's ``vlan_aware`` flag tells the
|
||||
caller whether a ``NICConfigDict.vlan_tag`` may additionally be set
|
||||
for a NIC using that target (see ``NetworkTargetDict`` for the
|
||||
per-kind rules).
|
||||
per-kind rules). ``uplinks``, where the driver can tell, names
|
||||
the physical NICs a bridge leaves the host through, grouped:
|
||||
a bond's slaves in one group, a single NIC in its own.
|
||||
"""
|
||||
...
|
||||
|
||||
@@ -635,6 +639,64 @@ class HypervisorDriver(PackageManagementMixin, HealthMetricsMixin, DeviceTypeDri
|
||||
"""
|
||||
...
|
||||
|
||||
def get_clusters(self) -> list[ClusterDict]:
|
||||
"""
|
||||
List the clusters this connection can see, each with its nodes, its
|
||||
storages and its networks.
|
||||
|
||||
A cluster member (a Proxmox node) reports its one cluster, the whole of
|
||||
it: every node, offline ones included with ``status`` ``"offline"``, and
|
||||
``local`` set on the node it is. A manager (vCenter) reports each
|
||||
cluster it runs, and each standalone host it runs as a cluster of one.
|
||||
A standalone host reports itself as a cluster of one, ``name`` None.
|
||||
|
||||
The VMs ``get_vms()`` lists name their node in ``node``; that name is
|
||||
the node's ``name`` here. A storage or network lists the nodes it is
|
||||
available on, so a caller can tell where a VM may go. A value the
|
||||
platform does not report is None, never a guess.
|
||||
|
||||
Facts only: which node is a good place for a VM is the caller's to
|
||||
decide.
|
||||
|
||||
Returns:
|
||||
List[ClusterDict]
|
||||
"""
|
||||
...
|
||||
|
||||
def get_vm_migration_preflight(self, name: str) -> MigrationPreflightDict:
|
||||
"""Which nodes of its cluster the guest could move to, and what blocks it
|
||||
(see ``vm_migration``).
|
||||
|
||||
:raises ValueError: If no VM with the given name/ID exists.
|
||||
"""
|
||||
...
|
||||
|
||||
def migrate_vm(
|
||||
self,
|
||||
name: str,
|
||||
target_node: str,
|
||||
online: bool = True,
|
||||
target_storage: str | None = None,
|
||||
timeout: int = 3600,
|
||||
) -> None:
|
||||
"""Move the guest to another node of its cluster; block until the
|
||||
platform's task ended. ``online`` keeps a running guest running.
|
||||
|
||||
:raises ValueError: Unknown VM, or a target that is not another
|
||||
member of the cluster.
|
||||
:raises RuntimeError: The migration failed, with the platform's cause.
|
||||
"""
|
||||
...
|
||||
|
||||
def set_node_maintenance(self, node: str, enabled: bool) -> None:
|
||||
"""Switch HA maintenance mode of a cluster node; HA guests leave it
|
||||
while it is on.
|
||||
|
||||
:raises ValueError: If ``node`` is not a member of the cluster.
|
||||
:raises RuntimeError: If the platform refuses or cannot be asked.
|
||||
"""
|
||||
...
|
||||
|
||||
def get_hypervisor_hosts(self) -> list[HypervisorHostDict]:
|
||||
"""The hosts this hypervisor or its manager runs, and how each stands:
|
||||
connected, in maintenance, with configuration issues, under a central
|
||||
|
||||
@@ -0,0 +1,97 @@
|
||||
"""LLDP neighbours as rows, and the two shapes NAPALM asks for.
|
||||
|
||||
A driver reads its neighbours one row per neighbour -- from ``lldpctl``
|
||||
(:func:`~napalm_device_types.lldpctl.parse_lldpctl_keyvalue`), an API or a CLI
|
||||
-- and ``get_lldp_neighbors`` and ``get_lldp_neighbors_detail`` are the same
|
||||
projection of those rows for every driver. A port with several neighbours (a
|
||||
phone and the PC behind it, an unmanaged switch) is several rows, and stays
|
||||
several entries in both shapes.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from collections.abc import Iterable
|
||||
from typing import NotRequired, TypedDict
|
||||
|
||||
from napalm.base.models import LLDPNeighborDetailDict
|
||||
|
||||
#: A chassis id of type ``mac`` that really is a MAC address.
|
||||
_MAC_RE = re.compile(r"^([0-9A-Fa-f]{2}:){5}[0-9A-Fa-f]{2}$")
|
||||
|
||||
|
||||
class LldpNeighborRow(TypedDict):
|
||||
"""One LLDP neighbour, as the local port heard it.
|
||||
|
||||
``chassis_id_type`` and ``port_id_type`` name what the id is: ``mac``,
|
||||
``ip``, ``ifname``, ``ifalias``, ``local`` or ``unhandled`` (an id type
|
||||
lldpd does not decode), ``""`` when the neighbour sent no id.
|
||||
``capabilities`` and ``enabled_capabilities`` use NAPALM's names
|
||||
(``bridge``, ``router``, ``wlan-access-point``, ``telephone``, ...).
|
||||
"""
|
||||
|
||||
local_port: str
|
||||
chassis_id: str
|
||||
chassis_id_type: str
|
||||
port_id: str
|
||||
port_id_type: str
|
||||
port_description: str
|
||||
system_name: str
|
||||
system_description: str
|
||||
management_addresses: list[str]
|
||||
capabilities: list[str]
|
||||
enabled_capabilities: list[str]
|
||||
|
||||
|
||||
class LldpNeighborDict(TypedDict):
|
||||
"""NAPALM's ``get_lldp_neighbors`` entry (``hostname``, ``port``), plus the chassis MAC.
|
||||
|
||||
``mac`` (lower case) is there only when the neighbour names its chassis by a
|
||||
MAC address; an application can match the neighbour by it, where a name
|
||||
may be ambiguous.
|
||||
"""
|
||||
|
||||
hostname: str
|
||||
port: str
|
||||
mac: NotRequired[str]
|
||||
|
||||
|
||||
def lldp_neighbors(rows: Iterable[LldpNeighborRow]) -> dict[str, list[LldpNeighborDict]]:
|
||||
"""``get_lldp_neighbors``: per local port, every neighbour's name, port and chassis MAC."""
|
||||
neighbors: dict[str, list[LldpNeighborDict]] = {}
|
||||
for row in rows:
|
||||
entry: LldpNeighborDict = {"hostname": row["system_name"], "port": row["port_id"]}
|
||||
if row["chassis_id_type"] == "mac" and _MAC_RE.match(row["chassis_id"]):
|
||||
entry["mac"] = row["chassis_id"].lower()
|
||||
neighbors.setdefault(row["local_port"], []).append(entry)
|
||||
return neighbors
|
||||
|
||||
|
||||
def lldp_neighbors_detail(
|
||||
rows: Iterable[LldpNeighborRow], interface: str = ""
|
||||
) -> dict[str, list[LLDPNeighborDetailDict]]:
|
||||
"""``get_lldp_neighbors_detail``: per local port, every neighbour in full.
|
||||
|
||||
With ``interface``, that port only. ``parent_interface`` stays empty: a row
|
||||
does not know which aggregate its port belongs to.
|
||||
"""
|
||||
details: dict[str, list[LLDPNeighborDetailDict]] = {}
|
||||
for row in rows:
|
||||
if interface and row["local_port"] != interface:
|
||||
continue
|
||||
details.setdefault(row["local_port"], []).append(
|
||||
{
|
||||
"parent_interface": "",
|
||||
"remote_port": row["port_id"],
|
||||
"remote_port_description": row["port_description"],
|
||||
"remote_chassis_id": row["chassis_id"],
|
||||
"remote_system_name": row["system_name"],
|
||||
"remote_system_description": row["system_description"],
|
||||
"remote_system_capab": list(row["capabilities"]),
|
||||
"remote_system_enable_capab": list(row["enabled_capabilities"]),
|
||||
}
|
||||
)
|
||||
return details
|
||||
|
||||
|
||||
__all__ = ["LldpNeighborDict", "LldpNeighborRow", "lldp_neighbors", "lldp_neighbors_detail"]
|
||||
@@ -0,0 +1,204 @@
|
||||
"""``lldpctl -f keyvalue``: lldpd's neighbours, one row each.
|
||||
|
||||
Every driver for a host that runs lldpd reads its neighbours the same way, so
|
||||
the command and its parse live here once and a driver only carries the command
|
||||
across. lldpd's keyvalue writer (``src/client/kv_writer.c``) prints one
|
||||
``key=value`` line per field, the key being the field's path joined by dots
|
||||
(a newline inside a value becomes a space)::
|
||||
|
||||
lldp.eth0.1.via=LLDP
|
||||
lldp.eth0.1.chassis.mac=00:11:22:33:44:01
|
||||
lldp.eth0.1.chassis.Tel.enabled=on
|
||||
lldp.eth0.1.port.local=gi1/0/7
|
||||
|
||||
**A neighbour is a block, not a port.** Every neighbour on a port prints the
|
||||
same keys, so grouping the lines by interface merges them. A neighbour opens
|
||||
with ``via`` (after the local interface's ``alias``, when it has one); the next
|
||||
``via`` on the same port is the next neighbour.
|
||||
|
||||
**An interface name may hold dots** (``eth0.1``, ``br-lan.10``). It is
|
||||
everything between ``lldp.`` and the neighbour's first field, never the second
|
||||
dotted part of the key.
|
||||
|
||||
**An id is printed under its type**, ``chassis.<type>=`` and ``port.<type>=``,
|
||||
with lldpd's names for the type (``mac``, ``ip``, ``ifname``, ``ifalias``,
|
||||
``local``, ``unhandled``); there is no ``chassis.id`` key.
|
||||
|
||||
**A failed read raises.** lldpctl exits non-zero when it cannot reach lldpd;
|
||||
:data:`LLDPCTL_KEYVALUE_COMMAND` brings that status and lldpctl's message back,
|
||||
and the parse raises on it instead of reading it as no neighbours. Output that
|
||||
carries no exit status (a driver that runs plain ``lldpctl -f keyvalue``) raises
|
||||
when it holds text but not a single neighbour line, as a shell's ``lldpctl: not
|
||||
found`` does.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
|
||||
from napalm_device_types.lldp import LldpNeighborRow
|
||||
|
||||
_RC = "__LLDPCTL_RC"
|
||||
|
||||
#: One line, POSIX ``sh``, read-only. Without arguments lldpctl shows the
|
||||
#: neighbours in detail; its error text and exit status come back with them.
|
||||
LLDPCTL_KEYVALUE_COMMAND = f"lldpctl -f keyvalue 2>&1; echo {_RC}=$?"
|
||||
|
||||
_RC_RE = re.compile(rf"^{_RC}=(\d+)$")
|
||||
_PREFIX = "lldp."
|
||||
#: The fields a neighbour opens with: the local interface's alias, then ``via``.
|
||||
_OPENERS = frozenset({"alias", "via"})
|
||||
#: The first part after the interface name of every key lldpd prints for a
|
||||
#: neighbour (``src/client/display.c``, ``display_interface``).
|
||||
_CATEGORIES = _OPENERS | {
|
||||
"rid",
|
||||
"age",
|
||||
"status",
|
||||
"chassis",
|
||||
"port",
|
||||
"ttl",
|
||||
"vlan",
|
||||
"ppvid",
|
||||
"pi",
|
||||
"lldp-med",
|
||||
"unknown-tlvs",
|
||||
}
|
||||
#: lldpd's id types (``src/lib/atoms/chassis.c``, ``port.c``), in the order read.
|
||||
_CHASSIS_ID_TYPES = ("mac", "local", "ip", "ifname", "ifalias", "unhandled")
|
||||
_PORT_ID_TYPES = ("ifname", "mac", "local", "ip", "ifalias", "unhandled")
|
||||
#: lldpd's capability symbols in NAPALM's names; one NAPALM lacks is lowercased.
|
||||
_CAPABILITIES = {
|
||||
"Other": "other",
|
||||
"Repeater": "repeater",
|
||||
"Bridge": "bridge",
|
||||
"Router": "router",
|
||||
"Wlan": "wlan-access-point",
|
||||
"Tel": "telephone",
|
||||
"Docsis": "docsis-cable-device",
|
||||
"Station": "station",
|
||||
}
|
||||
_SWITCH = {"on": True, "off": False}
|
||||
|
||||
_Fields = dict[str, list[str]]
|
||||
|
||||
|
||||
def parse_lldpctl_keyvalue(output: str) -> list[LldpNeighborRow]:
|
||||
"""One row per neighbour, in the order lldpctl listed them.
|
||||
|
||||
:raises ValueError: when lldpctl failed -- a non-zero exit status, or text
|
||||
without a single neighbour line and no status at all.
|
||||
"""
|
||||
return [_row(port, fields) for port, fields in _neighbours(_pairs(output))]
|
||||
|
||||
|
||||
def _pairs(output: str) -> list[tuple[str, str]]:
|
||||
"""``(key without "lldp.", value)`` of every neighbour line; raises on a failed run."""
|
||||
pairs: list[tuple[str, str]] = []
|
||||
other: list[str] = []
|
||||
status: int | None = None
|
||||
for line in output.splitlines():
|
||||
line = line.strip()
|
||||
key, sep, value = line.partition("=")
|
||||
if _RC_RE.match(line):
|
||||
status = int(value)
|
||||
elif sep and key.startswith(_PREFIX):
|
||||
pairs.append((key[len(_PREFIX) :], value))
|
||||
elif line:
|
||||
other.append(line)
|
||||
_raise_on_failure(status, other, bool(pairs))
|
||||
return pairs
|
||||
|
||||
|
||||
def _raise_on_failure(status: int | None, other: list[str], listed: bool) -> None:
|
||||
"""A non-zero status, or text without a status and without a neighbour line."""
|
||||
if status or (status is None and other and not listed):
|
||||
said = " ".join(other)[:200] or f"exit status {status}"
|
||||
raise ValueError(f"lldpctl did not list the neighbours: {said}")
|
||||
|
||||
|
||||
def _neighbours(pairs: list[tuple[str, str]]) -> list[tuple[str, _Fields]]:
|
||||
"""Split the lines into neighbours: ``(local interface, {field: [values]})``."""
|
||||
neighbours: list[tuple[str, _Fields]] = []
|
||||
port: str | None = None
|
||||
fields: _Fields = {}
|
||||
for key, value in pairs:
|
||||
split = _split(key, port)
|
||||
if split is None:
|
||||
continue
|
||||
name, field = split
|
||||
if name != port or field == "alias" or (field == "via" and "via" in fields):
|
||||
port, fields = name, {}
|
||||
neighbours.append((port, fields))
|
||||
fields.setdefault(field, []).append(value)
|
||||
return neighbours
|
||||
|
||||
|
||||
def _split(key: str, port: str | None) -> tuple[str, str] | None:
|
||||
"""``(interface, field)`` of a key, or None when it is no neighbour field.
|
||||
|
||||
``via`` and ``alias`` end their key, so they name the interface exactly;
|
||||
the keys after them continue that interface. Only a key that does neither
|
||||
ends its interface name at the first known field.
|
||||
"""
|
||||
head, _, last = key.rpartition(".")
|
||||
if head and last in _OPENERS:
|
||||
return head, last
|
||||
if port is not None and key.startswith(port + "."):
|
||||
field = key[len(port) + 1 :]
|
||||
if field.split(".", 1)[0] in _CATEGORIES:
|
||||
return port, field
|
||||
parts = key.split(".")
|
||||
for i in range(1, len(parts)):
|
||||
if parts[i] in _CATEGORIES:
|
||||
return ".".join(parts[:i]), ".".join(parts[i:])
|
||||
return None
|
||||
|
||||
|
||||
def _row(port: str, fields: _Fields) -> LldpNeighborRow:
|
||||
chassis_type, chassis_id = _id(fields, "chassis", _CHASSIS_ID_TYPES)
|
||||
port_type, port_id = _id(fields, "port", _PORT_ID_TYPES)
|
||||
capabilities, enabled = _capabilities(fields)
|
||||
return {
|
||||
"local_port": port,
|
||||
"chassis_id": chassis_id,
|
||||
"chassis_id_type": chassis_type,
|
||||
"port_id": port_id,
|
||||
"port_id_type": port_type,
|
||||
"port_description": _first(fields, "port.descr"),
|
||||
"system_name": _first(fields, "chassis.name"),
|
||||
"system_description": _first(fields, "chassis.descr"),
|
||||
"management_addresses": list(fields.get("chassis.mgmt-ip", [])),
|
||||
"capabilities": capabilities,
|
||||
"enabled_capabilities": enabled,
|
||||
}
|
||||
|
||||
|
||||
def _first(fields: _Fields, field: str) -> str:
|
||||
return fields.get(field, [""])[0]
|
||||
|
||||
|
||||
def _id(fields: _Fields, category: str, types: tuple[str, ...]) -> tuple[str, str]:
|
||||
"""``(type, id)`` of the chassis or the port; ``("", "")`` when none was sent."""
|
||||
for kind in types:
|
||||
if f"{category}.{kind}" in fields:
|
||||
return kind, _first(fields, f"{category}.{kind}")
|
||||
return "", ""
|
||||
|
||||
|
||||
def _capabilities(fields: _Fields) -> tuple[list[str], list[str]]:
|
||||
"""``chassis.<Symbol>.enabled=on|off``: the capabilities offered, and those enabled."""
|
||||
offered: list[str] = []
|
||||
enabled: list[str] = []
|
||||
for field, values in fields.items():
|
||||
parts = field.split(".")
|
||||
switch = _SWITCH.get(values[0])
|
||||
if len(parts) != 3 or parts[0] != "chassis" or parts[2] != "enabled" or switch is None:
|
||||
continue
|
||||
name = _CAPABILITIES.get(parts[1], parts[1].lower())
|
||||
offered.append(name)
|
||||
if switch:
|
||||
enabled.append(name)
|
||||
return offered, enabled
|
||||
|
||||
|
||||
__all__ = ["LLDPCTL_KEYVALUE_COMMAND", "parse_lldpctl_keyvalue"]
|
||||
@@ -0,0 +1,751 @@
|
||||
"""Personal login accounts on a host: reading them, adding one, locking one again.
|
||||
|
||||
A person who is to work on a host gets an account of their own: they log in
|
||||
with their public key and, where they are to administer the host, may use
|
||||
sudo with their own password. When they leave, the account is locked, not
|
||||
deleted: their key goes, sudo goes, the login is refused, and the home stays.
|
||||
|
||||
:class:`LocalAccountsMixin` holds what is the same on every POSIX host -- the
|
||||
checks, reading an account back, the order of the steps, the key file and the
|
||||
sudo rule. A driver supplies the commands that differ between operating
|
||||
systems: creating, opening and locking an account, reading its lock, and
|
||||
taking it out of a group.
|
||||
|
||||
**Every command root runs is one fixed command**, with no shell syntax of its
|
||||
own -- ``useradd ...``, ``install -m 440 ...``, ``rm -f -- <path>`` -- so a
|
||||
sudoers allowlist of commands can name each one. What goes into a file is
|
||||
staged first, without privilege; root never reads stdin, where a sudo that
|
||||
does not prompt would leave the password.
|
||||
|
||||
**Nothing root runs follows the person's files.** The key files live in a home
|
||||
the person owns, where a symbolic link may point anywhere, so they are read and
|
||||
written as that person (``sudo -n -u <name>``). Only the existence of a key
|
||||
file is asked as root.
|
||||
|
||||
**The sudo rule is a file of its own** in the drop-in directory sudo includes,
|
||||
``<name> ALL=(ALL) ALL``: the person's own password, as on a new VM. A group
|
||||
alone is not enough, as some distributions grant ``%wheel`` and others
|
||||
``%sudo``. ``visudo`` checks the rule before it is installed, and ``sudo -l``
|
||||
afterwards shows whether it took effect. A rule elsewhere -- cloud-init's
|
||||
``90-cloud-init-users`` -- is not edited; it is reported.
|
||||
|
||||
**The login user's own account and system accounts are never touched**:
|
||||
locking the first would lock the caller out.
|
||||
|
||||
**netOrk's own account** (``add_service_account``) is made the same way,
|
||||
through the account it logs in with today: a key and no password, and a rule
|
||||
of its own -- ``<name> ALL=(ALL) NOPASSWD: ALL`` in ``85-service-<name>`` --
|
||||
so it gains root without a password that could end up anywhere. Once netOrk
|
||||
logs in with it, the person's account it used before is one like any other.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import shlex
|
||||
from collections.abc import Iterable, Sequence
|
||||
from typing import TYPE_CHECKING, ClassVar
|
||||
|
||||
from typing_extensions import TypedDict
|
||||
|
||||
from napalm_device_types.fixes import run_fix_steps
|
||||
from napalm_device_types.models import FixResultDict, FixStepDict, FixStepOutcomeDict
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - for type checkers only
|
||||
from napalm_device_types.channel import CommandResult
|
||||
|
||||
#: A login name every POSIX host takes as it is: lowercase, a letter or an
|
||||
#: underscore first, at most 32 characters. It is also safe in a root command.
|
||||
ACCOUNT_NAME_RE = re.compile(r"[a-z_][a-z0-9_.-]{0,31}")
|
||||
|
||||
#: The crypt(3) prefixes of each password scheme a driver may take.
|
||||
PASSWORD_SCHEMES: dict[str, tuple[str, ...]] = {
|
||||
"sha512": ("$6$",),
|
||||
"bcrypt": ("$2b$", "$2a$", "$2y$"),
|
||||
}
|
||||
_CRYPT_RE = re.compile(r"\$[0-9a-z]+\$[./0-9A-Za-z$]+")
|
||||
|
||||
#: The public key types OpenSSH takes in a key file.
|
||||
_KEY_TYPE_RE = re.compile(
|
||||
r"(ssh-ed25519|ssh-rsa|ecdsa-sha2-nistp(256|384|521)"
|
||||
r"|sk-ssh-ed25519@openssh\.com|sk-ecdsa-sha2-nistp256@openssh\.com)"
|
||||
)
|
||||
_KEY_BLOB_RE = re.compile(r"[A-Za-z0-9+/]+={0,3}")
|
||||
|
||||
#: Before the rule's file name; after cloud-init's ``90-`` it would win, so before it.
|
||||
SUDOERS_FILE_PREFIX = "80-account-"
|
||||
#: Before the file name of netOrk's own account's rule; after a person's, before cloud-init's.
|
||||
SERVICE_SUDOERS_FILE_PREFIX = "85-service-"
|
||||
|
||||
#: A line of ``sudo -l`` that lets the account run every command as root.
|
||||
_RUN_ALL_RE = re.compile(r"^\s*\(ALL(\s*:\s*ALL)?\)\s+([A-Z_]+:\s*)*ALL\s*$", re.MULTILINE)
|
||||
#: The same, without a password.
|
||||
_RUN_ALL_NOPASSWD_RE = re.compile(
|
||||
r"^\s*\(ALL(\s*:\s*ALL)?\)\s+([A-Z_]+:\s*)*NOPASSWD:\s*([A-Z_]+:\s*)*ALL\s*$",
|
||||
re.MULTILINE,
|
||||
)
|
||||
|
||||
#: Login shells that refuse every login: an account with one belongs to a service.
|
||||
_NO_LOGIN_SHELLS = frozenset({"nologin", "false"})
|
||||
|
||||
#: Stages stdin as a file anyone may read -- the person reads the key file
|
||||
#: from it -- and prints its path. The content is a public key or a sudo rule.
|
||||
_STAGE = 't=$(mktemp) && chmod 644 "$t" && cat > "$t" && printf %s "$t"'
|
||||
|
||||
|
||||
class LocalAccountDict(TypedDict):
|
||||
"""A person's account as the host has it (``get_local_accounts``)."""
|
||||
|
||||
name: str
|
||||
uid: int
|
||||
home: str
|
||||
shell: str
|
||||
#: Primary group first.
|
||||
groups: list[str]
|
||||
#: No login, with a key or a password.
|
||||
locked: bool
|
||||
#: ``sudo -l`` lets it run every command as root.
|
||||
admin: bool
|
||||
#: The lines of its key files; None when they could not be read (a locked
|
||||
#: account cannot run a command any more).
|
||||
authorized_keys: list[str] | None
|
||||
|
||||
|
||||
class LocalAccountChangeDict(FixResultDict):
|
||||
"""The result of ``add_local_account`` and ``lock_local_account``."""
|
||||
|
||||
#: The account read back after the change; None when it is not there or
|
||||
#: could not be read.
|
||||
account: LocalAccountDict | None
|
||||
|
||||
|
||||
def sudoers_rule(name: str) -> str:
|
||||
"""The rule that lets *name* run everything as root, with its own password."""
|
||||
return f"{name} ALL=(ALL) ALL\n"
|
||||
|
||||
|
||||
def sudoers_rule_path(directory: str, name: str) -> str:
|
||||
"""Where *name*'s rule lives in *directory*.
|
||||
|
||||
sudo skips a file whose name holds a dot, so a dot becomes ``_2e``, and an
|
||||
underscore ``__``: no two names share a file.
|
||||
"""
|
||||
return f"{directory}/{SUDOERS_FILE_PREFIX}{_rule_file_name(name)}"
|
||||
|
||||
|
||||
def service_sudoers_rule(name: str) -> str:
|
||||
"""The rule that lets netOrk's own account *name* run everything as root, no password."""
|
||||
return f"{name} ALL=(ALL) NOPASSWD: ALL\n"
|
||||
|
||||
|
||||
def service_sudoers_rule_path(directory: str, name: str) -> str:
|
||||
"""Where the rule of netOrk's own account *name* lives in *directory*."""
|
||||
return f"{directory}/{SERVICE_SUDOERS_FILE_PREFIX}{_rule_file_name(name)}"
|
||||
|
||||
|
||||
def _rule_file_name(name: str) -> str:
|
||||
return name.replace("_", "__").replace(".", "_2e")
|
||||
|
||||
|
||||
def authorized_keys_paths(home: str) -> tuple[str, str]:
|
||||
"""The key files sshd reads in *home* unless configured otherwise."""
|
||||
return f"{home}/.ssh/authorized_keys", f"{home}/.ssh/authorized_keys2"
|
||||
|
||||
|
||||
def check_account_name(name: str) -> None:
|
||||
""":raises ValueError: when *name* is no login name every host takes."""
|
||||
if not ACCOUNT_NAME_RE.fullmatch(name):
|
||||
raise ValueError(f"not an account name: {name!r}")
|
||||
|
||||
|
||||
def _key_blob(line: str) -> str | None:
|
||||
"""The base64 part of a key file line, options and comment aside."""
|
||||
words = line.split()
|
||||
for i, word in enumerate(words[:-1]):
|
||||
if _KEY_TYPE_RE.fullmatch(word):
|
||||
return words[i + 1]
|
||||
return None
|
||||
|
||||
|
||||
def check_public_key(key: str) -> None:
|
||||
""":raises ValueError: when *key* is not one OpenSSH public key on one line."""
|
||||
words = key.split()
|
||||
if (
|
||||
"\n" in key.strip()
|
||||
or len(words) < 2
|
||||
or not _KEY_TYPE_RE.fullmatch(words[0])
|
||||
or not _KEY_BLOB_RE.fullmatch(words[1])
|
||||
):
|
||||
raise ValueError("not a public key: expected '<type> <base64> [comment]' on one line")
|
||||
|
||||
|
||||
def _command(argv: Sequence[str]) -> str:
|
||||
return shlex.join(argv)
|
||||
|
||||
|
||||
def _as(name: str, *argv: str) -> str:
|
||||
"""*argv* run as *name*, from root: nothing it does reaches beyond that account."""
|
||||
return _command(["sudo", "-n", "-u", name, *argv])
|
||||
|
||||
|
||||
def _said(result: CommandResult) -> str:
|
||||
return (result.stderr or result.stdout).strip() or f"exit {result.exit_code}"
|
||||
|
||||
|
||||
class LocalAccountsMixin:
|
||||
"""Reads, adds and locks personal accounts; a driver supplies the OS's commands.
|
||||
|
||||
Mixed in by the drivers that can, so ``callable(getattr(driver,
|
||||
"add_local_account", None))`` says whether a driver can.
|
||||
"""
|
||||
|
||||
#: The login shell of a new account.
|
||||
ACCOUNT_SHELL: ClassVar[str] = "/bin/sh"
|
||||
#: The password scheme the host's accounts take, a key of :data:`PASSWORD_SCHEMES`.
|
||||
ACCOUNT_PASSWORD_SCHEME: ClassVar[str] = "sha512"
|
||||
#: Groups that make an account an administrator; a person without sudo leaves them.
|
||||
ACCOUNT_ADMIN_GROUPS: ClassVar[tuple[str, ...]] = ("wheel",)
|
||||
#: The lowest uid of a person's account; below are the system's.
|
||||
ACCOUNT_MIN_UID: ClassVar[int] = 1000
|
||||
#: The drop-in directory sudo includes.
|
||||
SUDOERS_DIR: ClassVar[str] = "/etc/sudoers.d"
|
||||
#: The group of root's files.
|
||||
ROOT_GROUP: ClassVar[str] = "root"
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def run_command(
|
||||
self,
|
||||
command: str,
|
||||
*,
|
||||
privileged: bool = False,
|
||||
timeout: float = 60,
|
||||
stdin: bytes | None = None,
|
||||
) -> CommandResult: ...
|
||||
|
||||
# -- what a driver supplies ------------------------------------------------------
|
||||
#
|
||||
# ``_account_*`` are the driver's: the five below it must supply, the key
|
||||
# store (``_account_key_files``, ``_account_read_keys``, ``_account_add_key``,
|
||||
# ``_account_remove_keys``) it may replace. ``_accounts_*`` are this mixin's own.
|
||||
|
||||
def _account_create(self, name: str, password_hash: str | None) -> list[list[str]]:
|
||||
"""The commands that create *name* with a home and :attr:`ACCOUNT_SHELL`.
|
||||
|
||||
Without *password_hash* the account takes no password, only a key.
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
def _account_open(
|
||||
self, name: str, password_hash: str | None, *, locked: bool
|
||||
) -> list[list[str]]:
|
||||
"""The commands that let *name* log in again, with *password_hash* if given."""
|
||||
raise NotImplementedError
|
||||
|
||||
def _account_lock(self, name: str, *, locked: bool) -> list[list[str]]:
|
||||
"""The commands that refuse *name* every login, a key's too; none when *locked*."""
|
||||
raise NotImplementedError
|
||||
|
||||
def _account_locked(self, name: str, passwd: list[str]) -> bool:
|
||||
"""Whether *name* is locked; *passwd* is its ``getent passwd`` line, split.
|
||||
|
||||
:raises RuntimeError: when that cannot be read.
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
def _account_leave_group(self, name: str, group: str, groups: list[str]) -> list[str]:
|
||||
"""The command that takes *name* out of *group*; *groups* are all of its own."""
|
||||
raise NotImplementedError
|
||||
|
||||
def _account_create_with_uid(self, name: str, uid: int) -> list[list[str]]:
|
||||
"""The commands that create *name* as :meth:`_account_create` does, without a
|
||||
password, and with the uid *uid*.
|
||||
|
||||
Only netOrk's own account asks for one (``add_service_account(uid=)``).
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
# -- reading ---------------------------------------------------------------------
|
||||
|
||||
def get_local_accounts(self, names: Iterable[str]) -> list[LocalAccountDict]:
|
||||
"""The accounts of *names* the host has, in that order; a missing one is left out.
|
||||
|
||||
:raises ValueError: for a name that is no account name.
|
||||
:raises RuntimeError: when an account cannot be read.
|
||||
"""
|
||||
wanted = list(names)
|
||||
for name in wanted:
|
||||
check_account_name(name)
|
||||
found = (self._accounts_read(name) for name in wanted)
|
||||
return [account for account in found if account is not None]
|
||||
|
||||
def _accounts_passwd(self, name: str) -> list[str] | None:
|
||||
result = self.run_command(_command(["getent", "passwd", name]), timeout=30)
|
||||
if result.exit_code == 2:
|
||||
return None
|
||||
fields = result.stdout.strip().split(":")
|
||||
if result.exit_code != 0 or len(fields) < 7 or fields[0] != name:
|
||||
raise RuntimeError(f"cannot read the account {name}: {_said(result)}")
|
||||
return fields
|
||||
|
||||
def _accounts_read(self, name: str) -> LocalAccountDict | None:
|
||||
passwd = self._accounts_passwd(name)
|
||||
if passwd is None:
|
||||
return None
|
||||
groups = self.run_command(_command(["id", "-Gn", name]), timeout=30)
|
||||
if groups.exit_code != 0:
|
||||
raise RuntimeError(f"cannot read the groups of {name}: {_said(groups)}")
|
||||
home = passwd[5]
|
||||
return {
|
||||
"name": name,
|
||||
"uid": int(passwd[2]),
|
||||
"home": home,
|
||||
"shell": passwd[6],
|
||||
"groups": groups.stdout.split(),
|
||||
"locked": self._account_locked(name, passwd),
|
||||
"admin": self._accounts_admin(name),
|
||||
"authorized_keys": self._account_read_keys(name, home),
|
||||
}
|
||||
|
||||
def _accounts_admin(self, name: str) -> bool:
|
||||
result = self.run_command(_command(["sudo", "-l", "-U", name]), privileged=True, timeout=30)
|
||||
if result.exit_code != 0:
|
||||
raise RuntimeError(f"cannot ask sudo about {name}: {_said(result)}")
|
||||
return bool(_RUN_ALL_RE.search(result.stdout))
|
||||
|
||||
def _account_key_files(self, name: str, home: str) -> tuple[str, ...]:
|
||||
"""Every file sshd reads *name*'s keys from; a host that adds one of its own says so."""
|
||||
return authorized_keys_paths(home)
|
||||
|
||||
def _account_read_keys(self, name: str, home: str) -> list[str] | None:
|
||||
"""The lines of :meth:`_account_key_files`, read as *name*; None when it cannot."""
|
||||
lines: list[str] = []
|
||||
for path in self._account_key_files(name, home):
|
||||
there = self.run_command(_command(["test", "-e", path]), privileged=True, timeout=30)
|
||||
if there.exit_code != 0:
|
||||
continue
|
||||
result = self.run_command(_as(name, "cat", "--", path), privileged=True, timeout=30)
|
||||
if result.exit_code != 0:
|
||||
return None
|
||||
lines += [ln for ln in result.stdout.splitlines() if ln.strip()]
|
||||
return lines
|
||||
|
||||
# -- changing ----------------------------------------------------------------------
|
||||
|
||||
def add_local_account(
|
||||
self,
|
||||
name: str,
|
||||
public_key: str,
|
||||
*,
|
||||
password_hash: str | None = None,
|
||||
admin: bool = False,
|
||||
) -> LocalAccountChangeDict:
|
||||
"""Give *name* an account that logs in with *public_key*, or open it again.
|
||||
|
||||
An account that is there is opened again and keeps its other keys; a
|
||||
new one gets a home. *password_hash* (crypt(3), in
|
||||
:attr:`ACCOUNT_PASSWORD_SCHEME`) becomes its password; *admin* gives it
|
||||
the sudo rule, which asks for that password -- so an administrator
|
||||
needs one. Without *admin*, the rule and :attr:`ACCOUNT_ADMIN_GROUPS`
|
||||
are taken away.
|
||||
|
||||
Steps: ``account``, then ``sudo`` and ``keys``, each of which runs when
|
||||
the other failed.
|
||||
|
||||
:raises ValueError: for a name, key or hash no host takes, the login
|
||||
user's own account, or a system account.
|
||||
"""
|
||||
check_account_name(name)
|
||||
check_public_key(public_key)
|
||||
if password_hash is not None:
|
||||
self._accounts_check_hash(password_hash)
|
||||
existing = self._accounts_changeable(name)
|
||||
first = run_fix_steps(
|
||||
[("account", lambda: self._accounts_open_or_create(name, existing, password_hash))]
|
||||
)
|
||||
parts = [first]
|
||||
if first["success"]:
|
||||
parts += [
|
||||
run_fix_steps([("sudo", lambda: self._accounts_set_sudo(name, admin=admin))]),
|
||||
run_fix_steps([("keys", lambda: self._accounts_add_key(name, public_key))]),
|
||||
]
|
||||
return self._accounts_result(name, parts)
|
||||
|
||||
def lock_local_account(self, name: str) -> LocalAccountChangeDict:
|
||||
"""Lock *name*: its keys go, every login is refused, sudo goes; its home stays.
|
||||
|
||||
Steps ``keys``, ``lock`` and ``sudo``, each run when one before failed:
|
||||
the lock matters most. A sudo rule outside the drop-in directory is
|
||||
reported, not edited -- with the account locked it can no longer be used.
|
||||
|
||||
:raises ValueError: for a name no host takes, the login user's own
|
||||
account, or a system account.
|
||||
"""
|
||||
check_account_name(name)
|
||||
if self._accounts_changeable(name) is None:
|
||||
step: FixStepDict = {"step": "account", "ok": False, "output": f"no account {name}"}
|
||||
return {**_joined([step]), "account": None}
|
||||
parts = [
|
||||
run_fix_steps([("keys", lambda: self._accounts_remove_keys(name))]),
|
||||
run_fix_steps([("lock", lambda: self._accounts_lock(name))]),
|
||||
run_fix_steps(
|
||||
[("sudo", lambda: self._accounts_set_sudo(name, admin=False, locked=True))]
|
||||
),
|
||||
]
|
||||
return self._accounts_result(name, parts)
|
||||
|
||||
def add_service_account(
|
||||
self, name: str, public_key: str, *, uid: int | None = None
|
||||
) -> LocalAccountChangeDict:
|
||||
"""Make *name* netOrk's own account: it logs in with *public_key*, and gains
|
||||
root without a password.
|
||||
|
||||
Created through the account the driver logs in with now, or opened
|
||||
again when it is there; its other keys stay. It takes no password. Run
|
||||
again with the account the driver already logs in with, it only adds
|
||||
what is missing.
|
||||
|
||||
A new account gets *uid* when one is given -- the same on every host --
|
||||
once no other account has it; an account that is there keeps its own,
|
||||
as a uid is never changed under files that carry it.
|
||||
|
||||
Steps as in :meth:`add_local_account`: ``account``, then ``sudo`` and
|
||||
``keys``. The new login itself is for the caller to try.
|
||||
|
||||
An account that is there and cannot log in (``nologin``, ``false``) is
|
||||
another service's -- a satellite's ``netork``, say -- and is left alone.
|
||||
|
||||
:raises ValueError: for a name or key no host takes, root, a uid of 0 or
|
||||
less, or another service's account.
|
||||
:raises RuntimeError: when the login user cannot be told.
|
||||
"""
|
||||
check_public_key(public_key)
|
||||
login, existing = self._accounts_service_target(name, uid)
|
||||
if name == login:
|
||||
step: FixStepDict = {
|
||||
"step": "account",
|
||||
"ok": True,
|
||||
"output": f"this driver logs in as {name} already",
|
||||
}
|
||||
first: FixResultDict = _joined([step])
|
||||
else:
|
||||
first = run_fix_steps(
|
||||
[("account", lambda: self._accounts_service_account(name, existing, uid))]
|
||||
)
|
||||
parts = [first]
|
||||
if first["success"]:
|
||||
parts += [
|
||||
run_fix_steps([("sudo", lambda: self._accounts_grant_service_sudo(name))]),
|
||||
run_fix_steps([("keys", lambda: self._accounts_add_key(name, public_key))]),
|
||||
]
|
||||
return self._accounts_result(name, parts)
|
||||
|
||||
def _accounts_service_target(self, name: str, uid: int | None) -> tuple[str, list[str] | None]:
|
||||
"""The login user and *name*'s passwd line, once *name* may become netOrk's.
|
||||
|
||||
:raises ValueError: as :meth:`add_service_account` says.
|
||||
"""
|
||||
check_account_name(name)
|
||||
if uid is not None and uid <= 0:
|
||||
raise ValueError(f"not a uid for netOrk's own account: {uid}")
|
||||
login = self._accounts_login()
|
||||
existing = self._accounts_passwd(name)
|
||||
if existing is not None and int(existing[2]) == 0:
|
||||
raise ValueError(f"{name} is root")
|
||||
if existing is not None and existing[6].rsplit("/", 1)[-1] in _NO_LOGIN_SHELLS:
|
||||
raise ValueError(
|
||||
f"{name} is there and cannot log in ({existing[6]}): another service's account"
|
||||
)
|
||||
return login, existing
|
||||
|
||||
def _accounts_service_account(
|
||||
self, name: str, passwd: list[str] | None, uid: int | None
|
||||
) -> FixStepOutcomeDict:
|
||||
"""netOrk's own account opened or made, a new one with *uid* when given."""
|
||||
if uid is None:
|
||||
return self._accounts_open_or_create(name, passwd, None)
|
||||
if passwd is not None:
|
||||
outcome = self._accounts_open_or_create(name, passwd, None)
|
||||
if outcome["ok"] and int(passwd[2]) != uid:
|
||||
outcome["output"] += f"; it keeps its uid {passwd[2]}, not {uid}"
|
||||
return outcome
|
||||
owner = self._accounts_passwd_of_uid(uid)
|
||||
if owner is not None:
|
||||
return {"ok": False, "output": f"uid {uid} is the account {owner}'s"}
|
||||
try:
|
||||
create = self._account_create_with_uid(name, uid)
|
||||
except NotImplementedError:
|
||||
return {"ok": False, "output": f"this driver cannot give an account the uid {uid}"}
|
||||
failed = self._accounts_run(create)
|
||||
if failed is not None:
|
||||
return {"ok": False, "output": _said(failed)}
|
||||
return {"ok": True, "output": f"account {name} created, uid {uid}"}
|
||||
|
||||
def _accounts_passwd_of_uid(self, uid: int) -> str | None:
|
||||
"""The name of the account with *uid*, or None."""
|
||||
result = self.run_command(_command(["getent", "passwd", str(uid)]), timeout=30)
|
||||
if result.exit_code == 2:
|
||||
return None
|
||||
if result.exit_code != 0:
|
||||
raise RuntimeError(f"cannot ask who has uid {uid}: {_said(result)}")
|
||||
return result.stdout.split(":", 1)[0].strip() or None
|
||||
|
||||
def _accounts_check_hash(self, password_hash: str) -> None:
|
||||
prefixes = PASSWORD_SCHEMES[self.ACCOUNT_PASSWORD_SCHEME]
|
||||
if not _CRYPT_RE.fullmatch(password_hash) or not password_hash.startswith(prefixes):
|
||||
raise ValueError(f"not a {self.ACCOUNT_PASSWORD_SCHEME} password hash")
|
||||
|
||||
def _accounts_changeable(self, name: str) -> list[str] | None:
|
||||
"""*name*'s passwd line, None when there is none.
|
||||
|
||||
:raises ValueError: for the login user's own account or a system account.
|
||||
:raises RuntimeError: when the login user cannot be told.
|
||||
"""
|
||||
if name == self._accounts_login():
|
||||
raise ValueError(f"{name} is the account this driver logs in with")
|
||||
passwd = self._accounts_passwd(name)
|
||||
if passwd is not None and int(passwd[2]) < self.ACCOUNT_MIN_UID:
|
||||
raise ValueError(f"{name} is a system account (uid {passwd[2]})")
|
||||
return passwd
|
||||
|
||||
def _accounts_login(self) -> str:
|
||||
""":raises RuntimeError: when the login user cannot be told."""
|
||||
login = self.run_command("id -un", timeout=30).stdout.strip()
|
||||
if not login:
|
||||
raise RuntimeError("cannot tell which account this driver logs in with")
|
||||
return login
|
||||
|
||||
def _accounts_result(self, name: str, parts: list[FixResultDict]) -> LocalAccountChangeDict:
|
||||
steps = [s for part in parts for s in part["steps"]]
|
||||
result = _joined(steps)
|
||||
account: LocalAccountDict | None = None
|
||||
try:
|
||||
account = self._accounts_read(name)
|
||||
except RuntimeError as exc:
|
||||
result["output"] += f"\n[read back] {exc}"
|
||||
return {**result, "account": account}
|
||||
|
||||
def _accounts_root(self, argv: Sequence[str]) -> CommandResult:
|
||||
return self.run_command(_command(argv), privileged=True, timeout=60)
|
||||
|
||||
def _accounts_run(self, commands: list[list[str]]) -> CommandResult | None:
|
||||
"""Run *commands* as root in order; the first that failed, or None."""
|
||||
for argv in commands:
|
||||
result = self._accounts_root(argv)
|
||||
if result.exit_code != 0:
|
||||
return result
|
||||
return None
|
||||
|
||||
def _accounts_open_or_create(
|
||||
self, name: str, passwd: list[str] | None, password_hash: str | None
|
||||
) -> FixStepOutcomeDict:
|
||||
if passwd is None:
|
||||
failed = self._accounts_run(self._account_create(name, password_hash))
|
||||
done = f"account {name} created"
|
||||
else:
|
||||
locked = self._account_locked(name, passwd)
|
||||
failed = self._accounts_run(self._account_open(name, password_hash, locked=locked))
|
||||
done = f"account {name} opened again" if locked else f"account {name} is there"
|
||||
if failed is not None:
|
||||
return {"ok": False, "output": _said(failed)}
|
||||
return {"ok": True, "output": done + (", password set" if password_hash else "")}
|
||||
|
||||
def _accounts_lock(self, name: str) -> FixStepOutcomeDict:
|
||||
passwd = self._accounts_passwd(name)
|
||||
if passwd is None:
|
||||
return {"ok": False, "output": f"no account {name}"}
|
||||
locked = self._account_locked(name, passwd)
|
||||
failed = self._accounts_run(self._account_lock(name, locked=locked))
|
||||
if failed is not None:
|
||||
return {"ok": False, "output": _said(failed)}
|
||||
if not self._account_locked(name, passwd):
|
||||
return {"ok": False, "output": f"{name} is still not locked"}
|
||||
return {"ok": True, "output": f"{name} was locked already" if locked else f"{name} locked"}
|
||||
|
||||
# -- sudo --------------------------------------------------------------------------
|
||||
|
||||
def _accounts_set_sudo(
|
||||
self, name: str, *, admin: bool, locked: bool = False
|
||||
) -> FixStepOutcomeDict:
|
||||
path = sudoers_rule_path(self.SUDOERS_DIR, name)
|
||||
if admin:
|
||||
return self._accounts_grant_sudo(name, path)
|
||||
said: list[str] = []
|
||||
removed = self._accounts_root(["rm", "-f", "--", path])
|
||||
if removed.exit_code != 0:
|
||||
return {"ok": False, "output": f"{path}: {_said(removed)}"}
|
||||
groups = self.run_command(_command(["id", "-Gn", name]), timeout=30).stdout.split()
|
||||
for group in self.ACCOUNT_ADMIN_GROUPS:
|
||||
if group not in groups[1:]:
|
||||
continue
|
||||
left = self._accounts_root(self._account_leave_group(name, group, groups))
|
||||
if left.exit_code != 0:
|
||||
return {"ok": False, "output": f"cannot take {name} out of {group}: {_said(left)}"}
|
||||
groups.remove(group)
|
||||
said.append(f"{name} left {group}")
|
||||
if self._accounts_admin(name):
|
||||
rest = f"another rule than {path} still lets {name} run everything as root"
|
||||
if locked:
|
||||
return {"ok": True, "output": f"{rest}; the account is locked, so it cannot use it"}
|
||||
return {"ok": False, "output": rest}
|
||||
said.append(f"{name} has no sudo")
|
||||
return {"ok": True, "output": "\n".join(said)}
|
||||
|
||||
def _accounts_grant_sudo(self, name: str, path: str) -> FixStepOutcomeDict:
|
||||
failed = self._accounts_install_rule(sudoers_rule(name), path)
|
||||
if failed is not None:
|
||||
return failed
|
||||
if not self._accounts_admin(name):
|
||||
return {
|
||||
"ok": False,
|
||||
"output": f"{path} is written, but sudo does not read it: "
|
||||
f"does sudoers include {self.SUDOERS_DIR}?",
|
||||
}
|
||||
return {"ok": True, "output": f"{name} may use sudo with its password ({path})"}
|
||||
|
||||
def _accounts_grant_service_sudo(self, name: str) -> FixStepOutcomeDict:
|
||||
path = service_sudoers_rule_path(self.SUDOERS_DIR, name)
|
||||
failed = self._accounts_install_rule(service_sudoers_rule(name), path)
|
||||
if failed is not None:
|
||||
return failed
|
||||
result = self.run_command(_command(["sudo", "-l", "-U", name]), privileged=True, timeout=30)
|
||||
if result.exit_code != 0:
|
||||
return {"ok": False, "output": f"cannot ask sudo about {name}: {_said(result)}"}
|
||||
if not _RUN_ALL_NOPASSWD_RE.search(result.stdout):
|
||||
return {
|
||||
"ok": False,
|
||||
"output": f"{path} is written, but sudo does not read it: "
|
||||
f"does sudoers include {self.SUDOERS_DIR}?",
|
||||
}
|
||||
return {"ok": True, "output": f"{name} may use sudo without a password ({path})"}
|
||||
|
||||
def _accounts_install_rule(self, rule: str, path: str) -> FixStepOutcomeDict | None:
|
||||
"""Check *rule* with visudo and install it at *path*; the failure, or None."""
|
||||
staged = self._accounts_stage(rule)
|
||||
try:
|
||||
checked = self._accounts_root(["visudo", "-cf", staged])
|
||||
if checked.exit_code != 0:
|
||||
return {"ok": False, "output": f"visudo refuses the rule: {_said(checked)}"}
|
||||
installed = self._accounts_root(
|
||||
["install", "-m", "440", "-o", "root", "-g", self.ROOT_GROUP, "--", staged, path]
|
||||
)
|
||||
if installed.exit_code != 0:
|
||||
return {"ok": False, "output": f"{path}: {_said(installed)}"}
|
||||
finally:
|
||||
self._accounts_unstage(staged)
|
||||
return None
|
||||
|
||||
# -- keys --------------------------------------------------------------------------
|
||||
|
||||
def _accounts_stage(self, content: str) -> str:
|
||||
"""*content* in a fresh file of the login user's; its path.
|
||||
|
||||
:raises RuntimeError: when it cannot be staged.
|
||||
"""
|
||||
result = self.run_command(_STAGE, timeout=30, stdin=content.encode())
|
||||
path = result.stdout.strip().splitlines()[-1:] or [""]
|
||||
if result.exit_code != 0 or not path[0].startswith("/"):
|
||||
raise RuntimeError(f"cannot stage a file: {_said(result)}")
|
||||
return path[0]
|
||||
|
||||
def _accounts_unstage(self, path: str) -> None:
|
||||
self.run_command(_command(["rm", "-f", "--", path]), timeout=30)
|
||||
|
||||
def _accounts_home(self, name: str) -> str | None:
|
||||
passwd = self._accounts_passwd(name)
|
||||
return passwd[5] if passwd is not None else None
|
||||
|
||||
def _accounts_add_key(self, name: str, public_key: str) -> FixStepOutcomeDict:
|
||||
home = self._accounts_home(name)
|
||||
if home is None:
|
||||
return {"ok": False, "output": f"no account {name}"}
|
||||
return self._account_add_key(name, home, public_key)
|
||||
|
||||
def _accounts_remove_keys(self, name: str) -> FixStepOutcomeDict:
|
||||
home = self._accounts_home(name)
|
||||
if home is None:
|
||||
return {"ok": False, "output": f"no account {name}"}
|
||||
return self._account_remove_keys(name, home)
|
||||
|
||||
def _account_add_key(self, name: str, home: str, public_key: str) -> FixStepOutcomeDict:
|
||||
"""Add *public_key* to *name*'s first key file, as *name*; the others stay.
|
||||
|
||||
A driver whose host keeps keys elsewhere overrides this, with
|
||||
:meth:`_account_read_keys` and :meth:`_account_remove_keys`.
|
||||
"""
|
||||
path = authorized_keys_paths(home)[0]
|
||||
current = self._account_read_keys(name, home)
|
||||
if current is None:
|
||||
return {"ok": False, "output": f"cannot read the key files of {name}"}
|
||||
blob = _key_blob(public_key)
|
||||
if any(_key_blob(line) == blob for line in current):
|
||||
return {"ok": True, "output": f"the key is already in {path}"}
|
||||
mine = self._accounts_key_file(name, path)
|
||||
if mine is None:
|
||||
return {"ok": False, "output": f"cannot read {path}"}
|
||||
staged = self._accounts_stage("".join(f"{line}\n" for line in [*mine, public_key.strip()]))
|
||||
try:
|
||||
failed = self._accounts_run_as(
|
||||
name,
|
||||
[
|
||||
["mkdir", "-p", "-m", "700", "--", f"{home}/.ssh"],
|
||||
["install", "-m", "600", "--", staged, path],
|
||||
],
|
||||
)
|
||||
finally:
|
||||
self._accounts_unstage(staged)
|
||||
if failed is not None:
|
||||
return {"ok": False, "output": f"{path}: {_said(failed)}"}
|
||||
return {"ok": True, "output": f"key added to {path}"}
|
||||
|
||||
def _accounts_key_file(self, name: str, path: str) -> list[str] | None:
|
||||
there = self.run_command(_command(["test", "-e", path]), privileged=True, timeout=30)
|
||||
if there.exit_code != 0:
|
||||
return []
|
||||
result = self.run_command(_as(name, "cat", "--", path), privileged=True, timeout=30)
|
||||
if result.exit_code != 0:
|
||||
return None
|
||||
return [line for line in result.stdout.splitlines() if line.strip()]
|
||||
|
||||
def _accounts_run_as(self, name: str, commands: list[list[str]]) -> CommandResult | None:
|
||||
for argv in commands:
|
||||
result = self.run_command(_as(name, *argv), privileged=True, timeout=60)
|
||||
if result.exit_code != 0:
|
||||
return result
|
||||
return None
|
||||
|
||||
def _account_remove_keys(self, name: str, home: str) -> FixStepOutcomeDict:
|
||||
"""Remove *name*'s key files in *home*, as *name*; then no key may be left."""
|
||||
paths = authorized_keys_paths(home)
|
||||
failed = self._accounts_run_as(name, [["rm", "-f", "--", *paths]])
|
||||
if failed is not None:
|
||||
return {"ok": False, "output": f"cannot remove the keys of {name}: {_said(failed)}"}
|
||||
if self._account_read_keys(name, home):
|
||||
return {"ok": False, "output": f"{name} still has keys"}
|
||||
return {"ok": True, "output": f"keys of {name} removed"}
|
||||
|
||||
|
||||
def _joined(steps: list[FixStepDict]) -> FixResultDict:
|
||||
return {
|
||||
"success": bool(steps) and all(s["ok"] for s in steps),
|
||||
"steps": steps,
|
||||
"output": "\n".join(f"[{s['step']}] {s['output']}".rstrip() for s in steps),
|
||||
}
|
||||
|
||||
|
||||
__all__ = [
|
||||
"ACCOUNT_NAME_RE",
|
||||
"PASSWORD_SCHEMES",
|
||||
"SERVICE_SUDOERS_FILE_PREFIX",
|
||||
"SUDOERS_FILE_PREFIX",
|
||||
"LocalAccountChangeDict",
|
||||
"LocalAccountDict",
|
||||
"LocalAccountsMixin",
|
||||
"authorized_keys_paths",
|
||||
"check_account_name",
|
||||
"check_public_key",
|
||||
"service_sudoers_rule",
|
||||
"service_sudoers_rule_path",
|
||||
"sudoers_rule",
|
||||
"sudoers_rule_path",
|
||||
]
|
||||
@@ -5,9 +5,15 @@ These supplement the types defined in napalm.base.models and are used by the
|
||||
abstract device-type driver classes in this package.
|
||||
"""
|
||||
|
||||
from typing import Literal, get_args
|
||||
from typing import Literal, NotRequired, get_args
|
||||
|
||||
from typing_extensions import NotRequired, TypedDict
|
||||
from typing_extensions import TypedDict
|
||||
|
||||
# The SNMP health profile shapes live in snmp_health_profile.py; re-exported here, where they were.
|
||||
from napalm_device_types.snmp_health_profile import SnmpHealthProfile as SnmpHealthProfile
|
||||
from napalm_device_types.snmp_health_profile import SnmpInterfacesSpec as SnmpInterfacesSpec
|
||||
from napalm_device_types.snmp_health_profile import SnmpMemorySpec as SnmpMemorySpec
|
||||
from napalm_device_types.snmp_health_profile import SnmpValueSpec as SnmpValueSpec
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Common (shared across device types)
|
||||
@@ -1032,15 +1038,6 @@ class ProcessDict(TypedDict):
|
||||
command: str # full command line
|
||||
|
||||
|
||||
class CronJobDict(TypedDict):
|
||||
"""A scheduled cron task."""
|
||||
|
||||
user: str # owner of the crontab entry
|
||||
schedule: str # five-field cron expression, e.g. "0 * * * *"
|
||||
command: str # shell command to execute
|
||||
description: NotRequired[str] # optional inline comment
|
||||
|
||||
|
||||
class ApplyUpdatesResultDict(TypedDict):
|
||||
"""Result of an ``apply_updates()`` call."""
|
||||
|
||||
@@ -1230,53 +1227,6 @@ class SnmpAgentSettingsDict(TypedDict):
|
||||
listen: NotRequired[str] # the address the agent binds to; absent: every address
|
||||
|
||||
|
||||
class SnmpValueSpec(TypedDict, total=False):
|
||||
"""Where an SNMP agent publishes one value, inside a ``SnmpHealthProfile``.
|
||||
|
||||
Either ``oids``, scalars asked in order (the first non-empty answer wins), or
|
||||
``table``, a column walked and folded to one value by ``fold``. ``pattern`` is a
|
||||
regular expression whose first group is the number, for an agent that answers
|
||||
with text.
|
||||
"""
|
||||
|
||||
oids: list[str]
|
||||
table: str
|
||||
fold: Literal["first", "average", "max"]
|
||||
pattern: str
|
||||
|
||||
|
||||
class SnmpMemorySpec(TypedDict, total=False):
|
||||
"""Memory: ``total`` with ``used`` or ``free``, or ``percent`` alone."""
|
||||
|
||||
total: SnmpValueSpec
|
||||
used: SnmpValueSpec
|
||||
free: SnmpValueSpec
|
||||
percent: SnmpValueSpec
|
||||
unit: Literal["KiB", "bytes"] # of total, used and free; KiB, as UCD-MIB counts, by default
|
||||
|
||||
|
||||
class SnmpInterfacesSpec(TypedDict, total=False):
|
||||
"""How the IF-MIB counters of a device are to be read."""
|
||||
|
||||
skip: str # a pattern of interface names whose counters are noise, not signal
|
||||
tx_errors_are_drops: bool # the device counts dropped packets as TX errors (OPNsense)
|
||||
|
||||
|
||||
class SnmpHealthProfile(TypedDict, total=False):
|
||||
"""Where a device's SNMP agent publishes its health values.
|
||||
|
||||
A driver declares it as ``SNMP_HEALTH_PROFILE`` when its device does not answer
|
||||
the standard UCD-MIB and IF-MIB; the application polls and computes. What a
|
||||
profile does not name is read the standard way. Uptime is in TimeTicks, CPU in
|
||||
percent busy.
|
||||
"""
|
||||
|
||||
uptime: SnmpValueSpec
|
||||
cpu: SnmpValueSpec
|
||||
memory: SnmpMemorySpec
|
||||
interfaces: SnmpInterfacesSpec
|
||||
|
||||
|
||||
class NICConfigDict(TypedDict):
|
||||
"""Network interface configuration for VM provisioning."""
|
||||
|
||||
@@ -1310,17 +1260,17 @@ class NetworkTargetDict(TypedDict):
|
||||
Distinguishes real bridges (Linux or OVS) from SDN network segments (vnets),
|
||||
and tells the caller whether a separate ``vlan_tag`` may be applied on top:
|
||||
- Linux bridge: vlan_aware reflects the bridge's own ``bridge_vlan_aware`` flag.
|
||||
- OVS bridge: always vlan_aware (OVS bridges tag per-port regardless of a
|
||||
dedicated "VLAN aware" setting).
|
||||
- SDN vnet: never vlan_aware — the VLAN is already fixed by the vnet's zone/tag,
|
||||
so a NIC attached to it must not also carry a ``vlan_tag``. That fixed VLAN
|
||||
is surfaced via ``fixed_vlan_tag`` instead, for display purposes.
|
||||
- OVS bridge: always vlan_aware (OVS tags per port, with no "VLAN aware" setting).
|
||||
- SDN vnet: never vlan_aware — its zone/tag fixes the VLAN (``fixed_vlan_tag``).
|
||||
``uplinks``: the physical NICs it leaves the host through, one group per way out
|
||||
(a bond's slaves together, a single NIC alone); no internal, VLAN or virtual ports.
|
||||
"""
|
||||
|
||||
name: str # Bridge, vnet or port group name, usable directly as NICConfigDict.bridge
|
||||
kind: str # "bridge", "vnet" or "portgroup" (VMware: VLAN fixed like a vnet's)
|
||||
vlan_aware: bool # True if a NICConfigDict.vlan_tag may be set on top of this target
|
||||
fixed_vlan_tag: NotRequired[int | None] # vnet only: the VLAN ID already baked into it
|
||||
uplinks: NotRequired[list[list[str]]] # NIC groups it leaves through; absent = unknown
|
||||
|
||||
|
||||
class StorageTargetDict(TypedDict):
|
||||
|
||||
@@ -15,7 +15,6 @@ from typing import TYPE_CHECKING
|
||||
from napalm_device_types.base import DeviceTypeDriver
|
||||
from napalm_device_types.health_metrics import HealthMetricsMixin
|
||||
from napalm_device_types.models import (
|
||||
CronJobDict,
|
||||
DeviceActionResultDict,
|
||||
ProcessDict,
|
||||
SNMPConfigDict,
|
||||
@@ -131,34 +130,6 @@ class OSDriver(
|
||||
"""
|
||||
...
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Cron jobs
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def get_cron_jobs(self) -> list[CronJobDict]:
|
||||
"""
|
||||
Returns scheduled cron tasks from all user crontabs and ``/etc/cron.d``.
|
||||
|
||||
Each entry contains:
|
||||
|
||||
* user (string) - owner of the crontab entry
|
||||
* schedule (string) - five-field cron expression (e.g. ``"0 * * * *"``)
|
||||
* command (string) - shell command to execute
|
||||
* description (string, optional) - inline comment text if present
|
||||
|
||||
Example::
|
||||
|
||||
[
|
||||
{
|
||||
"user": "root",
|
||||
"schedule": "0 4 * * *",
|
||||
"command": "/usr/local/bin/backup.sh",
|
||||
"description": "nightly backup",
|
||||
},
|
||||
]
|
||||
"""
|
||||
...
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# SNMP
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
@@ -28,9 +28,12 @@ APT_UPGRADABLE_COMMAND = "{ LC_ALL=C apt list --upgradable 2>/dev/null; echo __A
|
||||
|
||||
#: Read-only. Lists the packages that a pending security advisory covers. A
|
||||
#: failed dnf prints nothing, the same as no advisories, so its exit status is
|
||||
#: printed inside the group, as apt's is.
|
||||
#: printed inside the group, as apt's is. It reads the cached metadata only, as
|
||||
#: apt reads its local index: an expired cache otherwise makes dnf download the
|
||||
#: repositories' metadata first, which can take minutes (#76).
|
||||
DNF_SECURITY_COMMAND = (
|
||||
"{ LC_ALL=C dnf updateinfo list --security --quiet 2>/dev/null; echo __DNF_RC=$?; } | cat"
|
||||
"{ LC_ALL=C dnf --cacheonly updateinfo list --security --quiet 2>/dev/null;"
|
||||
" echo __DNF_RC=$?; } | cat"
|
||||
)
|
||||
|
||||
# openssl/noble-updates,noble-security 3.0.13-0ubuntu3.6 amd64 [upgradable from: 3.0.13-0ubuntu3.5]
|
||||
|
||||
@@ -3,8 +3,9 @@ What provisioning a VM from a cloud image needs to know about the guest.
|
||||
|
||||
The same for every hypervisor: which agent a guest runs so a QEMU-based
|
||||
hypervisor can read its IP, the network-config cloud-init and FreeBSD's
|
||||
nuageinit both read, and how a packed cloud image is unpacked. How a
|
||||
hypervisor attaches, boots and talks to the VM stays in its driver.
|
||||
nuageinit both read, how a packed cloud image is unpacked, and how a failure
|
||||
names the VM it left behind. How a hypervisor attaches, boots and talks to
|
||||
the VM stays in its driver.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -37,6 +38,30 @@ _DECOMPRESSORS = {
|
||||
}
|
||||
|
||||
|
||||
class VMProvisionIncomplete(RuntimeError):
|
||||
"""``create_vm_from_cloud_init`` failed after the VM was created.
|
||||
|
||||
The VM is still on the hypervisor, half built: ``vmid`` and ``node`` name it
|
||||
as ``VMProvisionResultDict`` would have, so the caller can remove it with
|
||||
``destroy_vm``. The failure itself is the ``__cause__`` (``raise ... from``),
|
||||
and the message is its own. A failure before the VM exists raises as before.
|
||||
"""
|
||||
|
||||
def __init__(self, message: str, *, vmid: str, node: str) -> None:
|
||||
super().__init__(message)
|
||||
self.vmid = vmid
|
||||
self.node = node
|
||||
|
||||
def __reduce__(self) -> tuple[Any, ...]:
|
||||
# The keyword-only arguments are not in ``args``, which is all that
|
||||
# pickle passes back by default.
|
||||
return (_vm_provision_incomplete, (str(self), self.vmid, self.node))
|
||||
|
||||
|
||||
def _vm_provision_incomplete(message: str, vmid: str, node: str) -> VMProvisionIncomplete:
|
||||
return VMProvisionIncomplete(message, vmid=vmid, node=node)
|
||||
|
||||
|
||||
def network_config(nics: list[tuple[str, bool]]) -> dict[str, Any] | None:
|
||||
"""Network-config v2: DHCP on every ``(mac, dhcp)`` NIC that asks for it.
|
||||
|
||||
|
||||
@@ -0,0 +1,196 @@
|
||||
"""Scheduled jobs: what a host runs on its own schedule, and switching it on, off and now.
|
||||
|
||||
A host runs work on its own: cron lines, run-parts scripts, anacron, systemd
|
||||
timers, FreeBSD's ``periodic``, a firewall's or a NAS's own scheduled jobs, the
|
||||
Windows Task Scheduler. :class:`ScheduledJobsMixin` reads all of them as one
|
||||
shape, :class:`~napalm_device_types.models.ScheduledJobDict`, and acts on them.
|
||||
Declared under ``if TYPE_CHECKING`` like a role: nothing exists at runtime until
|
||||
a driver implements it. :class:`~napalm_device_types.scheduled_jobs_posix.PosixScheduledJobsMixin`
|
||||
is the implementation for a host reached through the command channel.
|
||||
|
||||
**Who owns a definition** is on every job (``managed_by``): ``package`` (installed
|
||||
by one), ``admin`` (written on the host), ``platform`` (generated by the device's
|
||||
own configuration -- OPNsense, OpenMediaVault, QTS, Windows), ``application`` (written
|
||||
through :meth:`put_scheduled_job`, under the ``name`` the application gave it).
|
||||
What the application makes of that is the application's.
|
||||
|
||||
**What may be done is on every job too** (``actions``): every job a driver can
|
||||
switch takes ``enable``, ``disable`` and ``run``; only an ``application`` job takes
|
||||
``edit`` and ``delete``. A driver never edits or removes a job it did not write.
|
||||
|
||||
**Never over a change it did not see.** Every write names the ``digest`` the caller
|
||||
saw. The driver reads the job again right before it writes and raises
|
||||
:class:`ScheduledJobChanged` when the definition differs.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import re
|
||||
from typing import TYPE_CHECKING, Any, NotRequired, TypedDict
|
||||
|
||||
from napalm_device_types.cron import is_reboot, parse_cron
|
||||
|
||||
#: What :meth:`ScheduledJobsMixin.manage_scheduled_job` takes.
|
||||
ACTIONS = ("enable", "disable", "run")
|
||||
|
||||
#: What a job's ``actions`` may list beyond :data:`ACTIONS`.
|
||||
EDIT = "edit"
|
||||
DELETE = "delete"
|
||||
|
||||
KINDS = ("cron", "run_parts", "anacron", "systemd_timer", "periodic", "task")
|
||||
MANAGED_BY = ("package", "admin", "platform", "application")
|
||||
|
||||
#: Seconds a run now may take on the host before ``timeout`` stops it.
|
||||
RUN_TIMEOUT = 300
|
||||
|
||||
#: A job's name becomes a file and a unit name: lower case, digits, dashes.
|
||||
_NAME = re.compile(r"[a-z0-9][a-z0-9-]{0,62}")
|
||||
#: A user name as useradd and the BSDs take it.
|
||||
_USER = re.compile(r"[A-Za-z_][A-Za-z0-9_.-]{0,31}\$?")
|
||||
|
||||
#: The parts of a job that are its definition; what the host says of its runs is not.
|
||||
_DEFINITION = ("kind", "source", "run_as", "schedule", "command", "enabled")
|
||||
|
||||
|
||||
class ScheduledJobDict(TypedDict):
|
||||
"""A job the host runs on its own schedule (``get_scheduled_jobs``).
|
||||
|
||||
A cron line, a run-parts script, an anacron job, a systemd timer, a FreeBSD
|
||||
``periodic`` script, a platform's own job, a Windows task -- one shape for all
|
||||
of them (``napalm_device_types.scheduled_jobs``).
|
||||
"""
|
||||
|
||||
key: str # stable name on this host: "timer:apt-daily.timer", "cron:/etc/crontab:1a2b3c4d"
|
||||
kind: str # cron | run_parts | anacron | systemd_timer | periodic | task
|
||||
source: str # the file, unit file or configuration it is defined in
|
||||
run_as: str # the user it runs as
|
||||
schedule: str # as the host has it: "*/5 * * * *", "OnCalendar=daily", "daily"
|
||||
schedule_kind: str # cron | calendar | monotonic | period | triggers
|
||||
command: str # as the host has it
|
||||
enabled: bool
|
||||
managed_by: str # package | admin | platform | application
|
||||
digest: str # job_digest(): the definition as it is now
|
||||
actions: list[str] # what the driver can do to this job: enable, disable, run, edit, delete
|
||||
name: NotRequired[str] # the name an application gave a job it wrote (put_scheduled_job)
|
||||
description: NotRequired[str]
|
||||
last_run: NotRequired[str | None] # ISO 8601, UTC; where the host knows it
|
||||
last_result: NotRequired[str | None] # "success", or what the host says went wrong
|
||||
next_run: NotRequired[str | None] # ISO 8601, UTC; where the host knows it
|
||||
|
||||
|
||||
class ScheduledJobsReadingDict(TypedDict):
|
||||
"""Every scheduled job of a host, and the time zone its schedules are in."""
|
||||
|
||||
jobs: list[ScheduledJobDict]
|
||||
time_zone: str | None # IANA name ("Europe/Berlin"); None when the host does not say
|
||||
|
||||
|
||||
class ScheduledJobSpecDict(TypedDict):
|
||||
"""A job an application wants on the host (``put_scheduled_job``)."""
|
||||
|
||||
name: str # the job's name on the host, chosen by the application: [a-z0-9][a-z0-9-]*
|
||||
schedule: str # a cron expression (napalm_device_types.cron), "@reboot" included
|
||||
run_as: str
|
||||
command: str # one or more lines of POSIX sh; a platform may take its own (OPNsense: configd)
|
||||
enabled: bool
|
||||
description: NotRequired[str]
|
||||
|
||||
|
||||
class ScheduledJobResultDict(TypedDict):
|
||||
"""What an action on a scheduled job did."""
|
||||
|
||||
success: bool
|
||||
output: str # what the host said; for a run, what the job printed
|
||||
|
||||
|
||||
class ScheduledJobChanged(Exception): # noqa: N818 - a state, as the callers word it
|
||||
"""The job's definition differs from the digest the caller saw: nothing was written."""
|
||||
|
||||
|
||||
class ScheduledJobRefused(ValueError):
|
||||
"""The job cannot take what was asked of it, or the request itself is not valid."""
|
||||
|
||||
|
||||
def job_digest(job: dict[str, Any], *, content: str = "") -> str:
|
||||
"""The definition of *job* in 16 hex characters.
|
||||
|
||||
Over kind, source, run_as, schedule, command and enabled -- so enabling a job
|
||||
changes it -- and *content*: what the definition points at that the job's
|
||||
fields do not show, such as a run-parts script's checksum.
|
||||
"""
|
||||
parts = [job.get(field) for field in _DEFINITION] + [content]
|
||||
return hashlib.sha256(json.dumps(parts).encode()).hexdigest()[:16]
|
||||
|
||||
|
||||
def check_job_name(name: str) -> None:
|
||||
""":raises ScheduledJobRefused: for a name a file or a unit could not carry safely."""
|
||||
if not _NAME.fullmatch(name or ""):
|
||||
raise ScheduledJobRefused(f"not a job name: {name!r} (a-z, 0-9 and -, at most 63)")
|
||||
|
||||
|
||||
def check_spec(spec: ScheduledJobSpecDict) -> None:
|
||||
""":raises ScheduledJobRefused: for a spec no host could be given as it is."""
|
||||
check_job_name(spec["name"])
|
||||
if not _USER.fullmatch(spec.get("run_as") or ""):
|
||||
raise ScheduledJobRefused(f"not a user name: {spec.get('run_as')!r}")
|
||||
if not (spec.get("command") or "").strip():
|
||||
raise ScheduledJobRefused("a job needs a command")
|
||||
if "\n" in (spec.get("description") or ""):
|
||||
raise ScheduledJobRefused("a description is one line")
|
||||
if not is_reboot(spec["schedule"]):
|
||||
try:
|
||||
parse_cron(spec["schedule"])
|
||||
except ValueError as exc:
|
||||
raise ScheduledJobRefused(str(exc)) from exc
|
||||
|
||||
|
||||
class ScheduledJobsMixin:
|
||||
"""Declares the scheduled-jobs contract; a driver that can implements it.
|
||||
|
||||
A verb a driver cannot do at all is ``None`` on the driver, as ``manage_service``
|
||||
is; a verb one job cannot take is missing from that job's ``actions``.
|
||||
"""
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def get_scheduled_jobs(self) -> ScheduledJobsReadingDict:
|
||||
"""Every scheduled job of the host.
|
||||
|
||||
:raises ValueError: when the reading came back cut short -- never a
|
||||
shorter list.
|
||||
"""
|
||||
...
|
||||
|
||||
def manage_scheduled_job(
|
||||
self, key: str, action: str, *, digest: str
|
||||
) -> ScheduledJobResultDict:
|
||||
"""Enable, disable or run the job *key* now.
|
||||
|
||||
:raises ScheduledJobChanged: when the job is no longer as *digest* says.
|
||||
:raises ScheduledJobRefused: when the job does not take *action*.
|
||||
:raises LookupError: when the host has no job *key*.
|
||||
"""
|
||||
...
|
||||
|
||||
def put_scheduled_job(
|
||||
self, spec: ScheduledJobSpecDict, *, digest: str | None = None
|
||||
) -> ScheduledJobResultDict:
|
||||
"""Create the job ``spec["name"]`` (*digest* None) or change it.
|
||||
|
||||
:raises ScheduledJobChanged: when the job is no longer as *digest*
|
||||
says, or when a job of that name is there although *digest* is None.
|
||||
:raises ScheduledJobRefused: for a spec :func:`check_spec` refuses, or
|
||||
one the host cannot take.
|
||||
"""
|
||||
...
|
||||
|
||||
def delete_scheduled_job(self, key: str, *, digest: str) -> ScheduledJobResultDict:
|
||||
"""Delete the job *key*, which an application wrote.
|
||||
|
||||
:raises ScheduledJobChanged: when the job is no longer as *digest* says.
|
||||
:raises ScheduledJobRefused: when the job is not one an application wrote.
|
||||
:raises LookupError: when the host has no job *key*.
|
||||
"""
|
||||
...
|
||||
@@ -0,0 +1,493 @@
|
||||
"""Scheduled jobs on a POSIX host, over the command channel: cron and systemd timers.
|
||||
|
||||
:class:`PosixScheduledJobsMixin` implements
|
||||
:class:`~napalm_device_types.scheduled_jobs.ScheduledJobsMixin` for a driver that has
|
||||
:meth:`~napalm_device_types.channel.CommandChannelMixin.run_command`. The driver
|
||||
mixes it in and sets the paths its host keeps jobs in; the defaults are Debian's.
|
||||
|
||||
**Reading** is one framed report as root (:mod:`napalm_device_types._scheduled_reading`).
|
||||
|
||||
**Switching** a cron or anacron line comments it out behind ``#napalm-disabled: ``
|
||||
and takes exactly that marker away again; a run-parts script loses or gets its
|
||||
execute bit; a timer is ``systemctl disable --now`` / ``enable --now``. A timer that
|
||||
cannot be switched (``static``) only runs.
|
||||
|
||||
**Writing** stages the new text as the login user and installs it as root with one
|
||||
fixed command: ``crontab -u <user> <file>`` for a user's crontab, ``install`` with
|
||||
the file's own mode and owner for a system file. Root never reads stdin. Before any
|
||||
write the job is read again and compared with the digest the caller saw.
|
||||
|
||||
**An application's own job** (:meth:`put_scheduled_job`) is a timer on a host with
|
||||
systemd -- ``<name>.timer`` and ``<name>.service`` in :attr:`UNIT_DIR`, the command
|
||||
in :attr:`JOB_SCRIPT_DIR` ``/<name>`` behind a marker and its cron expression --
|
||||
else a file in :attr:`OWN_CRON_D_DIR`, else marked lines in the user's crontab.
|
||||
|
||||
**Running now** is bounded by :data:`~napalm_device_types.scheduled_jobs.RUN_TIMEOUT`:
|
||||
a timer's unit by ``systemctl start`` and its journal since then, a cron line's
|
||||
command as its user the way cron runs it (``_crontab.run_script``).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import fnmatch
|
||||
import shlex
|
||||
from collections.abc import Sequence
|
||||
from typing import TYPE_CHECKING, ClassVar
|
||||
|
||||
from napalm_device_types._crontab import (
|
||||
JOB_MARKER,
|
||||
CronLine,
|
||||
marked_lines,
|
||||
parse_anacrontab,
|
||||
parse_crontab,
|
||||
run_script,
|
||||
switched,
|
||||
with_job,
|
||||
without_job,
|
||||
)
|
||||
from napalm_device_types._scheduled_reading import (
|
||||
OWNERS_SH,
|
||||
SCHEDULE_HEADER,
|
||||
TIME_ZONE_SH,
|
||||
Paths,
|
||||
Reading,
|
||||
parse_reading,
|
||||
reading_command,
|
||||
)
|
||||
from napalm_device_types.cron import is_reboot, to_on_calendar
|
||||
from napalm_device_types.scheduled_jobs import (
|
||||
ACTIONS,
|
||||
RUN_TIMEOUT,
|
||||
ScheduledJobChanged,
|
||||
ScheduledJobRefused,
|
||||
ScheduledJobResultDict,
|
||||
ScheduledJobsMixin,
|
||||
ScheduledJobSpecDict,
|
||||
ScheduledJobsReadingDict,
|
||||
check_spec,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from napalm_device_types.channel import CommandResult
|
||||
|
||||
_STAGE = 't=$(mktemp) && chmod 644 "$t" && cat > "$t" && printf %s "$t"'
|
||||
#: What a run keeps of the job's output.
|
||||
_OUTPUT_LIMIT = 64 * 1024
|
||||
_READ_TIMEOUT = 120
|
||||
|
||||
|
||||
def _command(argv: Sequence[str]) -> str:
|
||||
return shlex.join(argv)
|
||||
|
||||
|
||||
def _said(result: CommandResult) -> str:
|
||||
return "\n".join(p for p in (result.stdout.strip(), result.stderr.strip()) if p)[
|
||||
-_OUTPUT_LIMIT:
|
||||
]
|
||||
|
||||
|
||||
def _result(result: CommandResult, done: str) -> ScheduledJobResultDict:
|
||||
if result.exit_code != 0:
|
||||
return {"success": False, "output": _said(result) or f"exit {result.exit_code}"}
|
||||
return {"success": True, "output": _said(result) or done}
|
||||
|
||||
|
||||
class PosixScheduledJobsMixin(ScheduledJobsMixin):
|
||||
"""Scheduled jobs of a host reached through ``run_command``; the paths are the driver's."""
|
||||
|
||||
CRON_SYSTEM_FILES: ClassVar[tuple[str, ...]] = ("/etc/crontab",)
|
||||
CRON_D_DIRS: ClassVar[tuple[str, ...]] = ("/etc/cron.d",)
|
||||
CRON_SPOOL_DIRS: ClassVar[tuple[str, ...]] = (
|
||||
"/var/spool/cron/crontabs",
|
||||
"/var/spool/cron",
|
||||
"/etc/crontabs",
|
||||
)
|
||||
RUN_PARTS_DIRS: ClassVar[tuple[tuple[str, str], ...]] = (
|
||||
("hourly", "/etc/cron.hourly"),
|
||||
("daily", "/etc/cron.daily"),
|
||||
("weekly", "/etc/cron.weekly"),
|
||||
("monthly", "/etc/cron.monthly"),
|
||||
)
|
||||
ANACRONTAB: ClassVar[str] = "/etc/anacrontab"
|
||||
JOB_SCRIPT_DIR: ClassVar[str] = "/usr/local/libexec/scheduled-jobs"
|
||||
UNIT_DIR: ClassVar[str] = "/etc/systemd/system"
|
||||
#: Where a host without systemd takes an application's job as a file, if the directory
|
||||
#: is there; else, or None, the user's crontab.
|
||||
OWN_CRON_D_DIR: ClassVar[str | None] = "/etc/cron.d"
|
||||
ROOT_GROUP: ClassVar[str] = "root"
|
||||
#: Prints "<octal mode> <owner> <group>" of the path appended.
|
||||
STAT_ARGV: ClassVar[tuple[str, ...]] = ("stat", "-c", "%a %U %G", "--")
|
||||
#: ``install`` and how it makes missing directories (FreeBSD's ``-D`` is a destdir).
|
||||
INSTALL: ClassVar[tuple[str, ...]] = ("install", "-D")
|
||||
#: ``sh`` that reads definition paths on stdin and prints ``<package>: <path>``.
|
||||
OWNERS_SH: ClassVar[str] = OWNERS_SH
|
||||
#: ``sh`` that prints the time zone: an IANA name, or a path ending in ``zoneinfo/<name>``.
|
||||
TIME_ZONE_SH: ClassVar[str] = TIME_ZONE_SH
|
||||
#: ``fnmatch`` patterns on a job's ``source``: what the platform generates is ``platform``.
|
||||
PLATFORM_SOURCES: ClassVar[tuple[str, ...]] = ()
|
||||
#: Run as root after a user's crontab was written; for a host that keeps it elsewhere too.
|
||||
CRONTAB_AFTER_WRITE: ClassVar[tuple[str, ...] | None] = None
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def run_command(
|
||||
self,
|
||||
command: str,
|
||||
*,
|
||||
privileged: bool = False,
|
||||
timeout: float = 60,
|
||||
stdin: bytes | None = None,
|
||||
) -> CommandResult: ...
|
||||
|
||||
# -- reading ---------------------------------------------------------------------------------
|
||||
|
||||
def _sj_paths(self) -> Paths:
|
||||
return Paths(
|
||||
system_files=self.CRON_SYSTEM_FILES,
|
||||
cron_d_dirs=self.CRON_D_DIRS,
|
||||
spool_dirs=self.CRON_SPOOL_DIRS,
|
||||
run_parts=self.RUN_PARTS_DIRS,
|
||||
anacrontab=self.ANACRONTAB,
|
||||
script_dir=self.JOB_SCRIPT_DIR,
|
||||
owners_sh=self.OWNERS_SH,
|
||||
time_zone_sh=self.TIME_ZONE_SH,
|
||||
)
|
||||
|
||||
def _sj_read(self) -> Reading:
|
||||
result = self.run_command(
|
||||
reading_command(self._sj_paths()), privileged=True, timeout=_READ_TIMEOUT
|
||||
)
|
||||
reading = parse_reading(result.stdout)
|
||||
for job in reading.jobs:
|
||||
source = str(job["source"])
|
||||
if job["managed_by"] != "application" and any(
|
||||
fnmatch.fnmatchcase(source, p) for p in self.PLATFORM_SOURCES
|
||||
):
|
||||
job["managed_by"] = "platform"
|
||||
return reading
|
||||
|
||||
def get_scheduled_jobs(self) -> ScheduledJobsReadingDict:
|
||||
"""Every scheduled job of the host, in one round trip.
|
||||
|
||||
:raises ValueError: when the report came back cut short.
|
||||
"""
|
||||
reading = self._sj_read()
|
||||
return {"jobs": reading.jobs, "time_zone": reading.time_zone} # type: ignore[typeddict-item]
|
||||
|
||||
def _sj_job(self, reading: Reading, key: str, digest: str) -> dict[str, object]:
|
||||
job = next((j for j in reading.jobs if j["key"] == key), None)
|
||||
if job is None:
|
||||
raise LookupError(f"no scheduled job {key}")
|
||||
if job["digest"] != digest:
|
||||
raise ScheduledJobChanged(f"{key} changed on the host since it was read")
|
||||
return job
|
||||
|
||||
# -- commands --------------------------------------------------------------------------------
|
||||
|
||||
def _sj_root(self, argv: Sequence[str], timeout: float = 60) -> CommandResult:
|
||||
return self.run_command(_command(argv), privileged=True, timeout=timeout)
|
||||
|
||||
def _sj_checked(self, argv: Sequence[str]) -> CommandResult:
|
||||
result = self._sj_root(argv)
|
||||
if result.exit_code != 0:
|
||||
raise RuntimeError(f"{_command(argv)}: {_said(result) or result.exit_code}")
|
||||
return result
|
||||
|
||||
def _sj_stage(self, content: str) -> str:
|
||||
result = self.run_command(_STAGE, timeout=30, stdin=content.encode())
|
||||
path = result.stdout.strip().splitlines()[-1:] or [""]
|
||||
if result.exit_code != 0 or not path[0].startswith("/"):
|
||||
raise RuntimeError(f"cannot stage a file: {_said(result)}")
|
||||
return path[0]
|
||||
|
||||
def _sj_install(self, content: str, path: str, mode: str, owner: str, group: str) -> None:
|
||||
staged = self._sj_stage(content)
|
||||
try:
|
||||
self._sj_checked(
|
||||
[*self.INSTALL, "-m", mode, "-o", owner, "-g", group, "--", staged, path]
|
||||
)
|
||||
finally:
|
||||
self.run_command(_command(["rm", "-f", "--", staged]), timeout=30)
|
||||
|
||||
# -- crontab files ---------------------------------------------------------------------------
|
||||
|
||||
def _sj_spool_user(self, path: str) -> str | None:
|
||||
"""The user whose crontab *path* is; None for a system file."""
|
||||
directory, _, user = path.rpartition("/")
|
||||
return user if directory in self.CRON_SPOOL_DIRS else None
|
||||
|
||||
def _sj_crontab(self, user: str) -> str:
|
||||
result = self._sj_root(["crontab", "-u", user, "-l"])
|
||||
if result.exit_code != 0:
|
||||
if "no crontab" in (result.stderr + result.stdout).lower():
|
||||
return ""
|
||||
raise RuntimeError(f"cannot read the crontab of {user}: {_said(result)}")
|
||||
return result.stdout
|
||||
|
||||
def _sj_text(self, path: str) -> str:
|
||||
user = self._sj_spool_user(path)
|
||||
if user is not None:
|
||||
return self._sj_crontab(user)
|
||||
return self._sj_checked(["cat", "--", path]).stdout
|
||||
|
||||
def _sj_write_crontab(self, user: str, text: str) -> None:
|
||||
staged = self._sj_stage(text)
|
||||
try:
|
||||
self._sj_checked(["crontab", "-u", user, staged])
|
||||
finally:
|
||||
self.run_command(_command(["rm", "-f", "--", staged]), timeout=30)
|
||||
if self.CRONTAB_AFTER_WRITE:
|
||||
self._sj_checked(self.CRONTAB_AFTER_WRITE)
|
||||
|
||||
def _sj_write(self, path: str, text: str) -> None:
|
||||
user = self._sj_spool_user(path)
|
||||
if user is not None:
|
||||
self._sj_write_crontab(user, text)
|
||||
return
|
||||
# Through a symbolic link to the file it names, with that file's mode: the
|
||||
# link's own mode is 777, and cron ignores a file anyone may write.
|
||||
real = self._sj_checked(["readlink", "-f", "--", path]).stdout.strip() or path
|
||||
mode, owner, group = self._sj_checked([*self.STAT_ARGV, real]).stdout.split()[:3]
|
||||
self._sj_install(text, real, mode, owner, group)
|
||||
|
||||
def _sj_line(self, job: dict[str, object], text: str) -> CronLine:
|
||||
"""The job's line in *text* as it is now, read again for the write."""
|
||||
path = str(job["source"])
|
||||
if job["kind"] == "anacron":
|
||||
found = parse_anacrontab(text)
|
||||
else:
|
||||
user = self._sj_spool_user(path)
|
||||
found = parse_crontab(path, text, system=user is None, user=user or "")
|
||||
line = next((ln for ln in found if ln.key == job["key"]), None)
|
||||
if line is None:
|
||||
raise ScheduledJobChanged(f"{job['key']} is no longer in {path}")
|
||||
return line
|
||||
|
||||
# -- switching and running -------------------------------------------------------------------
|
||||
|
||||
def _sj_timer_unit(self, job: dict[str, object]) -> str:
|
||||
key = str(job["key"])
|
||||
return f"{job['name']}.timer" if key.startswith("job:") else key.removeprefix("timer:")
|
||||
|
||||
def _sj_switch(self, job: dict[str, object], enabled: bool) -> ScheduledJobResultDict:
|
||||
word = "enabled" if enabled else "disabled"
|
||||
if job["kind"] == "systemd_timer":
|
||||
verb = "enable" if enabled else "disable"
|
||||
return _result(
|
||||
self._sj_root(["systemctl", verb, "--now", "--", self._sj_timer_unit(job)]), word
|
||||
)
|
||||
if job["kind"] == "run_parts":
|
||||
return _result(
|
||||
self._sj_root(["chmod", "+x" if enabled else "-x", "--", str(job["source"])]), word
|
||||
)
|
||||
path = str(job["source"])
|
||||
text = self._sj_text(path)
|
||||
self._sj_write(path, switched(text, self._sj_line(job, text), enabled=enabled))
|
||||
return {"success": True, "output": f"{job['key']} {word}"}
|
||||
|
||||
def _sj_as(self, user: str, script: str) -> list[str]:
|
||||
"""*script* run by ``sh`` as *user*, from root."""
|
||||
if user == "root":
|
||||
return ["sh", "-c", script]
|
||||
return ["su", "-s", "/bin/sh", user, "-c", script]
|
||||
|
||||
def _sj_run(self, job: dict[str, object], reading: Reading) -> ScheduledJobResultDict:
|
||||
bound = ["timeout", str(RUN_TIMEOUT)]
|
||||
wait = RUN_TIMEOUT + 30
|
||||
if job["kind"] == "systemd_timer":
|
||||
unit = reading.units[str(job["key"])]
|
||||
since = self.run_command("date +%s", timeout=30).stdout.strip()
|
||||
started = self._sj_root([*bound, "systemctl", "start", "--", unit], timeout=wait)
|
||||
journal = self._sj_root(
|
||||
[
|
||||
"journalctl",
|
||||
"-u",
|
||||
unit,
|
||||
"--since",
|
||||
f"@{since}",
|
||||
"--no-pager",
|
||||
"-o",
|
||||
"cat",
|
||||
"-n",
|
||||
"200",
|
||||
]
|
||||
)
|
||||
output = "\n".join(p for p in (_said(started), journal.stdout.strip()) if p)
|
||||
return {"success": started.exit_code == 0, "output": output or f"{unit} ran"}
|
||||
if job["kind"] == "run_parts":
|
||||
return _result(self._sj_root([*bound, str(job["source"])], timeout=wait), "ran")
|
||||
path = str(job["source"])
|
||||
line = self._sj_line(job, self._sj_text(path))
|
||||
script = run_script(line.command, line.env)
|
||||
return _result(
|
||||
self._sj_root([*bound, *self._sj_as(line.run_as, script)], timeout=wait), "ran"
|
||||
)
|
||||
|
||||
def manage_scheduled_job(self, key: str, action: str, *, digest: str) -> ScheduledJobResultDict:
|
||||
"""Enable, disable or run the job *key* now; see :class:`ScheduledJobsMixin`."""
|
||||
if action not in ACTIONS:
|
||||
raise ScheduledJobRefused(f"unknown action {action!r}; allowed: {list(ACTIONS)}")
|
||||
reading = self._sj_read()
|
||||
job = self._sj_job(reading, key, digest)
|
||||
if action not in job["actions"]: # type: ignore[operator]
|
||||
raise ScheduledJobRefused(f"{key} cannot be told to {action} on this host")
|
||||
if action == "run":
|
||||
return self._sj_run(job, reading)
|
||||
return self._sj_switch(job, action == "enable")
|
||||
|
||||
# -- an application's own jobs ---------------------------------------------------------------
|
||||
|
||||
def _sj_own(
|
||||
self, reading: Reading, spec: ScheduledJobSpecDict, digest: str | None
|
||||
) -> dict[str, object] | None:
|
||||
current = next((j for j in reading.jobs if j["key"] == f"job:{spec['name']}"), None)
|
||||
if digest is None:
|
||||
if current is not None:
|
||||
raise ScheduledJobChanged(f"a job {spec['name']} is there already")
|
||||
return None
|
||||
if current is None:
|
||||
raise LookupError(f"no scheduled job job:{spec['name']}")
|
||||
if current["digest"] != digest:
|
||||
raise ScheduledJobChanged(f"job:{spec['name']} changed on the host since it was read")
|
||||
return current
|
||||
|
||||
def _sj_group(self, user: str) -> str:
|
||||
result = self.run_command(_command(["id", "-gn", "--", user]), timeout=30)
|
||||
if result.exit_code != 0:
|
||||
raise ScheduledJobRefused(f"the host has no user {user}")
|
||||
return result.stdout.strip()
|
||||
|
||||
def _sj_calendar(self, schedule: str) -> list[str]:
|
||||
if is_reboot(schedule):
|
||||
return ["OnBootSec=1min"]
|
||||
lines = to_on_calendar(schedule)
|
||||
for line in lines:
|
||||
checked = self.run_command(
|
||||
_command(["systemd-analyze", "calendar", "--", line]), timeout=30
|
||||
)
|
||||
if checked.exit_code != 0:
|
||||
raise ScheduledJobRefused(f"systemd does not take {line!r}: {_said(checked)}")
|
||||
return [f"OnCalendar={line}" for line in lines]
|
||||
|
||||
def _sj_put_timer(self, spec: ScheduledJobSpecDict) -> None:
|
||||
name, description = spec["name"], spec.get("description") or spec["name"]
|
||||
script = f"{self.JOB_SCRIPT_DIR}/{name}"
|
||||
unit_description = description.replace("%", "%%")
|
||||
timer = "\n".join(
|
||||
[
|
||||
"[Unit]",
|
||||
f"Description={unit_description}",
|
||||
"",
|
||||
"[Timer]",
|
||||
*self._sj_calendar(spec["schedule"]),
|
||||
f"Unit={name}.service",
|
||||
"",
|
||||
"[Install]",
|
||||
"WantedBy=timers.target",
|
||||
"",
|
||||
]
|
||||
)
|
||||
service = "\n".join(
|
||||
[
|
||||
"[Unit]",
|
||||
f"Description={unit_description}",
|
||||
"",
|
||||
"[Service]",
|
||||
"Type=oneshot",
|
||||
f"User={spec['run_as']}",
|
||||
f"ExecStart={script}",
|
||||
"",
|
||||
]
|
||||
)
|
||||
body = f"#!/bin/sh\n{JOB_MARKER}{name} {spec.get('description') or ''}".rstrip()
|
||||
body += f"\n{SCHEDULE_HEADER}{spec['schedule']}\n{spec['command'].strip()}\n"
|
||||
self._sj_install(body, script, "750", "root", self._sj_group(spec["run_as"]))
|
||||
self._sj_install(service, f"{self.UNIT_DIR}/{name}.service", "644", "root", self.ROOT_GROUP)
|
||||
self._sj_install(timer, f"{self.UNIT_DIR}/{name}.timer", "644", "root", self.ROOT_GROUP)
|
||||
self._sj_checked(["systemctl", "daemon-reload"])
|
||||
verb = "enable" if spec["enabled"] else "disable"
|
||||
self._sj_checked(["systemctl", verb, "--now", "--", f"{name}.timer"])
|
||||
|
||||
def _sj_cron_text(self, spec: ScheduledJobSpecDict, *, system: bool) -> list[str]:
|
||||
if "\n" in spec["command"].strip():
|
||||
raise ScheduledJobRefused("a cron line holds one line of command")
|
||||
owner = f" {spec['run_as']}" if system else ""
|
||||
line = f"{spec['schedule']}{owner} {spec['command'].strip()}"
|
||||
return marked_lines(
|
||||
spec["name"], spec.get("description") or "", line, enabled=spec["enabled"]
|
||||
)
|
||||
|
||||
def _sj_cron_d(self) -> str | None:
|
||||
""":attr:`OWN_CRON_D_DIR` where the host has it; BusyBox crond reads none."""
|
||||
directory = self.OWN_CRON_D_DIR
|
||||
if directory is None:
|
||||
return None
|
||||
found = self.run_command(_command(["test", "-d", directory]), timeout=30)
|
||||
return directory if found.exit_code == 0 else None
|
||||
|
||||
def _sj_put_cron(self, spec: ScheduledJobSpecDict, current: dict[str, object] | None) -> None:
|
||||
if self._sj_cron_d() is not None:
|
||||
self._sj_group(spec["run_as"])
|
||||
text = "\n".join(self._sj_cron_text(spec, system=True)) + "\n"
|
||||
self._sj_install(
|
||||
text, f"{self.OWN_CRON_D_DIR}/{spec['name']}", "644", "root", self.ROOT_GROUP
|
||||
)
|
||||
return
|
||||
if current is not None and current["run_as"] != spec["run_as"]:
|
||||
self._sj_delete_cron(current)
|
||||
user = spec["run_as"]
|
||||
text = self._sj_crontab(user)
|
||||
jobs = parse_crontab("", text, system=False, user=user)
|
||||
self._sj_write_crontab(
|
||||
user, with_job(text, jobs, spec["name"], self._sj_cron_text(spec, system=False))
|
||||
)
|
||||
|
||||
def put_scheduled_job(
|
||||
self, spec: ScheduledJobSpecDict, *, digest: str | None = None
|
||||
) -> ScheduledJobResultDict:
|
||||
"""Create or change the application's own job; see :class:`ScheduledJobsMixin`."""
|
||||
check_spec(spec)
|
||||
reading = self._sj_read()
|
||||
current = self._sj_own(reading, spec, digest)
|
||||
if reading.systemd:
|
||||
self._sj_put_timer(spec)
|
||||
else:
|
||||
self._sj_put_cron(spec, current)
|
||||
return {
|
||||
"success": True,
|
||||
"output": f"job:{spec['name']} {'changed' if current else 'created'}",
|
||||
}
|
||||
|
||||
def _sj_delete_cron(self, job: dict[str, object]) -> None:
|
||||
path = str(job["source"])
|
||||
if self._sj_spool_user(path) is None:
|
||||
self._sj_checked(["rm", "-f", "--", path])
|
||||
return
|
||||
text = self._sj_text(path)
|
||||
jobs = parse_crontab(path, text, system=False, user=str(job["run_as"]))
|
||||
self._sj_write(path, without_job(text, jobs, str(job["name"])))
|
||||
|
||||
def delete_scheduled_job(self, key: str, *, digest: str) -> ScheduledJobResultDict:
|
||||
"""Delete the application's job *key*; see :class:`ScheduledJobsMixin`."""
|
||||
job = self._sj_job(self._sj_read(), key, digest)
|
||||
if job["managed_by"] != "application":
|
||||
raise ScheduledJobRefused(
|
||||
f"{key} was not written by an application; it is only switched"
|
||||
)
|
||||
name = str(job["name"])
|
||||
if job["kind"] != "systemd_timer":
|
||||
self._sj_delete_cron(job)
|
||||
return {"success": True, "output": f"{key} deleted"}
|
||||
self._sj_root(["systemctl", "disable", "--now", "--", f"{name}.timer"])
|
||||
self._sj_checked(
|
||||
[
|
||||
"rm",
|
||||
"-f",
|
||||
"--",
|
||||
f"{self.UNIT_DIR}/{name}.timer",
|
||||
f"{self.UNIT_DIR}/{name}.service",
|
||||
f"{self.JOB_SCRIPT_DIR}/{name}",
|
||||
]
|
||||
)
|
||||
self._sj_checked(["systemctl", "daemon-reload"])
|
||||
return {"success": True, "output": f"{key} deleted"}
|
||||
@@ -0,0 +1,60 @@
|
||||
"""The shape a driver states its device's SNMP health OIDs in (``SNMP_HEALTH_PROFILE``).
|
||||
|
||||
Apart from ``models.py``, which is over the file size limit; ``models`` still exports them.
|
||||
"""
|
||||
|
||||
from typing import Literal
|
||||
|
||||
from typing_extensions import TypedDict
|
||||
|
||||
|
||||
class SnmpValueSpec(TypedDict, total=False):
|
||||
"""Where an SNMP agent publishes one value, inside a ``SnmpHealthProfile``.
|
||||
|
||||
Either ``oids``, scalars asked in order (the first non-empty answer wins), or
|
||||
``table``, a column walked and folded to one value by ``fold``. ``pattern`` is a
|
||||
regular expression whose first group is the number, for an agent that answers
|
||||
with text.
|
||||
|
||||
``unknown`` says the agent answers this value wrongly, or not at all, where it
|
||||
should: the application reports no value -- neither the standard reading nor a
|
||||
default. It stands alone, without ``oids`` or ``table``.
|
||||
"""
|
||||
|
||||
oids: list[str]
|
||||
table: str
|
||||
fold: Literal["first", "average", "max"]
|
||||
pattern: str
|
||||
unknown: bool
|
||||
|
||||
|
||||
class SnmpMemorySpec(TypedDict, total=False):
|
||||
"""Memory: ``total`` with ``used`` or ``free``, or ``percent`` alone."""
|
||||
|
||||
total: SnmpValueSpec
|
||||
used: SnmpValueSpec
|
||||
free: SnmpValueSpec
|
||||
percent: SnmpValueSpec
|
||||
unit: Literal["KiB", "bytes"] # of total, used and free; KiB, as UCD-MIB counts, by default
|
||||
|
||||
|
||||
class SnmpInterfacesSpec(TypedDict, total=False):
|
||||
"""How the IF-MIB counters of a device are to be read."""
|
||||
|
||||
skip: str # a pattern of interface names whose counters are noise, not signal
|
||||
tx_errors_are_drops: bool # the device counts dropped packets as TX errors (OPNsense)
|
||||
|
||||
|
||||
class SnmpHealthProfile(TypedDict, total=False):
|
||||
"""Where a device's SNMP agent publishes its health values.
|
||||
|
||||
A driver declares it as ``SNMP_HEALTH_PROFILE`` when its device does not answer
|
||||
the standard UCD-MIB and IF-MIB; the application polls and computes. What a
|
||||
profile does not name is read the standard way, and what it marks ``unknown`` is
|
||||
not read at all. Uptime is in TimeTicks, CPU in percent busy.
|
||||
"""
|
||||
|
||||
uptime: SnmpValueSpec
|
||||
cpu: SnmpValueSpec
|
||||
memory: SnmpMemorySpec
|
||||
interfaces: SnmpInterfacesSpec
|
||||
@@ -27,6 +27,11 @@ A caller that passes neither keeps working as before (first use, every time),
|
||||
and a driver that ignores both still connects for a caller that passes them:
|
||||
old and new versions on either side do not break each other.
|
||||
|
||||
A connection sshd turns away before its banner -- OpenSSH 9.8+ does that to a
|
||||
source it has just penalised -- is tried again after a pause
|
||||
(:func:`retry_refused`, :data:`REFUSED_RETRY_WAITS`); :meth:`HostKeyCheck.connect`
|
||||
does so itself, a netmiko driver wraps its ``ConnectHandler`` in it.
|
||||
|
||||
:class:`HostKeyCheck` is the one implementation, for a paramiko ``SSHClient``
|
||||
(:meth:`~HostKeyCheck.connect`) and for netmiko (:meth:`~HostKeyCheck.netmiko`,
|
||||
then :meth:`~HostKeyCheck.observe`). Like :mod:`napalm_device_types.channel` it
|
||||
@@ -40,7 +45,8 @@ import hashlib
|
||||
import logging
|
||||
import os
|
||||
import tempfile
|
||||
from collections.abc import Iterator, Mapping
|
||||
import time
|
||||
from collections.abc import Callable, Iterator, Mapping
|
||||
from contextlib import contextmanager
|
||||
from dataclasses import dataclass
|
||||
from typing import Any
|
||||
@@ -54,6 +60,13 @@ KNOWN_KEYS_ARG = "ssh_host_keys"
|
||||
#: The ``optional_args`` key that carries the callable every seen key goes to.
|
||||
OBSERVER_ARG = "ssh_host_key_observer"
|
||||
|
||||
#: The waits before each new attempt when sshd turns a connection away before its
|
||||
#: banner. OpenSSH 9.8+ does that to every connection from a source it has just
|
||||
#: penalised (``PerSourcePenalties``) -- one whose connections did not
|
||||
#: authenticate, a scanner's probes -- for seconds to minutes. 26 s in all
|
||||
#: outlasts the 16 s penalty seen on a new Ubuntu 26.04 host (NetOrk/netork#1800).
|
||||
REFUSED_RETRY_WAITS = (1.0, 3.0, 7.0, 15.0)
|
||||
|
||||
#: The netmiko arguments this module decides; a caller's own values give way.
|
||||
_NETMIKO_HOST_KEY_ARGS = ("ssh_strict", "system_host_keys", "alt_host_keys", "alt_key_file")
|
||||
|
||||
@@ -185,6 +198,48 @@ def _refused_key(exc: BaseException) -> Any | None:
|
||||
return None
|
||||
|
||||
|
||||
def refused_before_banner(exc: BaseException) -> bool:
|
||||
"""Whether *exc*, or what it wraps, is sshd closing the connection before its banner.
|
||||
|
||||
paramiko says "Error reading SSH protocol banner" (netmiko and drivers wrap
|
||||
it), with the connection's EOF or reset underneath. A failed login, a
|
||||
silent host and a closed port are something else: waiting does not end them.
|
||||
"""
|
||||
seen: set[int] = set()
|
||||
stack: list[BaseException | None] = [exc]
|
||||
while stack:
|
||||
current = stack.pop()
|
||||
if current is None or id(current) in seen:
|
||||
continue
|
||||
seen.add(id(current))
|
||||
if "protocol banner" in str(current):
|
||||
return True
|
||||
stack.extend([current.__cause__, current.__context__])
|
||||
return False
|
||||
|
||||
|
||||
def retry_refused[T](
|
||||
attempt: Callable[[], T],
|
||||
*,
|
||||
waits: tuple[float, ...] = REFUSED_RETRY_WAITS,
|
||||
sleep: Callable[[float], None] | None = None,
|
||||
) -> T:
|
||||
"""*attempt*, again after each of *waits* while sshd turns it away before its banner.
|
||||
|
||||
Any other error, and the last refusal, go to the caller unchanged.
|
||||
"""
|
||||
pause = sleep or time.sleep
|
||||
for wait in waits:
|
||||
try:
|
||||
return attempt()
|
||||
except Exception as exc:
|
||||
if not refused_before_banner(exc):
|
||||
raise
|
||||
logger.info("SSH connection turned away before its banner; again in %s s", wait)
|
||||
pause(wait)
|
||||
return attempt()
|
||||
|
||||
|
||||
class HostKeyCheck:
|
||||
"""How one driver connection verifies its host's key.
|
||||
|
||||
@@ -220,7 +275,7 @@ class HostKeyCheck:
|
||||
"""``client.connect()`` to this host with *kwargs*, verified and reported."""
|
||||
self.prepare(client)
|
||||
with self.refusing():
|
||||
client.connect(hostname=self.host, port=self.port, **kwargs)
|
||||
retry_refused(lambda: client.connect(hostname=self.host, port=self.port, **kwargs))
|
||||
return self.observe(client.get_transport())
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
@@ -15,7 +15,9 @@ accident.
|
||||
``/etc/ssh/sshd_config.d/*.conf`` -- Debian and Ubuntu do, at the top -- the
|
||||
lists go to :data:`DROPIN_PATH`, which sorts first and so wins. sshd_config
|
||||
stays as the distribution shipped it, and removing the file undoes the change.
|
||||
Only without such an ``Include`` is sshd_config itself rewritten, after a copy.
|
||||
Only without such an ``Include`` is sshd_config itself rewritten, after a copy --
|
||||
and not at all on a platform that generates it
|
||||
(:attr:`OpenSshServerMixin.SSHD_CONFIG_GENERATED`).
|
||||
|
||||
**Nobody is locked out, by three checks.**
|
||||
|
||||
@@ -30,10 +32,19 @@ Only without such an ``Include`` is sshd_config itself rewritten, after a copy.
|
||||
3. After the reload, *verify* -- the caller's -- opens a new session. The
|
||||
running one survives a reload, so if the new one cannot get in, this one
|
||||
puts the old state back and reloads again.
|
||||
|
||||
**The managing client is exempt from per-source penalties** where sshd has them
|
||||
(OpenSSH 9.8+, :meth:`OpenSshServerMixin.exempt_from_ssh_penalties`). sshd
|
||||
penalises a source whose connections do not authenticate -- a scanner's probes
|
||||
-- and then drops every connection from it, the managing session's included.
|
||||
The address is the one the host sees (``$SSH_CONNECTION``); it joins the
|
||||
host's own ``PerSourcePenaltyExemptList`` in :data:`PENALTY_DROPIN_PATH`, with
|
||||
the same three checks. An older sshd is not touched: it refuses the directive.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import ipaddress
|
||||
import time
|
||||
from collections.abc import Callable, Iterable, Mapping, Sequence
|
||||
from dataclasses import dataclass, field
|
||||
@@ -57,6 +68,9 @@ MAIN_CONFIG = "/etc/ssh/sshd_config"
|
||||
DROPIN_DIR = "/etc/ssh/sshd_config.d"
|
||||
#: ``00-`` so it sorts before every other drop-in: sshd keeps the first value it reads.
|
||||
DROPIN_PATH = f"{DROPIN_DIR}/00-netork-ssh-hardening.conf"
|
||||
#: Where :meth:`OpenSshServerMixin.exempt_from_ssh_penalties` names the managing
|
||||
#: client; ``00-`` like the hardening drop-in, since sshd keeps the first value.
|
||||
PENALTY_DROPIN_PATH = f"{DROPIN_DIR}/00-netork-penalty-exempt.conf"
|
||||
#: What marks a line this module disabled in sshd_config.
|
||||
DISABLED_PREFIX = "# netork: "
|
||||
|
||||
@@ -65,6 +79,11 @@ _HEADER = (
|
||||
"# taken out of what this server offers. Delete this file to undo it.\n"
|
||||
)
|
||||
|
||||
_PENALTY_HEADER = (
|
||||
"# Written by netOrk: the address netOrk manages this host from, exempt from\n"
|
||||
"# sshd's per-source penalties (PerSourcePenalties). Delete this file to undo it.\n"
|
||||
)
|
||||
|
||||
#: Host key type, as ``ssh-keygen -l`` prints it -> the algorithms that key serves.
|
||||
#: Certificate algorithms are left out: they also need a HostCertificate.
|
||||
_KEY_TYPE_ALGORITHMS: dict[str, tuple[str, ...]] = {
|
||||
@@ -302,6 +321,56 @@ def render_main_config(text: str, keep: Mapping[str, Sequence[str]], kinds: Iter
|
||||
return render_dropin(keep, wanted) + "\n".join(out) + ("\n" if body.endswith("\n") else "")
|
||||
|
||||
|
||||
def parse_penalty_exemptions(text: str) -> list[str] | None:
|
||||
"""The ``persourcepenaltyexemptlist`` from ``sshd -T`` output; None without penalties.
|
||||
|
||||
Per-source penalties came with OpenSSH 9.8. An older sshd prints neither
|
||||
line and refuses ``PerSourcePenaltyExemptList`` outright -- it does not
|
||||
start with it in its configuration -- so None means: write nothing.
|
||||
"""
|
||||
penalties = False
|
||||
exempt: list[str] = []
|
||||
for raw in text.splitlines():
|
||||
parts = raw.strip().split(None, 1)
|
||||
if not parts:
|
||||
continue
|
||||
word = parts[0].lower()
|
||||
if word == "persourcepenalties":
|
||||
penalties = True
|
||||
elif word == "persourcepenaltyexemptlist" and len(parts) == 2:
|
||||
value = parts[1].strip()
|
||||
exempt = [] if value.lower() == "none" else [a for a in value.split(",") if a]
|
||||
return exempt if penalties else None
|
||||
|
||||
|
||||
def client_address(ssh_connection: str) -> str | None:
|
||||
"""The client's address in ``$SSH_CONNECTION`` (``client port server port``), or None."""
|
||||
parts = ssh_connection.split()
|
||||
if len(parts) != 4:
|
||||
return None
|
||||
try:
|
||||
return str(ipaddress.ip_address(parts[0]))
|
||||
except ValueError:
|
||||
return None
|
||||
|
||||
|
||||
def is_exempt(address: str, exempt: Iterable[str]) -> bool:
|
||||
"""Whether *address* is one of *exempt*'s addresses or inside one of its networks."""
|
||||
ip = ipaddress.ip_address(address)
|
||||
for entry in exempt:
|
||||
try:
|
||||
if ip in ipaddress.ip_network(entry, strict=False):
|
||||
return True
|
||||
except ValueError:
|
||||
continue
|
||||
return False
|
||||
|
||||
|
||||
def render_penalty_dropin(exempt: Sequence[str]) -> str:
|
||||
"""The drop-in stating the full exempt list."""
|
||||
return _PENALTY_HEADER + f"PerSourcePenaltyExemptList {','.join(exempt)}\n"
|
||||
|
||||
|
||||
@dataclass
|
||||
class SshHardeningResult:
|
||||
"""What :meth:`OpenSshServerMixin.harden_ssh_server` did.
|
||||
@@ -319,6 +388,38 @@ class SshHardeningResult:
|
||||
message: str = ""
|
||||
|
||||
|
||||
@dataclass
|
||||
class SshPenaltyExemption:
|
||||
"""What :meth:`OpenSshServerMixin.exempt_from_ssh_penalties` did.
|
||||
|
||||
*address* is the one the host sees the managing client at. *applied* is
|
||||
True only when sshd uses the exemption and a new session got in;
|
||||
*rolled_back* says the host was changed and then put back.
|
||||
"""
|
||||
|
||||
address: str | None = None
|
||||
applied: bool = False
|
||||
rolled_back: bool = False
|
||||
path: str | None = None
|
||||
message: str = ""
|
||||
|
||||
|
||||
#: Why an algorithm stays, by :class:`Refusal` reason.
|
||||
_WHYS = {
|
||||
"last": "it is the last one",
|
||||
"client": "netOrk could no longer connect",
|
||||
"host_keys_unknown": "the host keys could not be read",
|
||||
}
|
||||
|
||||
|
||||
def _unchanged(
|
||||
result: SshPenaltyExemption, message: str, say: Callable[[str], None]
|
||||
) -> SshPenaltyExemption:
|
||||
result.message = message
|
||||
say(message)
|
||||
return result
|
||||
|
||||
|
||||
class OpenSshServerMixin:
|
||||
"""Adds :meth:`harden_ssh_server` to a driver that can run commands as root on an OpenSSH host.
|
||||
|
||||
@@ -327,9 +428,16 @@ class OpenSshServerMixin:
|
||||
whose platform writes sshd_config itself (an appliance that regenerates it
|
||||
from its own database) must not inherit this, or must opt out with
|
||||
``harden_ssh_server = None``: its file would be overwritten on the next
|
||||
apply.
|
||||
apply. Where that generated file still includes the drop-in directory, the
|
||||
driver sets :attr:`SSHD_CONFIG_GENERATED` instead and keeps the hardening.
|
||||
"""
|
||||
|
||||
#: True where the platform writes sshd_config from its own database
|
||||
#: (OpenMediaVault): only the drop-in directory is written, and a
|
||||
#: sshd_config that does not include it is left alone, since the platform
|
||||
#: would write over a change to it on its next apply.
|
||||
SSHD_CONFIG_GENERATED: bool = False
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def run_command(
|
||||
@@ -453,24 +561,7 @@ class OpenSshServerMixin:
|
||||
:raises RuntimeError: if the current state cannot be read.
|
||||
"""
|
||||
say = log or (lambda _line: None)
|
||||
say("Reading the algorithms sshd offers (sshd -T)")
|
||||
sshd_t = self._sshd_effective_text()
|
||||
host_keys: set[str] | None = None
|
||||
if remove.get("hostkey"):
|
||||
say("Reading which host keys the server has (ssh-keygen -l)")
|
||||
host_keys = self._ssh_host_key_algorithms(sshd_t)
|
||||
plan = plan_ssh_hardening(
|
||||
parse_sshd_effective(sshd_t), remove, client=client, host_keys=host_keys
|
||||
)
|
||||
whys = {
|
||||
"last": "it is the last one",
|
||||
"client": "netOrk could no longer connect",
|
||||
"host_keys_unknown": "the host keys could not be read",
|
||||
}
|
||||
for r in plan.refused:
|
||||
last_key = r.reason == "last" and r.kind == "hostkey"
|
||||
why = "it is the last one the host has a key for" if last_key else whys[r.reason]
|
||||
say(f"Keeping {r.algorithm}: {why}")
|
||||
plan = self._plan_hardening(remove, client, say)
|
||||
kinds = plan.changed_classes
|
||||
result = SshHardeningResult(plan=plan)
|
||||
if not kinds:
|
||||
@@ -483,8 +574,57 @@ class OpenSshServerMixin:
|
||||
main = self._ssh_read(MAIN_CONFIG)
|
||||
if main is None:
|
||||
raise RuntimeError(f"cannot read {MAIN_CONFIG}")
|
||||
if self.SSHD_CONFIG_GENERATED and not has_dropin_include(main):
|
||||
result.message = (
|
||||
f"Not changed: {MAIN_CONFIG} does not include {DROPIN_DIR}, and this"
|
||||
" platform writes it from its own settings."
|
||||
)
|
||||
say(result.message)
|
||||
return result
|
||||
restore = self._write_change(plan, main, result, say)
|
||||
return self._apply_or_restore(plan, kinds, result, restore, verify, say)
|
||||
return self._apply_or_restore(
|
||||
result,
|
||||
restore,
|
||||
verify,
|
||||
say,
|
||||
ignored=lambda: self._algorithms_ignored(plan, result.path),
|
||||
done="The server no longer offers the selected algorithms.",
|
||||
)
|
||||
|
||||
def _plan_hardening(
|
||||
self,
|
||||
remove: Mapping[str, Iterable[str]],
|
||||
client: Mapping[str, Iterable[str]] | None,
|
||||
say: Callable[[str], None],
|
||||
) -> SshHardeningPlan:
|
||||
"""Read what sshd offers and which host keys it has; plan, and say what stays."""
|
||||
say("Reading the algorithms sshd offers (sshd -T)")
|
||||
sshd_t = self._sshd_effective_text()
|
||||
host_keys: set[str] | None = None
|
||||
if remove.get("hostkey"):
|
||||
say("Reading which host keys the server has (ssh-keygen -l)")
|
||||
host_keys = self._ssh_host_key_algorithms(sshd_t)
|
||||
plan = plan_ssh_hardening(
|
||||
parse_sshd_effective(sshd_t), remove, client=client, host_keys=host_keys
|
||||
)
|
||||
for r in plan.refused:
|
||||
last_key = r.reason == "last" and r.kind == "hostkey"
|
||||
why = "it is the last one the host has a key for" if last_key else _WHYS[r.reason]
|
||||
say(f"Keeping {r.algorithm}: {why}")
|
||||
return plan
|
||||
|
||||
def _algorithms_ignored(self, plan: SshHardeningPlan, path: str | None) -> str | None:
|
||||
"""Why sshd does not use the new lists; None when it does."""
|
||||
now = self.get_ssh_server_algorithms()
|
||||
names = [
|
||||
SSH_ALGORITHM_CLASSES[k][1] for k in plan.changed_classes if now[k] != plan.keep[k]
|
||||
]
|
||||
if not names:
|
||||
return None
|
||||
return (
|
||||
f"sshd does not use the new {', '.join(names)}: a setting read before"
|
||||
f" {path} decides them"
|
||||
)
|
||||
|
||||
def _write_change(
|
||||
self,
|
||||
@@ -520,43 +660,33 @@ class OpenSshServerMixin:
|
||||
|
||||
return restore_main
|
||||
|
||||
def _apply_or_restore(
|
||||
def _apply_or_restore[R: (SshHardeningResult, SshPenaltyExemption)](
|
||||
self,
|
||||
plan: SshHardeningPlan,
|
||||
kinds: list[str],
|
||||
result: SshHardeningResult,
|
||||
result: R,
|
||||
restore: Callable[[], None],
|
||||
verify: Callable[[], bool] | None,
|
||||
say: Callable[[str], None],
|
||||
) -> SshHardeningResult:
|
||||
def back(message: str, reload: bool) -> SshHardeningResult:
|
||||
say(message)
|
||||
say("Putting the previous configuration back")
|
||||
try:
|
||||
restore()
|
||||
if reload:
|
||||
self._reload_ssh_server()
|
||||
except Exception as exc: # noqa: BLE001 - report it, the caller must look
|
||||
result.message = f"{message}; putting the previous configuration back failed: {exc}"
|
||||
say(result.message)
|
||||
return result
|
||||
result.rolled_back = True
|
||||
result.message = message
|
||||
return result
|
||||
*,
|
||||
ignored: Callable[[], str | None],
|
||||
done: str,
|
||||
) -> R:
|
||||
"""Check what was written, reload, verify; put the old state back on any failure.
|
||||
|
||||
*ignored* reads ``sshd -T`` after ``sshd -t`` and says why sshd does not
|
||||
use the change, or None when it does; *done* is the message of success.
|
||||
"""
|
||||
|
||||
def back(message: str, reload: bool) -> R:
|
||||
return self._put_back(result, restore, say, message, reload=reload)
|
||||
|
||||
try:
|
||||
say("Checking the configuration (sshd -t)")
|
||||
complaint = self._sshd_accepts()
|
||||
if complaint:
|
||||
return back(f"sshd rejected the new configuration: {complaint}", reload=False)
|
||||
now = self.get_ssh_server_algorithms()
|
||||
ignored = [SSH_ALGORITHM_CLASSES[k][1] for k in kinds if now[k] != plan.keep[k]]
|
||||
if ignored:
|
||||
return back(
|
||||
f"sshd does not use the new {', '.join(ignored)}: a setting read before"
|
||||
f" {result.path} decides them",
|
||||
reload=False,
|
||||
)
|
||||
why = ignored()
|
||||
if why:
|
||||
return back(why, reload=False)
|
||||
say("Reloading sshd")
|
||||
self._reload_ssh_server()
|
||||
if verify is not None:
|
||||
@@ -566,6 +696,125 @@ class OpenSshServerMixin:
|
||||
except Exception as exc: # noqa: BLE001 - whatever went wrong, the host goes back
|
||||
return back(f"The change failed: {exc}", reload=True)
|
||||
result.applied = True
|
||||
result.message = "The server no longer offers the selected algorithms."
|
||||
result.message = done
|
||||
say(result.message)
|
||||
return result
|
||||
|
||||
def _put_back[R: (SshHardeningResult, SshPenaltyExemption)](
|
||||
self,
|
||||
result: R,
|
||||
restore: Callable[[], None],
|
||||
say: Callable[[str], None],
|
||||
message: str,
|
||||
*,
|
||||
reload: bool,
|
||||
) -> R:
|
||||
"""Say *message*, put the previous state back, and say whether that worked."""
|
||||
say(message)
|
||||
say("Putting the previous configuration back")
|
||||
try:
|
||||
restore()
|
||||
if reload:
|
||||
self._reload_ssh_server()
|
||||
except Exception as exc: # noqa: BLE001 - report it, the caller must look
|
||||
result.message = f"{message}; putting the previous configuration back failed: {exc}"
|
||||
say(result.message)
|
||||
return result
|
||||
result.rolled_back = True
|
||||
result.message = message
|
||||
return result
|
||||
|
||||
# ── the penalty exemption ───────────────────────────────────────────
|
||||
|
||||
def exempt_from_ssh_penalties(
|
||||
self,
|
||||
*,
|
||||
verify: Callable[[], bool] | None = None,
|
||||
log: Callable[[str], None] | None = None,
|
||||
) -> SshPenaltyExemption:
|
||||
"""Exempt the address this session comes from from sshd's per-source penalties.
|
||||
|
||||
OpenSSH 9.8+ penalises a source whose connections do not authenticate --
|
||||
a scanner's probes -- and then drops every connection from it, the
|
||||
managing client's included (NetOrk/netork#1800). The address is the
|
||||
one the host sees (``$SSH_CONNECTION``), so NAT and relays do not
|
||||
matter. The host's own exemptions stay: the new list is the one sshd
|
||||
uses now plus this address.
|
||||
|
||||
Only into the drop-in directory: a sshd_config that does not include it
|
||||
is left alone, and the result says so. An sshd before 9.8 is not
|
||||
touched -- it would not start with the directive. Checked like
|
||||
:meth:`harden_ssh_server`: ``sshd -t``, ``sshd -T`` shows the address,
|
||||
reload, then *verify*; on any failure the previous state goes back.
|
||||
|
||||
:raises RuntimeError: if the session names no client address, or
|
||||
``sshd -T`` or sshd_config cannot be read.
|
||||
"""
|
||||
say = log or (lambda _line: None)
|
||||
address = self._session_client_address()
|
||||
result = SshPenaltyExemption(address=address)
|
||||
say(f"netOrk connects from {address}")
|
||||
say("Reading sshd's per-source penalties (sshd -T)")
|
||||
exempt = parse_penalty_exemptions(self._sshd_effective_text())
|
||||
reason = self._penalty_exemption_unchanged(address, exempt)
|
||||
if reason is not None or exempt is None:
|
||||
return _unchanged(result, reason or "", say)
|
||||
|
||||
restore = self._write_penalty_dropin([*exempt, address])
|
||||
result.path = PENALTY_DROPIN_PATH
|
||||
say(f"Wrote {PENALTY_DROPIN_PATH}")
|
||||
return self._apply_or_restore(
|
||||
result,
|
||||
restore,
|
||||
verify,
|
||||
say,
|
||||
ignored=lambda: self._exemption_ignored(address),
|
||||
done=f"{address} is exempt from sshd's per-source penalties.",
|
||||
)
|
||||
|
||||
def _session_client_address(self) -> str:
|
||||
"""The address the host sees this session come from (``$SSH_CONNECTION``)."""
|
||||
session = self.run_command('printf %s "$SSH_CONNECTION"', privileged=False, timeout=30)
|
||||
address = client_address(session.stdout)
|
||||
if address is None:
|
||||
raise RuntimeError("the session names no client address ($SSH_CONNECTION)")
|
||||
return address
|
||||
|
||||
def _penalty_exemption_unchanged(self, address: str, exempt: list[str] | None) -> str | None:
|
||||
"""Why the exemption is left as it is; None when it is to be written."""
|
||||
if exempt is None:
|
||||
return "Nothing to change: this sshd has no per-source penalties (before OpenSSH 9.8)."
|
||||
if is_exempt(address, exempt):
|
||||
return f"Nothing to change: {address} is already exempt."
|
||||
main = self._ssh_read(MAIN_CONFIG)
|
||||
if main is None:
|
||||
raise RuntimeError(f"cannot read {MAIN_CONFIG}")
|
||||
if not has_dropin_include(main):
|
||||
return (
|
||||
f"Not changed: {MAIN_CONFIG} does not include {DROPIN_DIR},"
|
||||
" where the exemption would go."
|
||||
)
|
||||
return None
|
||||
|
||||
def _exemption_ignored(self, address: str) -> str | None:
|
||||
"""Why sshd does not use the new exemption; None when it does."""
|
||||
if is_exempt(address, parse_penalty_exemptions(self._sshd_effective_text()) or []):
|
||||
return None
|
||||
return (
|
||||
"sshd does not use the new PerSourcePenaltyExemptList: a setting read before"
|
||||
f" {PENALTY_DROPIN_PATH} decides it"
|
||||
)
|
||||
|
||||
def _write_penalty_dropin(self, exempt: Sequence[str]) -> Callable[[], None]:
|
||||
"""Write the exemption list; return what puts the previous drop-in back."""
|
||||
previous = self._ssh_read(PENALTY_DROPIN_PATH)
|
||||
self._ssh_run(f"mkdir -p -- {quote(DROPIN_DIR)}")
|
||||
self._ssh_write(PENALTY_DROPIN_PATH, render_penalty_dropin(exempt))
|
||||
|
||||
def restore() -> None:
|
||||
if previous is None:
|
||||
self._ssh_remove(PENALTY_DROPIN_PATH)
|
||||
else:
|
||||
self._ssh_write(PENALTY_DROPIN_PATH, previous)
|
||||
|
||||
return restore
|
||||
@@ -0,0 +1,312 @@
|
||||
"""The sudo password an earlier driver left in shell histories, and taking it back.
|
||||
|
||||
A driver that typed ``echo '<password>' | sudo -S -p "" <command>`` into an
|
||||
interactive shell left that line, password and all, in the login user's shell
|
||||
history, once per command -- and where only the part before the first ``&&``
|
||||
ran as root, it left files root should own to the login user. A driver that has
|
||||
stopped doing so still leaves both behind on every host it ever reached.
|
||||
|
||||
:class:`SudoHistoryMixin` reads the first as a fact and takes both back. The
|
||||
commands are the same on every POSIX host, so they live here once; a driver
|
||||
mixes it in, supplies ``run_command`` and says what it once left behind:
|
||||
:attr:`~SudoHistoryMixin.SUDO_HISTORY_MARKER` and
|
||||
:attr:`~SudoHistoryMixin.SUDO_ROOT_FILES`.
|
||||
|
||||
**The password never travels.** Lines are counted with ``grep -c`` and deleted
|
||||
with ``sed``, both matching the marker only, as the login user who owns the
|
||||
history. Nothing of a line's content is read back, and nothing goes to stdin.
|
||||
|
||||
**What is deleted is gone from the disk too, as far as a file can say.** ``sed
|
||||
-i`` with a backup suffix renames the old file to the backup and writes a new
|
||||
one; the backup -- the old file's blocks -- is overwritten with ``shred -u``,
|
||||
or removed with ``rm`` where there is no ``shred``. Backups, snapshots and an
|
||||
audit log of the host may still hold the password: only rotating it ends that.
|
||||
|
||||
**A file given back to root** must exist as a regular file, not be a symbolic
|
||||
link, and be owned by the login user. ``chown -h`` never follows a link that
|
||||
appears in between.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable
|
||||
from shlex import quote
|
||||
from typing import TYPE_CHECKING, ClassVar
|
||||
|
||||
from typing_extensions import TypedDict
|
||||
|
||||
from napalm_device_types.fixes import run_fix_steps
|
||||
from napalm_device_types.models import FixResultDict, FixStepDict, FixStepOutcomeDict
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - for type checkers only
|
||||
from napalm_device_types.channel import CommandResult
|
||||
|
||||
#: The shell history files looked at, in the login user's home.
|
||||
HISTORY_FILES = (".bash_history", ".zsh_history", ".ash_history", ".sh_history")
|
||||
|
||||
#: The suffix of the moment's backup ``sed -i`` makes; it carries no application's name.
|
||||
HISTORY_BACKUP_SUFFIX = ".napalm-bak"
|
||||
|
||||
_COUNT = 'LC_ALL=C grep -c -F -e "$m" -- "$p"'
|
||||
|
||||
|
||||
class HistorySudoLinesDict(TypedDict):
|
||||
"""A history file holding lines with the sudo password (``get_history_sudo_lines``)."""
|
||||
|
||||
#: The file, as ``~/<name>``.
|
||||
file: str
|
||||
lines: int
|
||||
|
||||
|
||||
class HistoryCleanupDict(TypedDict):
|
||||
"""What one history file held before the cleanup and holds after it."""
|
||||
|
||||
file: str
|
||||
before: int
|
||||
after: int
|
||||
|
||||
|
||||
class SudoHistoryCleanupDict(FixResultDict):
|
||||
"""The result of ``remove_history_sudo_lines``: the fix, and what it changed."""
|
||||
|
||||
#: Every history file there is, with its lines before and after.
|
||||
files: list[HistoryCleanupDict]
|
||||
#: How many lines were deleted, over all files.
|
||||
removed: int
|
||||
#: The files given back to root.
|
||||
owners_restored: list[str]
|
||||
|
||||
|
||||
def _bre(text: str) -> str:
|
||||
"""*text* as a POSIX basic regular expression matching it literally, for ``sed``."""
|
||||
return "".join("\\" + c if c in "\\.[]*^$/" else c for c in text)
|
||||
|
||||
|
||||
def _history_loop(body: str) -> str:
|
||||
names = " ".join(HISTORY_FILES)
|
||||
return f'for f in {names}; do p="$HOME/$f"; [ -f "$p" ] || continue; {body}; done'
|
||||
|
||||
|
||||
def count_script(marker: str) -> str:
|
||||
"""Print ``<count> <name>`` per history file there is; exit 2 when one cannot be read."""
|
||||
return f"m={quote(marker)}; " + _history_loop(
|
||||
f'n=$({_COUNT}); [ $? -le 1 ] || exit 2; printf "%s %s\\n" "$n" "$f"'
|
||||
)
|
||||
|
||||
|
||||
def cleanup_script(marker: str) -> str:
|
||||
"""Delete the lines holding *marker* from every history file there is.
|
||||
|
||||
Prints per file: ``done <before> <after> <name>``, ``link <count> <name>``
|
||||
(left alone), ``unreadable <name>`` or ``failed <before> <name>``.
|
||||
"""
|
||||
backup = f'"$p"{quote(HISTORY_BACKUP_SUFFIX)}'
|
||||
return f"m={quote(marker)}; r={quote(_bre(marker))}; " + _history_loop(
|
||||
f"b=$({_COUNT}); "
|
||||
'[ $? -le 1 ] || { printf "unreadable %s\\n" "$f"; continue; }; '
|
||||
'if [ -L "$p" ]; then printf "link %s %s\\n" "$b" "$f"; continue; fi; '
|
||||
'if [ "$b" -gt 0 ]; then '
|
||||
f'LC_ALL=C sed -i{quote(HISTORY_BACKUP_SUFFIX)} -e "/$r/d" "$p"; s=$?; '
|
||||
# A sed that renamed the file away and wrote no new one: the backup is the history.
|
||||
f'[ -e "$p" ] || mv -f {backup} "$p"; '
|
||||
f"[ -e {backup} ] && {{ shred -u {backup} 2>/dev/null || rm -f {backup}; }}; "
|
||||
'[ $s -eq 0 ] || { printf "failed %s %s\\n" "$b" "$f"; continue; }; '
|
||||
"fi; "
|
||||
f'a=$({_COUNT}); printf "done %s %s %s\\n" "$b" "$a" "$f"'
|
||||
)
|
||||
|
||||
|
||||
def owners_script(user: str, paths: tuple[str, ...]) -> str:
|
||||
"""As root: give each of *paths* back to root:root where *user* owns it.
|
||||
|
||||
Prints ``fixed <path>`` or ``failed <path>`` per file it changed or tried to.
|
||||
"""
|
||||
return (
|
||||
f"u={quote(user)}; for p in {' '.join(quote(p) for p in paths)}; do "
|
||||
'[ -f "$p" ] && [ ! -L "$p" ] || continue; '
|
||||
'[ -n "$(find "$p" -prune -user "$u" 2>/dev/null)" ] || continue; '
|
||||
'if chown -h 0:0 "$p"; then printf "fixed %s\\n" "$p"; '
|
||||
'else printf "failed %s\\n" "$p"; fi; done'
|
||||
)
|
||||
|
||||
|
||||
def _home(name: str) -> str:
|
||||
return f"~/{name}"
|
||||
|
||||
|
||||
def _lines(n: int) -> str:
|
||||
return "1 line" if n == 1 else f"{n} lines"
|
||||
|
||||
|
||||
def _done(rest: list[str]) -> tuple[bool, str, HistoryCleanupDict | None]:
|
||||
before, after, name = int(rest[0]), int(rest[1]), _home(rest[2])
|
||||
removed = before - after
|
||||
text = (
|
||||
f"{name}: {_lines(removed)} removed ({before} before, {after} after)"
|
||||
if before
|
||||
else f"{name}: no such line"
|
||||
)
|
||||
return after == 0, text, {"file": name, "before": before, "after": after}
|
||||
|
||||
|
||||
def _link(rest: list[str]) -> tuple[bool, str, HistoryCleanupDict | None]:
|
||||
count = int(rest[0])
|
||||
return count == 0, f"{_home(rest[1])}: a symbolic link, left alone ({_lines(count)})", None
|
||||
|
||||
|
||||
def _unreadable(rest: list[str]) -> tuple[bool, str, HistoryCleanupDict | None]:
|
||||
return False, f"{_home(rest[0])}: cannot be read", None
|
||||
|
||||
|
||||
def _failed(rest: list[str]) -> tuple[bool, str, HistoryCleanupDict | None]:
|
||||
kept = _lines(int(rest[0]))
|
||||
return False, f"{_home(rest[1])}: could not be rewritten ({kept} kept)", None
|
||||
|
||||
|
||||
#: Per kind of line :func:`cleanup_script` prints: how many words follow, and what it says.
|
||||
_CLEANUP_LINES: dict[
|
||||
str, tuple[int, Callable[[list[str]], tuple[bool, str, HistoryCleanupDict | None]]]
|
||||
] = {
|
||||
"done": (3, _done),
|
||||
"link": (2, _link),
|
||||
"unreadable": (1, _unreadable),
|
||||
"failed": (2, _failed),
|
||||
}
|
||||
|
||||
|
||||
class SudoHistoryMixin:
|
||||
"""Reads and removes the sudo password an earlier driver left in shell histories.
|
||||
|
||||
Mixed in by the drivers that once typed it into a shell, not by a base
|
||||
class, so ``callable(getattr(driver, "get_history_sudo_lines", None))``
|
||||
stays a truthful answer.
|
||||
"""
|
||||
|
||||
#: What every line carries that typed the sudo password into the shell, as a
|
||||
#: fixed string -- ``"| sudo -S -p "`` for ``echo '<password>' | sudo -S -p ""
|
||||
#: <command>``. None for a driver that never did: nothing is looked for.
|
||||
SUDO_HISTORY_MARKER: ClassVar[str | None] = None
|
||||
|
||||
#: Files an earlier version wrote as the login user where root should have.
|
||||
SUDO_ROOT_FILES: ClassVar[tuple[str, ...]] = ()
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def run_command(
|
||||
self,
|
||||
command: str,
|
||||
*,
|
||||
privileged: bool = False,
|
||||
timeout: float = 60,
|
||||
stdin: bytes | None = None,
|
||||
) -> CommandResult: ...
|
||||
|
||||
def get_history_sudo_lines(self) -> list[HistorySudoLinesDict]:
|
||||
"""The login user's history files that hold lines with the sudo password, and how many.
|
||||
|
||||
A fact: counted with ``grep -c`` as the login user; no line is read.
|
||||
An empty list when no file holds one, or the driver never typed it.
|
||||
|
||||
:raises RuntimeError: when a history file cannot be read.
|
||||
"""
|
||||
marker = self.SUDO_HISTORY_MARKER
|
||||
if not marker:
|
||||
return []
|
||||
result = self.run_command(count_script(marker), timeout=30)
|
||||
if result.exit_code != 0:
|
||||
raise RuntimeError(
|
||||
f"cannot count the shell history: {(result.stderr or result.stdout).strip()}"
|
||||
)
|
||||
found: list[HistorySudoLinesDict] = []
|
||||
for line in result.stdout.splitlines():
|
||||
count, _, name = line.strip().partition(" ")
|
||||
if count.isdigit() and int(count) and name:
|
||||
found.append({"file": _home(name), "lines": int(count)})
|
||||
return found
|
||||
|
||||
def remove_history_sudo_lines(self) -> SudoHistoryCleanupDict:
|
||||
"""Delete those lines, as the login user, and give :attr:`SUDO_ROOT_FILES` back to root.
|
||||
|
||||
Two steps, ``history`` and ``owners``; the second runs when the first
|
||||
failed, too. Succeeds when no history file holds such a line any more
|
||||
and every file the login user owned is root's again.
|
||||
"""
|
||||
files: list[HistoryCleanupDict] = []
|
||||
restored: list[str] = []
|
||||
parts = [
|
||||
run_fix_steps([("history", lambda: self._clean_histories(files))]),
|
||||
run_fix_steps([("owners", lambda: self._restore_owners(restored))]),
|
||||
]
|
||||
steps: list[FixStepDict] = [s for part in parts for s in part["steps"]]
|
||||
return {
|
||||
"success": all(part["success"] for part in parts),
|
||||
"steps": steps,
|
||||
"output": "\n".join(f"[{s['step']}] {s['output']}".rstrip() for s in steps),
|
||||
"files": files,
|
||||
"removed": sum(f["before"] - f["after"] for f in files),
|
||||
"owners_restored": restored,
|
||||
}
|
||||
|
||||
def _clean_histories(self, files: list[HistoryCleanupDict]) -> FixStepOutcomeDict:
|
||||
marker = self.SUDO_HISTORY_MARKER
|
||||
if not marker:
|
||||
return {"ok": True, "output": "this driver never typed the sudo password"}
|
||||
result = self.run_command(cleanup_script(marker), timeout=60)
|
||||
said: list[str] = []
|
||||
ok = result.exit_code == 0
|
||||
for line in result.stdout.splitlines():
|
||||
ok = self._history_line(line.split(), files, said) and ok
|
||||
if result.exit_code != 0:
|
||||
said.append((result.stderr or result.stdout).strip())
|
||||
return {"ok": ok, "output": "\n".join(said) or "no shell history"}
|
||||
|
||||
@staticmethod
|
||||
def _history_line(words: list[str], files: list[HistoryCleanupDict], said: list[str]) -> bool:
|
||||
"""Take one line of :func:`cleanup_script` into *files* and *said*; whether it is clean."""
|
||||
kind, rest = (words[0], words[1:]) if words else ("", [])
|
||||
read = _CLEANUP_LINES.get(kind)
|
||||
if read is None or len(rest) != read[0]:
|
||||
return True
|
||||
clean, text, entry = read[1](rest)
|
||||
said.append(text)
|
||||
if entry is not None:
|
||||
files.append(entry)
|
||||
return clean
|
||||
|
||||
def _restore_owners(self, restored: list[str]) -> FixStepOutcomeDict:
|
||||
paths = tuple(self.SUDO_ROOT_FILES)
|
||||
if not paths:
|
||||
return {"ok": True, "output": "no file to check"}
|
||||
who = self.run_command("id -u; id -un", timeout=30).stdout.split()
|
||||
if len(who) < 2:
|
||||
return {"ok": False, "output": "cannot tell who the login user is"}
|
||||
uid, user = who[0], who[1]
|
||||
if uid == "0":
|
||||
return {"ok": True, "output": "logged in as root: every file it wrote is root's"}
|
||||
result = self.run_command(owners_script(user, paths), privileged=True, timeout=30)
|
||||
said: list[str] = []
|
||||
ok = result.exit_code == 0
|
||||
for line in result.stdout.splitlines():
|
||||
kind, _, path = line.strip().partition(" ")
|
||||
if kind == "fixed":
|
||||
restored.append(path)
|
||||
said.append(f"{path}: owned by root again")
|
||||
elif kind == "failed":
|
||||
ok = False
|
||||
said.append(f"{path}: could not be given back to root")
|
||||
if result.exit_code != 0:
|
||||
said.append((result.stderr or result.stdout).strip())
|
||||
return {"ok": ok, "output": "\n".join(said) or f"no file owned by {user}"}
|
||||
|
||||
|
||||
__all__ = [
|
||||
"HISTORY_BACKUP_SUFFIX",
|
||||
"HISTORY_FILES",
|
||||
"HistoryCleanupDict",
|
||||
"HistorySudoLinesDict",
|
||||
"SudoHistoryCleanupDict",
|
||||
"SudoHistoryMixin",
|
||||
"cleanup_script",
|
||||
"count_script",
|
||||
"owners_script",
|
||||
]
|
||||
@@ -0,0 +1,285 @@
|
||||
"""TLS certificates: trust on first use by fingerprint, a changed one refused before any login.
|
||||
|
||||
Network devices ship self-signed certificates no CA vouches for. A driver that
|
||||
then turns verification off logs in with the device's credentials at whatever
|
||||
answers on its address (NetOrk/netork#1167, NAPALM/napalm-hpe-aruba-procurve#17).
|
||||
The same answer as for SSH host keys (:mod:`napalm_device_types.ssh_host_keys`):
|
||||
the layer above keeps the store of the certificates it trusts, a driver verifies.
|
||||
|
||||
* It receives the fingerprints to expect in ``optional_args["tls_fingerprints"]``
|
||||
(:data:`KNOWN_FINGERPRINTS_ARG`), by the port they were seen on:
|
||||
``{"443": "SHA256:AB:CD:…"}``, the SHA-256 of the DER certificate as
|
||||
``openssl x509 -fingerprint -sha256`` prints it (the ``SHA256:`` prefix,
|
||||
colons and case are optional, :func:`normalise_fingerprint`). With a
|
||||
fingerprint, any other certificate is refused before a password leaves the
|
||||
caller; no CA is consulted. Without one, the first certificate is accepted:
|
||||
trust on first use.
|
||||
* It reports every certificate it sees through
|
||||
``optional_args["tls_certificate_observer"]`` (:data:`OBSERVER_ARG`), a
|
||||
callable that receives one dict per connection::
|
||||
|
||||
{"host": str, "port": int,
|
||||
"seen": {"fingerprint"},
|
||||
"expected": {"fingerprint"} | None,
|
||||
"accepted": bool}
|
||||
|
||||
``accepted`` is False only for a refused certificate; the connection then
|
||||
raises :class:`TlsCertificateMismatch`. A failing observer is logged and
|
||||
ignored.
|
||||
|
||||
A caller that passes neither keeps working as before (first use, every time),
|
||||
and a driver that ignores both still connects for a caller that passes them.
|
||||
|
||||
:class:`CertificateCheck` is the one implementation. :meth:`~CertificateCheck.verify_server`
|
||||
takes the certificate from a handshake of its own, before any request;
|
||||
:meth:`~CertificateCheck.pin` makes every later connection of a ``requests``
|
||||
session present that same certificate (urllib3's ``assert_fingerprint``), so
|
||||
another host cannot step in between the check and the login;
|
||||
:meth:`~CertificateCheck.refusing` turns such a refusal into
|
||||
:class:`TlsCertificateMismatch`; :meth:`~CertificateCheck.observe` reports the
|
||||
certificate once the login stands.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import logging
|
||||
import re
|
||||
import socket
|
||||
import ssl
|
||||
from collections.abc import Callable, Iterator, Mapping
|
||||
from contextlib import contextmanager
|
||||
from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
from napalm.base.exceptions import ConnectionException
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
#: The ``optional_args`` key that carries the fingerprints to expect, by port.
|
||||
KNOWN_FINGERPRINTS_ARG = "tls_fingerprints"
|
||||
#: The ``optional_args`` key that carries the callable every seen certificate goes to.
|
||||
OBSERVER_ARG = "tls_certificate_observer"
|
||||
|
||||
_PREFIX = "SHA256:"
|
||||
_SHA256_HEX = re.compile(r"^[0-9A-F]{64}$")
|
||||
#: What urllib3 says when a connection presents another certificate than the pinned one
|
||||
#: (``got "…"``; 1.26 writes the digest as ``b'…'``, escaped once requests wraps it).
|
||||
_URLLIB3_MISMATCH = re.compile(r"Fingerprints did not match.*?got \"(?:b\\?')?([0-9a-fA-F]{64})")
|
||||
|
||||
|
||||
def fingerprint(der: bytes) -> str:
|
||||
"""``SHA256:AB:CD:…``: the SHA-256 of a DER certificate, as openssl prints it."""
|
||||
return _format(hashlib.sha256(der).hexdigest())
|
||||
|
||||
|
||||
def _format(hex_digest: str) -> str:
|
||||
upper = hex_digest.upper()
|
||||
return _PREFIX + ":".join(upper[i : i + 2] for i in range(0, len(upper), 2))
|
||||
|
||||
|
||||
def normalise_fingerprint(value: Any) -> str:
|
||||
"""*value* as :func:`fingerprint` writes it; ``SHA256:``, colons, spaces and case optional."""
|
||||
if not isinstance(value, str):
|
||||
raise ValueError(f"not a SHA-256 certificate fingerprint: {value!r}")
|
||||
text = value.strip()
|
||||
if text[: len(_PREFIX)].upper() == _PREFIX:
|
||||
text = text[len(_PREFIX) :]
|
||||
digits = re.sub(r"[\s:]", "", text).upper()
|
||||
if not _SHA256_HEX.match(digits):
|
||||
raise ValueError(f"not a SHA-256 certificate fingerprint: {value!r}")
|
||||
return _format(digits)
|
||||
|
||||
|
||||
def _hex(fingerprint_: str) -> str:
|
||||
"""The bare lower-case hex urllib3's ``assert_fingerprint`` takes."""
|
||||
return fingerprint_.removeprefix(_PREFIX).replace(":", "").lower()
|
||||
|
||||
|
||||
def fetch_certificate(
|
||||
host: str,
|
||||
port: int,
|
||||
timeout: float,
|
||||
*,
|
||||
connect: Callable[..., socket.socket] = socket.create_connection,
|
||||
) -> bytes:
|
||||
"""The DER certificate *host* presents on *port*, from a handshake that sends nothing else.
|
||||
|
||||
No CA and no host name are checked: the certificate is judged by its fingerprint.
|
||||
"""
|
||||
context = ssl.create_default_context()
|
||||
context.check_hostname = False
|
||||
context.verify_mode = ssl.CERT_NONE
|
||||
with connect((host, port), timeout=timeout) as sock:
|
||||
with context.wrap_socket(sock, server_hostname=host) as tls:
|
||||
der = tls.getpeercert(binary_form=True)
|
||||
if not der: # pragma: no cover - a TLS server always presents one
|
||||
raise ConnectionException(f"{host}:{port} presented no TLS certificate")
|
||||
return der
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CertificateObservation:
|
||||
"""The certificate one connection saw, against the one it was to expect."""
|
||||
|
||||
host: str
|
||||
port: int
|
||||
#: The fingerprint of the certificate presented.
|
||||
seen: str
|
||||
#: The fingerprint it was to present; None on first use.
|
||||
expected: str | None
|
||||
|
||||
@property
|
||||
def accepted(self) -> bool:
|
||||
"""True on first use and for the expected certificate; False for a refused one."""
|
||||
return self.expected is None or self.expected == self.seen
|
||||
|
||||
def as_dict(self) -> dict[str, Any]:
|
||||
return {
|
||||
"host": self.host,
|
||||
"port": self.port,
|
||||
"seen": {"fingerprint": self.seen},
|
||||
"expected": {"fingerprint": self.expected} if self.expected else None,
|
||||
"accepted": self.accepted,
|
||||
}
|
||||
|
||||
|
||||
class TlsCertificateMismatch(ConnectionException): # type: ignore[misc]
|
||||
"""The device presented another TLS certificate than the one it is known by.
|
||||
|
||||
Raised before any request that carries credentials: nothing was sent to it.
|
||||
"""
|
||||
|
||||
def __init__(self, observation: CertificateObservation) -> None:
|
||||
self.observation = observation
|
||||
super().__init__(
|
||||
f"TLS certificate mismatch at {observation.host}:{observation.port}: it presented "
|
||||
f"{observation.seen}, the trusted certificate is {observation.expected or 'none'}. "
|
||||
"Nothing was sent to it."
|
||||
)
|
||||
|
||||
|
||||
def _expected(known: Any, port: int) -> str | None:
|
||||
"""The fingerprint in *known* (``{port: fingerprint}``, port as text or number) for *port*."""
|
||||
if not known:
|
||||
return None
|
||||
if not isinstance(known, Mapping):
|
||||
raise ValueError(f"TLS fingerprints must be given by port, not as {type(known).__name__}")
|
||||
value = known.get(str(port)) or known.get(port)
|
||||
return normalise_fingerprint(value) if value else None
|
||||
|
||||
|
||||
def _refused_fingerprint(exc: BaseException) -> str | None:
|
||||
"""The fingerprint urllib3 saw when it refused a pinned connection, in *exc*'s chain."""
|
||||
seen: set[int] = set()
|
||||
stack: list[BaseException | None] = [exc]
|
||||
while stack:
|
||||
current = stack.pop()
|
||||
if current is None or id(current) in seen:
|
||||
continue
|
||||
seen.add(id(current))
|
||||
match = _URLLIB3_MISMATCH.search(str(current))
|
||||
if match:
|
||||
return _format(match.group(1))
|
||||
stack.extend([current.__cause__, current.__context__])
|
||||
return None
|
||||
|
||||
|
||||
class CertificateCheck:
|
||||
"""How one driver connection verifies its host's TLS certificate.
|
||||
|
||||
*optional_args* are the driver's (both keys optional), *host* and *port* the
|
||||
address the connection goes to.
|
||||
"""
|
||||
|
||||
def __init__(self, optional_args: Mapping[str, Any] | None, host: str, port: int) -> None:
|
||||
args = optional_args or {}
|
||||
self.host = host
|
||||
self.port = int(port)
|
||||
self.expected = _expected(args.get(KNOWN_FINGERPRINTS_ARG), self.port)
|
||||
self._observer = args.get(OBSERVER_ARG)
|
||||
self._verified: CertificateObservation | None = None
|
||||
|
||||
def verify_server(
|
||||
self,
|
||||
timeout: float,
|
||||
*,
|
||||
connect: Callable[..., socket.socket] = socket.create_connection,
|
||||
) -> CertificateObservation:
|
||||
"""Take the certificate from a handshake of its own and :meth:`verify` it."""
|
||||
return self.verify(fetch_certificate(self.host, self.port, timeout, connect=connect))
|
||||
|
||||
def verify(self, der: bytes) -> CertificateObservation:
|
||||
"""Judge the DER certificate *der*: a refused one is reported and raises.
|
||||
|
||||
An accepted one is reported by :meth:`observe`, once the login stands.
|
||||
"""
|
||||
observation = CertificateObservation(self.host, self.port, fingerprint(der), self.expected)
|
||||
if not observation.accepted:
|
||||
self._report(observation)
|
||||
raise TlsCertificateMismatch(observation)
|
||||
self._verified = observation
|
||||
return observation
|
||||
|
||||
def pin(self, session: Any) -> None:
|
||||
"""Every HTTPS connection of the ``requests`` *session* must present the verified one."""
|
||||
verified = self._require_verified()
|
||||
session.mount("https://", _pinned_adapter(_hex(verified.seen)))
|
||||
|
||||
@contextmanager
|
||||
def refusing(self) -> Iterator[None]:
|
||||
"""Around requests on a pinned session: a refused certificate becomes a mismatch."""
|
||||
try:
|
||||
yield
|
||||
except Exception as exc:
|
||||
offered = _refused_fingerprint(exc)
|
||||
if offered is None:
|
||||
raise
|
||||
trusted = self.expected or self._require_verified().seen
|
||||
observation = CertificateObservation(self.host, self.port, offered, trusted)
|
||||
self._report(observation)
|
||||
raise TlsCertificateMismatch(observation) from exc
|
||||
|
||||
def observe(self) -> CertificateObservation:
|
||||
"""After the login: the verified certificate, reported."""
|
||||
verified = self._require_verified()
|
||||
self._report(verified)
|
||||
return verified
|
||||
|
||||
def _require_verified(self) -> CertificateObservation:
|
||||
if self._verified is None:
|
||||
raise RuntimeError("verify the certificate first (verify_server or verify)")
|
||||
return self._verified
|
||||
|
||||
def _report(self, observation: CertificateObservation) -> None:
|
||||
if not callable(self._observer):
|
||||
return
|
||||
try:
|
||||
self._observer(observation.as_dict())
|
||||
except Exception: # the store is the caller's; a connection does not hang on it
|
||||
logger.warning(
|
||||
"Could not report the TLS certificate of %s:%s",
|
||||
self.host,
|
||||
self.port,
|
||||
exc_info=True,
|
||||
)
|
||||
|
||||
|
||||
def _pinned_adapter(hex_digest: str) -> Any:
|
||||
"""A ``requests`` adapter whose connections, direct or through a proxy, must match *hex_digest*.
|
||||
|
||||
``requests`` is imported here: it comes with napalm, and a module that only
|
||||
verifies a fingerprint does not need it loaded.
|
||||
"""
|
||||
from requests.adapters import HTTPAdapter
|
||||
|
||||
class _PinnedAdapter(HTTPAdapter): # type: ignore[misc,unused-ignore]
|
||||
def init_poolmanager(self, *args: Any, **kwargs: Any) -> None:
|
||||
kwargs["assert_fingerprint"] = hex_digest
|
||||
super().init_poolmanager(*args, **kwargs)
|
||||
|
||||
def proxy_manager_for(self, proxy: str, **kwargs: Any) -> Any:
|
||||
kwargs["assert_fingerprint"] = hex_digest
|
||||
return super().proxy_manager_for(proxy, **kwargs)
|
||||
|
||||
return _PinnedAdapter()
|
||||
@@ -0,0 +1,26 @@
|
||||
"""Moving a guest to another node of its cluster (``HypervisorDriver.migrate_vm``).
|
||||
|
||||
``get_vm_migration_preflight`` reports what the platform says before a move:
|
||||
which nodes would take the guest, which refuse it and why, and what the guest
|
||||
holds on its current node. ``migrate_vm`` then moves it and blocks until the
|
||||
platform's task ended; a failure raises with the platform's own cause.
|
||||
``set_node_maintenance`` switches a node's HA maintenance mode, which makes the
|
||||
HA manager move its HA guests away while it is on.
|
||||
|
||||
Facts only: which node to pick, and whether a local disk is worth copying, is
|
||||
the caller's to decide.
|
||||
"""
|
||||
|
||||
from typing import TypedDict
|
||||
|
||||
|
||||
class MigrationPreflightDict(TypedDict):
|
||||
"""Where a guest could move to, and what stands in the way."""
|
||||
|
||||
running: bool
|
||||
allowed_nodes: list[str] # nodes the platform would move it to
|
||||
refused_nodes: dict[str, str] # node -> the platform's reason, in plain words
|
||||
# volume ids on storage the target does not share; they block nothing by
|
||||
# themselves, the migration copies them
|
||||
local_disks: list[str]
|
||||
local_resources: list[str] # e.g. "hostpci0", "usb0": they block the migration
|
||||
@@ -0,0 +1,29 @@
|
||||
"""Asking for a VM's clean shutdown without waiting for it.
|
||||
|
||||
``HypervisorDriver.stop_vm`` blocks until the VM is stopped, within a limit
|
||||
of the driver's own. A caller that keeps its waits as timers asks with
|
||||
``begin_stop_vm`` and reads ``get_vm_status`` until the VM says it stopped.
|
||||
|
||||
Declared under ``if TYPE_CHECKING``: a driver mixes this in when its
|
||||
hypervisor can take the request on its own; ``hasattr`` tells a caller
|
||||
whether it did. No role base carries it.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
|
||||
class VmStopRequestMixin:
|
||||
if TYPE_CHECKING:
|
||||
|
||||
def begin_stop_vm(self, name: str) -> None:
|
||||
"""
|
||||
Asks the hypervisor to shut a virtual machine down cleanly (ACPI)
|
||||
and returns as soon as the request was accepted.
|
||||
|
||||
:param name: VM name or its ``vmid``.
|
||||
:raises ValueError: If no VM with the given name/ID exists.
|
||||
:raises RuntimeError: If the hypervisor refuses the request.
|
||||
"""
|
||||
...
|
||||
+6
-7
@@ -4,10 +4,10 @@ build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "napalm-device-types"
|
||||
version = "4.22.0"
|
||||
version = "4.27.0"
|
||||
description = "Abstract device-type base classes for NAPALM drivers"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.10"
|
||||
requires-python = ">=3.12"
|
||||
license = "Apache-2.0"
|
||||
authors = [
|
||||
{ name = "Christian Manivong", email = "christian@manivong.de" },
|
||||
@@ -30,8 +30,6 @@ classifiers = [
|
||||
"Intended Audience :: System Administrators",
|
||||
"Operating System :: OS Independent",
|
||||
"Programming Language :: Python :: 3",
|
||||
"Programming Language :: Python :: 3.10",
|
||||
"Programming Language :: Python :: 3.11",
|
||||
"Programming Language :: Python :: 3.12",
|
||||
"Programming Language :: Python :: 3.13",
|
||||
"Topic :: System :: Networking",
|
||||
@@ -39,7 +37,8 @@ classifiers = [
|
||||
"Typing :: Typed",
|
||||
]
|
||||
dependencies = [
|
||||
"napalm>=4.0",
|
||||
"napalm>=5.1", # older releases import pkg_resources, which a 3.12 venv lacks
|
||||
"requests>=2.25.0",
|
||||
"typing-extensions>=4.0",
|
||||
]
|
||||
|
||||
@@ -65,7 +64,7 @@ napalm_device_types = ["py.typed"]
|
||||
|
||||
[tool.ruff]
|
||||
line-length = 100
|
||||
target-version = "py310"
|
||||
target-version = "py312"
|
||||
|
||||
[tool.ruff.lint]
|
||||
select = ["E", "F", "I", "UP", "B"]
|
||||
@@ -75,7 +74,7 @@ select = ["E", "F", "I", "UP", "B"]
|
||||
"tests/**" = ["E501"]
|
||||
|
||||
[tool.mypy]
|
||||
python_version = "3.10"
|
||||
python_version = "3.12"
|
||||
strict = true
|
||||
ignore_missing_imports = true
|
||||
|
||||
|
||||
@@ -25,7 +25,6 @@
|
||||
"napalm_device_types/firewall_rules.py::FirewallRuleMixin.diff_firewall_rules": 56,
|
||||
"napalm_device_types/hypervisor.py::HypervisorDriver.create_vm_from_cloud_init": 84,
|
||||
"napalm_device_types/hypervisor.py::HypervisorDriver.get_vm_config": 77,
|
||||
"napalm_device_types/ssh_server.py::OpenSshServerMixin.harden_ssh_server": 52,
|
||||
"napalm_device_types/ssh_server.py::plan_ssh_hardening": 60,
|
||||
"napalm_device_types/storage.py::StorageDriver.get_volumes": 52,
|
||||
"napalm_device_types/switch.py::SwitchDriver.get_dot1x_ports": 51,
|
||||
@@ -33,7 +32,7 @@
|
||||
},
|
||||
"mypy_relaxed": [],
|
||||
"python_file_lines": {
|
||||
"napalm_device_types/models.py": 1504
|
||||
"napalm_device_types/models.py": 1454
|
||||
},
|
||||
"untyped_public": []
|
||||
}
|
||||
+306
-13
@@ -20,6 +20,7 @@ from napalm_device_types.apt_proxy import (
|
||||
AptProxyMixin,
|
||||
apt_fetch_command,
|
||||
comment_out_command,
|
||||
https_proxy_for,
|
||||
parse_apt_fetch,
|
||||
parse_apt_proxy,
|
||||
same_proxy_url,
|
||||
@@ -141,7 +142,7 @@ class TestWhereItIsSet:
|
||||
|
||||
|
||||
class TestSources:
|
||||
def test_https_sources_bypass_an_http_proxy(self):
|
||||
def test_sources_by_scheme(self):
|
||||
report = parse_apt_proxy(
|
||||
_report(
|
||||
sources=[
|
||||
@@ -155,6 +156,102 @@ class TestSources:
|
||||
assert report["https_sources"] == ["https://download.docker.com/linux/ubuntu/dists/noble/"]
|
||||
|
||||
|
||||
class TestHttpsProxy:
|
||||
"""apt's https method asks ``Acquire::https::Proxy`` and then ``Acquire::http::Proxy``:
|
||||
an http proxy alone takes the https sources too (NetOrk/netork#1811). Measured with
|
||||
apt 2.8.3's apt-helper against a dead port."""
|
||||
|
||||
def test_the_http_proxy_alone_takes_https_too(self):
|
||||
report = parse_apt_proxy(_report(dump=['Acquire::http::Proxy "http://c:3142";']))
|
||||
assert report["https_proxy"] == "http://c:3142"
|
||||
|
||||
def test_an_https_proxy_of_its_own_wins(self):
|
||||
report = parse_apt_proxy(
|
||||
_report(
|
||||
dump=[
|
||||
'Acquire::http::Proxy "http://c:3142";',
|
||||
'Acquire::https::Proxy "http://other:3128";',
|
||||
]
|
||||
)
|
||||
)
|
||||
|
||||
assert (report["proxy"], report["https_proxy"]) == ("http://c:3142", "http://other:3128")
|
||||
|
||||
@pytest.mark.parametrize("value", ["DIRECT", "direct"])
|
||||
def test_direct_https_is_none(self, value):
|
||||
report = parse_apt_proxy(
|
||||
_report(
|
||||
dump=['Acquire::http::Proxy "http://c:3142";', f'Acquire::https::Proxy "{value}";']
|
||||
)
|
||||
)
|
||||
assert (report["proxy"], report["https_proxy"]) == ("http://c:3142", None)
|
||||
|
||||
def test_an_empty_https_proxy_is_not_set(self):
|
||||
"""apt takes an empty value for unset and goes on to the http one."""
|
||||
report = parse_apt_proxy(
|
||||
_report(dump=['Acquire::http::Proxy "http://c:3142";', 'Acquire::https::Proxy "";'])
|
||||
)
|
||||
assert report["https_proxy"] == "http://c:3142"
|
||||
|
||||
def test_per_host_https_then_http(self):
|
||||
report = parse_apt_proxy(
|
||||
_report(
|
||||
dump=[
|
||||
'Acquire::http::Proxy::a.example "DIRECT";',
|
||||
'Acquire::http::Proxy::b.example "DIRECT";',
|
||||
'Acquire::https::Proxy::b.example "http://c:3142";',
|
||||
]
|
||||
)
|
||||
)
|
||||
|
||||
assert report["https_per_host"] == {"a.example": "DIRECT", "b.example": "http://c:3142"}
|
||||
assert report["per_host"] == {"a.example": "DIRECT", "b.example": "DIRECT"}
|
||||
|
||||
def test_where_the_https_proxy_is_set(self):
|
||||
report = parse_apt_proxy(
|
||||
_report(
|
||||
parts=[
|
||||
f'{_D}00proxy:1:Acquire::http::Proxy "http://c:3142";',
|
||||
f'{_D}90curtin-aptproxy:2:Acquire::https::Proxy "http://c:3142";',
|
||||
f'{_D}91x:1:Acquire::https {{ Proxy "http://n:1"; }};',
|
||||
f'{_D}92x:1:// Acquire::https::Proxy "http://c:3142";',
|
||||
f'{_D}93x:1:Acquire::https::Proxy::h.example "DIRECT";',
|
||||
]
|
||||
)
|
||||
)
|
||||
|
||||
assert report["https_settings"] == [
|
||||
{
|
||||
"file": f"{_D}90curtin-aptproxy",
|
||||
"line": 2,
|
||||
"value": "http://c:3142",
|
||||
"editable": True,
|
||||
},
|
||||
{"file": f"{_D}91x", "line": 1, "value": "http://n:1", "editable": False},
|
||||
]
|
||||
assert [s["file"] for s in report["settings"]] == [f"{_D}00proxy"]
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"host, proxy",
|
||||
[
|
||||
("deb.example", "http://c:3142"),
|
||||
("direct.example", None),
|
||||
("Other.Example", "http://o:1"),
|
||||
],
|
||||
)
|
||||
def test_the_proxy_for_one_host(self, host, proxy):
|
||||
report = parse_apt_proxy(
|
||||
_report(
|
||||
dump=[
|
||||
'Acquire::http::Proxy "http://c:3142";',
|
||||
'Acquire::https::Proxy::direct.example "DIRECT";',
|
||||
'Acquire::http::Proxy::other.example "http://o:1";',
|
||||
]
|
||||
)
|
||||
)
|
||||
assert https_proxy_for(report, host) == proxy
|
||||
|
||||
|
||||
class TestSameProxyUrl:
|
||||
@pytest.mark.parametrize(
|
||||
"a, b",
|
||||
@@ -199,6 +296,12 @@ E: Download Failed"""
|
||||
_FETCHED = """Get:1 http://172.22.8.50:3142/acng-report.html
|
||||
Fetched 14.0 kB in 0s (0 B/s)"""
|
||||
|
||||
#: apt-cacher-ng without a PassThroughPattern, as apt 2.8.3 tells it (NetOrk/netork#1811).
|
||||
_CONNECT_DENIED = """Err:1 https://download.docker.com/linux/ubuntu/dists/noble/InRelease
|
||||
Invalid response from proxy: HTTP/1.0 403 CONNECT denied (ask the admin to allow HTTPS tunnels) [IP: 10.40.100.10 3142]
|
||||
E: Failed to fetch https://download.docker.com/linux/ubuntu/dists/noble/InRelease Invalid response from proxy: HTTP/1.0 403 CONNECT denied (ask the admin to allow HTTPS tunnels) [IP: 10.40.100.10 3142]
|
||||
E: Download Failed"""
|
||||
|
||||
|
||||
def _fetch(output, rc, match=None):
|
||||
lines = ["APROXY_BEGIN", "[fetch]", *output.splitlines(), "[rc]", str(rc)]
|
||||
@@ -243,6 +346,12 @@ class TestFetch:
|
||||
assert result["expected"] is False
|
||||
assert result["error"] == "503 Host not found [IP: 172.22.8.50 3142]"
|
||||
|
||||
def test_a_refused_tunnel(self):
|
||||
result = parse_apt_fetch(_fetch(_CONNECT_DENIED, 100))
|
||||
|
||||
assert result["reachable"] is False
|
||||
assert "403 CONNECT denied" in result["error"]
|
||||
|
||||
def test_without_apt_helper_nothing_could_be_tried(self):
|
||||
with pytest.raises(ValueError, match="apt-helper"):
|
||||
parse_apt_fetch(_fetch("sh: 1: /usr/lib/apt/apt-helper: not found", 127))
|
||||
@@ -265,6 +374,21 @@ class TestFetchCommand:
|
||||
command = apt_fetch_command("http://archive.ubuntu.com/ubuntu/dists/noble/InRelease")
|
||||
assert "DIRECT" not in command
|
||||
|
||||
def test_an_https_source_through_apt_s_own_proxy(self):
|
||||
command = apt_fetch_command(
|
||||
"https://download.docker.com/linux/ubuntu/dists/noble/InRelease"
|
||||
)
|
||||
assert "DIRECT" not in command
|
||||
assert "download-file https://download.docker.com/" in command
|
||||
|
||||
def test_past_the_proxy_to_an_https_host(self):
|
||||
"""For https the host's own https setting comes first; the http one alone
|
||||
would lose to an https proxy for that host."""
|
||||
command = apt_fetch_command("https://h.example/x", direct=True)
|
||||
|
||||
assert "'Acquire::https::Proxy::h.example=DIRECT'" in command
|
||||
assert "'Acquire::http::Proxy::h.example=DIRECT'" in command
|
||||
|
||||
@pytest.mark.parametrize("url", ["http://c:3142/x'; rm -rf /", "ftp://c/x", "c:3142"])
|
||||
def test_a_url_that_would_break_the_command_is_refused(self, url):
|
||||
with pytest.raises(ValueError):
|
||||
@@ -275,6 +399,7 @@ def test_the_commands_are_valid_posix_sh():
|
||||
for command in (
|
||||
APT_PROXY_COMMAND,
|
||||
apt_fetch_command("http://c:3142/x", direct=True, expect="y"),
|
||||
apt_fetch_command("https://h.example/dists/x/InRelease"),
|
||||
):
|
||||
subprocess.run(["sh", "-n", "-c", command], check=True)
|
||||
|
||||
@@ -283,16 +408,28 @@ def test_the_commands_are_valid_posix_sh():
|
||||
def test_the_report_command_on_this_host():
|
||||
output = subprocess.run(["sh", "-c", APT_PROXY_COMMAND], capture_output=True, text=True).stdout
|
||||
report = parse_apt_proxy(output)
|
||||
assert set(report) >= {"proxy", "settings", "https_sources"}
|
||||
assert set(report) >= {"proxy", "https_proxy", "settings", "https_settings", "https_sources"}
|
||||
|
||||
|
||||
_DOCKER = "https://download.docker.com/linux/ubuntu/dists/noble/"
|
||||
|
||||
|
||||
class _Host(AptProxyMixin):
|
||||
"""A Debian host: its apt files, and what apt-config would make of them."""
|
||||
"""A Debian host: its apt files, and what apt-config would make of them.
|
||||
|
||||
def __init__(self, files=None, fetch_rc=0, apt=True):
|
||||
*https_sources* are its sources over https. *tunnel* is whether the proxy lets
|
||||
them through; *direct_rc* is how a fetch of one goes that takes no proxy.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self, files=None, fetch_rc=0, apt=True, https_sources=(), tunnel=True, direct_rc=0
|
||||
):
|
||||
self.files = dict(files or {})
|
||||
self.fetch_rc = fetch_rc
|
||||
self.apt = apt
|
||||
self.https_sources = list(https_sources)
|
||||
self.tunnel = tunnel
|
||||
self.direct_rc = direct_rc
|
||||
self.root = []
|
||||
self.fetched = []
|
||||
|
||||
@@ -309,26 +446,36 @@ class _Host(AptProxyMixin):
|
||||
out.append((path, n, text))
|
||||
return out
|
||||
|
||||
def _effective(self):
|
||||
def _effective(self, scheme="http"):
|
||||
value = None
|
||||
for _path, _n, text in self._settings():
|
||||
m = re.match(r'\s*Acquire::http::Proxy\s+"([^"]*)"', text, re.I)
|
||||
m = re.match(rf'\s*Acquire::{scheme}::Proxy\s+"([^"]*)"', text, re.I)
|
||||
if m:
|
||||
value = m.group(1)
|
||||
return value
|
||||
|
||||
def _run_apt_command(self, command):
|
||||
if command == APT_PROXY_COMMAND:
|
||||
value = self._effective()
|
||||
value, https = self._effective(), self._effective("https")
|
||||
dump = [f'Acquire::http::Proxy "{value}";'] if value else []
|
||||
dump += [f'Acquire::https::Proxy "{https}";'] if https else []
|
||||
return _report(
|
||||
dump=[f'Acquire::http::Proxy "{value}";'] if value else [],
|
||||
dump=dump,
|
||||
parts=[f"{p}:{n}:{t}" for p, n, t in self._settings() if p.startswith(_D)],
|
||||
main=[f"{p}:{n}:{t}" for p, n, t in self._settings() if not p.startswith(_D)],
|
||||
sources=["http://deb.debian.org/debian/dists/trixie/"],
|
||||
sources=["http://deb.debian.org/debian/dists/trixie/", *self.https_sources],
|
||||
)
|
||||
self.fetched.append(command)
|
||||
if re.search(r"download-file '?https://", command):
|
||||
return self._fetch_https(command)
|
||||
return _fetch(_FETCHED if self.fetch_rc == 0 else _REFUSED, self.fetch_rc, match=1)
|
||||
|
||||
def _fetch_https(self, command):
|
||||
https = self._effective("https") or self._effective()
|
||||
if "DIRECT" in command or not https or https.upper() == "DIRECT":
|
||||
return _fetch(_FETCHED if self.direct_rc == 0 else _UNRESOLVED, self.direct_rc)
|
||||
return _fetch(_FETCHED, 0) if self.tunnel else _fetch(_CONNECT_DENIED, 100)
|
||||
|
||||
def _write_apt_file(self, path, content):
|
||||
self.files[path] = content
|
||||
return {"ok": True, "output": f"wrote {path}"}
|
||||
@@ -346,6 +493,41 @@ class _Host(AptProxyMixin):
|
||||
return {"ok": True, "output": done}
|
||||
|
||||
|
||||
class TestCheckHttpsSource:
|
||||
def test_one_source_the_way_apt_fetches_it(self):
|
||||
host = _Host(
|
||||
{f"{_D}00proxy": 'Acquire::http::Proxy "http://c:3142";\n'},
|
||||
https_sources=[_DOCKER],
|
||||
tunnel=False,
|
||||
)
|
||||
|
||||
result = host.check_https_source()
|
||||
|
||||
assert result["url"] == f"{_DOCKER}InRelease"
|
||||
assert result["proxy"] == "http://c:3142"
|
||||
assert result["reachable"] is False and "403 CONNECT denied" in result["error"]
|
||||
assert "DIRECT" not in host.fetched[0]
|
||||
|
||||
def test_none_without_an_https_source(self):
|
||||
host = _Host({f"{_D}00proxy": 'Acquire::http::Proxy "http://c:3142";\n'})
|
||||
assert host.check_https_source() is None
|
||||
|
||||
def test_none_where_https_goes_direct(self):
|
||||
host = _Host(
|
||||
{
|
||||
f"{_D}00proxy": (
|
||||
'Acquire::http::Proxy "http://c:3142";\nAcquire::https::Proxy "DIRECT";\n'
|
||||
)
|
||||
},
|
||||
https_sources=[_DOCKER],
|
||||
)
|
||||
assert host.check_https_source() is None
|
||||
|
||||
def test_not_on_a_host_without_apt(self):
|
||||
with pytest.raises(NotImplementedError):
|
||||
_Host(apt=False).check_https_source()
|
||||
|
||||
|
||||
class TestTheMixin:
|
||||
def test_it_is_a_package_proxy(self):
|
||||
assert issubclass(AptProxyMixin, PackageProxyMixin)
|
||||
@@ -371,6 +553,10 @@ class TestTheMixin:
|
||||
assert "Proxy::c=DIRECT" in host.fetched[0]
|
||||
|
||||
|
||||
def _step(result, name):
|
||||
return next(s for s in result["steps"] if s["step"] == name)
|
||||
|
||||
|
||||
class TestConfigurePackageProxy:
|
||||
def test_a_fresh_host(self):
|
||||
host = _Host()
|
||||
@@ -378,7 +564,14 @@ class TestConfigurePackageProxy:
|
||||
result = host.configure_package_proxy("http://c:3142")
|
||||
|
||||
assert result["success"] is True
|
||||
assert [s["step"] for s in result["steps"]] == ["configure", "conflicts", "verify", "fetch"]
|
||||
assert [s["step"] for s in result["steps"]] == [
|
||||
"configure",
|
||||
"conflicts",
|
||||
"verify",
|
||||
"fetch",
|
||||
"https",
|
||||
]
|
||||
assert "no https source" in result["steps"][-1]["output"]
|
||||
assert host.files[f"{_D}00proxy"] == 'Acquire::http::Proxy "http://c:3142";\n'
|
||||
assert "InRelease" in host.fetched[0] and "DIRECT" not in host.fetched[0]
|
||||
|
||||
@@ -410,7 +603,7 @@ class TestConfigurePackageProxy:
|
||||
|
||||
def test_a_setting_it_cannot_comment_out_fails_verification(self):
|
||||
host = _Host({"/etc/apt/apt.conf": 'Acquire::http { Proxy "http://older:8080"; };\n'})
|
||||
host._effective = lambda: "http://older:8080"
|
||||
host._effective = lambda scheme="http": "http://older:8080"
|
||||
|
||||
result = host.configure_package_proxy("http://c:3142")
|
||||
|
||||
@@ -428,6 +621,100 @@ class TestConfigurePackageProxy:
|
||||
assert result["steps"][-1]["step"] == "fetch"
|
||||
assert "Connection refused" in result["steps"][-1]["output"]
|
||||
|
||||
def test_an_https_source_the_cache_tunnels_stays_with_it(self):
|
||||
host = _Host(https_sources=[_DOCKER])
|
||||
|
||||
result = host.configure_package_proxy("http://c:3142")
|
||||
|
||||
assert result["success"] is True
|
||||
assert host.files[f"{_D}00proxy"] == 'Acquire::http::Proxy "http://c:3142";\n'
|
||||
assert result["steps"][-1]["output"] == (
|
||||
f"fetched {_DOCKER}InRelease through http://c:3142"
|
||||
)
|
||||
|
||||
def test_an_https_source_the_cache_refuses_goes_direct(self):
|
||||
"""NetOrk/netork#1811: apt-cacher-ng without a PassThroughPattern answers
|
||||
403 to every https source apt sends it."""
|
||||
host = _Host(https_sources=[_DOCKER], tunnel=False)
|
||||
|
||||
result = host.configure_package_proxy("http://c:3142")
|
||||
|
||||
assert result["success"] is True
|
||||
assert host.files[f"{_D}00proxy"] == (
|
||||
'Acquire::http::Proxy "http://c:3142";\nAcquire::https::Proxy "DIRECT";\n'
|
||||
)
|
||||
output = result["steps"][-1]["output"]
|
||||
assert "http://c:3142 refused https" in output and "403 CONNECT denied" in output
|
||||
assert f"https sources go direct now: fetched {_DOCKER}InRelease" in output
|
||||
|
||||
def test_a_later_https_line_to_the_cache_is_commented_out(self):
|
||||
"""Ubuntu's installer writes the https proxy too; it would win over 00proxy."""
|
||||
curtin = f"{_D}90curtin-aptproxy"
|
||||
host = _Host(
|
||||
{
|
||||
curtin: (
|
||||
'Acquire::http::Proxy "http://c:3142";\n'
|
||||
'Acquire::https::Proxy "http://c:3142";\n'
|
||||
)
|
||||
},
|
||||
https_sources=[_DOCKER],
|
||||
tunnel=False,
|
||||
)
|
||||
|
||||
result = host.configure_package_proxy("http://c:3142")
|
||||
|
||||
assert result["success"] is True
|
||||
assert host.files[curtin] == (
|
||||
'Acquire::http::Proxy "http://c:3142";\n// Acquire::https::Proxy "http://c:3142";\n'
|
||||
)
|
||||
assert f"{curtin}:2 (http://c:3142) commented out" in result["steps"][-1]["output"]
|
||||
|
||||
def test_an_https_proxy_of_its_own_is_not_the_fix_s(self):
|
||||
host = _Host(
|
||||
{f"{_D}01https": 'Acquire::https::Proxy "http://corp:3128";\n'},
|
||||
https_sources=[_DOCKER],
|
||||
tunnel=False,
|
||||
)
|
||||
|
||||
result = host.configure_package_proxy("http://c:3142")
|
||||
|
||||
assert result["success"] is True
|
||||
assert "no https source goes through http://c:3142" in result["steps"][-1]["output"]
|
||||
assert f"{_D}00proxy" in host.files and "https" not in host.files[f"{_D}00proxy"]
|
||||
|
||||
def test_an_http_error_through_the_tunnel_is_a_tunnel(self):
|
||||
"""The mirror answered, so the cache let the source through."""
|
||||
host = _Host(https_sources=[_DOCKER])
|
||||
host._fetch_https = lambda command: _fetch(_HTTP_ERROR, 100)
|
||||
|
||||
result = host.configure_package_proxy("http://c:3142")
|
||||
|
||||
assert result["success"] is True
|
||||
assert "https" not in host.files[f"{_D}00proxy"]
|
||||
|
||||
def test_refused_and_no_way_direct_fails(self):
|
||||
host = _Host(https_sources=[_DOCKER], tunnel=False, direct_rc=100)
|
||||
|
||||
result = host.configure_package_proxy("http://c:3142")
|
||||
|
||||
assert result["success"] is False
|
||||
assert result["steps"][-1]["step"] == "https"
|
||||
assert "Could not resolve" in result["steps"][-1]["output"]
|
||||
|
||||
def test_a_later_https_line_it_cannot_comment_out_fails(self):
|
||||
host = _Host(
|
||||
{"/etc/apt/apt.conf": 'Acquire::https { Proxy "http://c:3142"; };\n'},
|
||||
https_sources=[_DOCKER],
|
||||
tunnel=False,
|
||||
)
|
||||
host._effective = lambda scheme="http": "http://c:3142"
|
||||
|
||||
result = host.configure_package_proxy("http://c:3142")
|
||||
|
||||
assert result["success"] is False
|
||||
assert result["steps"][-1]["step"] == "https"
|
||||
assert "/etc/apt/apt.conf:1" in result["steps"][-1]["output"]
|
||||
|
||||
def test_without_an_http_source_the_fetch_says_so(self):
|
||||
host = _Host()
|
||||
real = host._run_apt_command
|
||||
@@ -440,7 +727,7 @@ class TestConfigurePackageProxy:
|
||||
result = host.configure_package_proxy("http://c:3142")
|
||||
|
||||
assert result["success"] is True
|
||||
assert "no http source" in result["steps"][-1]["output"]
|
||||
assert "no http source" in _step(result, "fetch")["output"]
|
||||
|
||||
def test_a_direct_source_is_not_proof(self):
|
||||
host = _Host()
|
||||
@@ -451,7 +738,7 @@ class TestConfigurePackageProxy:
|
||||
|
||||
result = host.configure_package_proxy("http://c:3142")
|
||||
|
||||
assert "no http source" in result["steps"][-1]["output"]
|
||||
assert "no http source" in _step(result, "fetch")["output"]
|
||||
|
||||
@pytest.mark.parametrize("url", ['http://x"; Acquire::x "y', "ftp://c", "c:3142"])
|
||||
def test_a_url_that_would_break_the_file_is_refused(self, url):
|
||||
@@ -531,3 +818,9 @@ def test_the_comment_out_command():
|
||||
"sed -i -e '2{/^[[:space:]]*Acquire::http::Proxy[[:space:]]/Is|^|// |}' "
|
||||
"-e '4{/^[[:space:]]*Acquire::http::Proxy[[:space:]]/Is|^|// |}' /etc/apt/apt.conf"
|
||||
)
|
||||
|
||||
|
||||
def test_the_comment_out_command_for_https():
|
||||
assert comment_out_command("/etc/apt/apt.conf", [3], scheme="https") == (
|
||||
"sed -i -e '3{/^[[:space:]]*Acquire::https::Proxy[[:space:]]/Is|^|// |}' /etc/apt/apt.conf"
|
||||
)
|
||||
@@ -0,0 +1,27 @@
|
||||
"""An ARP or neighbour entry whose age the device does not report says so.
|
||||
|
||||
NAPALM's ``get_arp_table`` and ``get_ipv6_neighbors_table`` require an ``age``
|
||||
on every entry, and a driver that could not read one used to fill in ``0.0``.
|
||||
To a caller that is the freshest sighting there is: netOrk took a neighbour the
|
||||
device had not confirmed for hours as seen this second (NetOrk/netork#1327).
|
||||
Unknown is ``-1.0``, as in NAPALM's own IOS driver; ``0.0`` means "just now".
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from napalm_device_types import ARP_AGE_UNKNOWN
|
||||
from napalm_device_types.arp import ARP_AGE_UNKNOWN as FROM_MODULE
|
||||
|
||||
|
||||
def test_an_unknown_age_is_minus_one():
|
||||
assert ARP_AGE_UNKNOWN == -1.0
|
||||
assert isinstance(ARP_AGE_UNKNOWN, float)
|
||||
|
||||
|
||||
def test_the_package_exports_the_modules_constant():
|
||||
assert ARP_AGE_UNKNOWN is FROM_MODULE
|
||||
|
||||
|
||||
def test_no_real_age_can_be_mistaken_for_unknown():
|
||||
# An age is seconds since the entry was last confirmed: never negative.
|
||||
assert ARP_AGE_UNKNOWN < 0
|
||||
@@ -0,0 +1,234 @@
|
||||
"""A backup server is a role of its own (NAPALM/napalm-device-types#82).
|
||||
|
||||
What makes a device a backup server -- datastores, the backup groups in them,
|
||||
the jobs that keep them (garbage collection, verify, prune, sync, tape) and the
|
||||
remotes it syncs with -- has no counterpart in ``StorageDriver``. A backup
|
||||
server that also manages its own disks lists ``StorageDriver`` as a second role.
|
||||
|
||||
The declarations sit under ``TYPE_CHECKING`` (see test_role_contracts), so the
|
||||
signatures are read from the source rather than from the class.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import ast
|
||||
import inspect
|
||||
from typing import NotRequired, get_args, get_type_hints
|
||||
|
||||
import pytest
|
||||
|
||||
import napalm_device_types.backup_server as backup_server_module
|
||||
from napalm_device_types import (
|
||||
BACKUP_JOB_KINDS,
|
||||
BackupDatastoreDict,
|
||||
BackupGroupDict,
|
||||
BackupRemoteDict,
|
||||
BackupServerDriver,
|
||||
BackupServerJobDict,
|
||||
BackupServerRefDict,
|
||||
BackupSnapshotDict,
|
||||
BackupSnapshotRefDict,
|
||||
BackupTaskDict,
|
||||
BackupTaskStatusDict,
|
||||
ClusterStorageDict,
|
||||
OSDriver,
|
||||
StorageDriver,
|
||||
)
|
||||
from napalm_device_types.backups import BackupJobKind, BackupRunState, BackupVerifyState
|
||||
from napalm_device_types.roles import primary_role_of, role_keys_of
|
||||
|
||||
|
||||
def _declared(name: str) -> ast.FunctionDef:
|
||||
tree = ast.parse(inspect.getsource(backup_server_module))
|
||||
for node in ast.walk(tree):
|
||||
if isinstance(node, ast.FunctionDef) and node.name == name:
|
||||
return node
|
||||
raise AssertionError(f"BackupServerDriver does not declare {name}()")
|
||||
|
||||
|
||||
def _signature(name: str) -> tuple[list[str], str]:
|
||||
fn = _declared(name)
|
||||
params = ast.unparse(fn.args).split(", ")[1:]
|
||||
return params, ast.unparse(fn.returns) if fn.returns else ""
|
||||
|
||||
|
||||
class TestRole:
|
||||
def test_role_key_and_label(self):
|
||||
assert vars(BackupServerDriver)["ROLE"] == "backup_server"
|
||||
assert BackupServerDriver.TYPE_LABEL == "Backup Server"
|
||||
|
||||
def test_backup_server_first_storage_second(self):
|
||||
"""A PBS manages its disks and ZFS pools too; that part is storage."""
|
||||
|
||||
class Pbs(BackupServerDriver, StorageDriver, OSDriver):
|
||||
pass
|
||||
|
||||
assert role_keys_of(Pbs) == ["backup_server", "storage", "linux"]
|
||||
assert primary_role_of(Pbs) == "backup_server"
|
||||
|
||||
|
||||
class TestDeclarations:
|
||||
@pytest.mark.parametrize(
|
||||
("name", "params", "returns"),
|
||||
[
|
||||
("get_backup_datastores", [], "list[BackupDatastoreDict]"),
|
||||
("get_backup_groups", ["datastore: str=''"], "list[BackupGroupDict]"),
|
||||
(
|
||||
"get_backup_snapshots",
|
||||
["datastore: str", "namespace: str", "group_type: str", "group_id: str"],
|
||||
"list[BackupSnapshotDict]",
|
||||
),
|
||||
("get_backup_jobs", [], "list[BackupServerJobDict]"),
|
||||
("get_backup_remotes", [], "list[BackupRemoteDict]"),
|
||||
("start_garbage_collection", ["datastore: str"], "BackupTaskDict"),
|
||||
("run_backup_job", ["kind: BackupJobKind", "job_id: str"], "BackupTaskDict"),
|
||||
("verify_backup_snapshot", ["snapshot: BackupSnapshotRefDict"], "BackupTaskDict"),
|
||||
(
|
||||
"set_backup_snapshot_protected",
|
||||
["snapshot: BackupSnapshotRefDict", "protected: bool"],
|
||||
"None",
|
||||
),
|
||||
("delete_backup_snapshot", ["snapshot: BackupSnapshotRefDict"], "None"),
|
||||
("get_backup_task", ["task: str"], "BackupTaskStatusDict"),
|
||||
],
|
||||
)
|
||||
def test_declared_with_its_signature(self, name, params, returns):
|
||||
got_params, got_returns = _signature(name)
|
||||
assert [p.replace(" ", "") for p in got_params] == [p.replace(" ", "") for p in params]
|
||||
assert got_returns == returns
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"name",
|
||||
[
|
||||
"get_backup_datastores",
|
||||
"get_backup_groups",
|
||||
"get_backup_snapshots",
|
||||
"get_backup_jobs",
|
||||
"get_backup_remotes",
|
||||
"start_garbage_collection",
|
||||
"run_backup_job",
|
||||
"verify_backup_snapshot",
|
||||
"set_backup_snapshot_protected",
|
||||
"delete_backup_snapshot",
|
||||
"get_backup_task",
|
||||
],
|
||||
)
|
||||
def test_absent_until_a_driver_implements_it(self, name):
|
||||
assert not hasattr(BackupServerDriver, name)
|
||||
|
||||
def test_every_declaration_has_a_docstring(self):
|
||||
tree = ast.parse(inspect.getsource(backup_server_module))
|
||||
missing = [
|
||||
n.name
|
||||
for n in ast.walk(tree)
|
||||
if isinstance(n, ast.FunctionDef) and not ast.get_docstring(n)
|
||||
]
|
||||
assert missing == []
|
||||
|
||||
|
||||
class TestShapes:
|
||||
"""Neutral names: nothing here is Proxmox's. A value the platform does not
|
||||
report is None; it is never guessed."""
|
||||
|
||||
def test_datastore(self):
|
||||
assert set(get_type_hints(BackupDatastoreDict)) == {
|
||||
"name",
|
||||
"path",
|
||||
"backend",
|
||||
"total",
|
||||
"used",
|
||||
"available",
|
||||
"estimated_full",
|
||||
"maintenance",
|
||||
"comment",
|
||||
}
|
||||
|
||||
def test_group_carries_its_newest_snapshot_not_all_of_them(self):
|
||||
assert set(get_type_hints(BackupGroupDict)) == {
|
||||
"datastore",
|
||||
"namespace",
|
||||
"type",
|
||||
"id",
|
||||
"owner",
|
||||
"count",
|
||||
"newest",
|
||||
"oldest_time",
|
||||
"comment",
|
||||
}
|
||||
assert get_type_hints(BackupGroupDict)["newest"] == BackupSnapshotDict | None
|
||||
|
||||
def test_snapshot(self):
|
||||
assert set(get_type_hints(BackupSnapshotDict)) == {
|
||||
"time",
|
||||
"size",
|
||||
"verify",
|
||||
"protected",
|
||||
"comment",
|
||||
}
|
||||
assert set(get_args(BackupVerifyState)) == {"ok", "failed"}
|
||||
|
||||
def test_snapshot_reference_names_one_snapshot(self):
|
||||
assert set(get_type_hints(BackupSnapshotRefDict)) == {
|
||||
"datastore",
|
||||
"namespace",
|
||||
"group_type",
|
||||
"group_id",
|
||||
"time",
|
||||
}
|
||||
|
||||
def test_job_kinds_include_garbage_collection(self):
|
||||
"""GC is a job with a schedule and a last run like the others, so it is
|
||||
listed with them rather than restated on the datastore."""
|
||||
assert set(get_args(BackupJobKind)) == {"gc", "verify", "prune", "sync", "tape"}
|
||||
assert set(BACKUP_JOB_KINDS) == set(get_args(BackupJobKind))
|
||||
|
||||
def test_job(self):
|
||||
assert set(get_type_hints(BackupServerJobDict)) == {
|
||||
"id",
|
||||
"kind",
|
||||
"datastore",
|
||||
"namespace",
|
||||
"schedule",
|
||||
"enabled",
|
||||
"last_run",
|
||||
"last_state",
|
||||
"last_error",
|
||||
"next_run",
|
||||
"remote",
|
||||
"direction",
|
||||
"comment",
|
||||
}
|
||||
assert set(get_args(BackupRunState)) == {"ok", "warning", "error", "running"}
|
||||
|
||||
def test_remote_never_carries_its_secret(self):
|
||||
fields = set(get_type_hints(BackupRemoteDict))
|
||||
assert fields == {"name", "host", "port", "fingerprint", "auth_id", "comment"}
|
||||
assert not fields & {"password", "secret", "token"}
|
||||
|
||||
def test_task(self):
|
||||
assert set(get_type_hints(BackupTaskDict)) == {"task"}
|
||||
assert set(get_type_hints(BackupTaskStatusDict)) == {
|
||||
"task",
|
||||
"running",
|
||||
"state",
|
||||
"error",
|
||||
"log",
|
||||
}
|
||||
|
||||
|
||||
class TestClusterStorageNamesItsBackupServer:
|
||||
"""A hypervisor's backup storage says which server and datastore it points
|
||||
at, so an application can tie the server's groups to the cluster's guests."""
|
||||
|
||||
def test_backup_server_reference_is_optional_on_a_storage(self):
|
||||
hints = get_type_hints(ClusterStorageDict, include_extras=True)
|
||||
assert hints["backup_server"] == NotRequired[BackupServerRefDict | None]
|
||||
|
||||
def test_reference(self):
|
||||
assert set(get_type_hints(BackupServerRefDict)) == {
|
||||
"server",
|
||||
"port",
|
||||
"datastore",
|
||||
"namespace",
|
||||
"fingerprint",
|
||||
}
|
||||
@@ -0,0 +1,173 @@
|
||||
"""A hypervisor says which cluster it belongs to, and what that cluster holds.
|
||||
|
||||
A Proxmox node is one member of a cluster and can read the whole of it; a vCenter
|
||||
manages several clusters and standalone hosts at once; a standalone host is a
|
||||
cluster of one. One getter serves all three: ``get_clusters()`` lists every
|
||||
cluster the connection can see, each with its nodes, its storages and its
|
||||
networks, and which nodes see which. What a caller does with that (placement,
|
||||
headroom, identity) is the caller's.
|
||||
|
||||
A new VM can be placed on a named node (``create_vm_from_cloud_init(node=…)``),
|
||||
so a manager that runs several hosts takes the caller's choice instead of making
|
||||
its own.
|
||||
|
||||
The declarations sit under ``TYPE_CHECKING`` (see test_role_contracts), so the
|
||||
signatures are read from the source rather than from the class.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import ast
|
||||
import inspect
|
||||
from typing import NotRequired, get_type_hints
|
||||
|
||||
import napalm_device_types.hypervisor as hypervisor_module
|
||||
from napalm_device_types import (
|
||||
BackupDict,
|
||||
BackupJobDict,
|
||||
BackupServerRefDict,
|
||||
ClusterDict,
|
||||
ClusterNetworkDict,
|
||||
ClusterNodeDict,
|
||||
ClusterStorageDict,
|
||||
HAResourceDict,
|
||||
HypervisorDriver,
|
||||
)
|
||||
|
||||
|
||||
def _declared(name: str) -> ast.FunctionDef:
|
||||
tree = ast.parse(inspect.getsource(hypervisor_module))
|
||||
for node in ast.walk(tree):
|
||||
if isinstance(node, ast.FunctionDef) and node.name == name:
|
||||
return node
|
||||
raise AssertionError(f"HypervisorDriver does not declare {name}()")
|
||||
|
||||
|
||||
class TestClusterListing:
|
||||
def test_is_declared(self):
|
||||
fn = _declared("get_clusters")
|
||||
assert ast.unparse(fn.returns) == "list[ClusterDict]"
|
||||
|
||||
def test_absent_until_a_driver_implements_it(self):
|
||||
"""netOrk probes capabilities with hasattr; a hypervisor that cannot
|
||||
describe its cluster must not seem to."""
|
||||
assert not hasattr(HypervisorDriver, "get_clusters")
|
||||
|
||||
|
||||
class TestClusterShape:
|
||||
def test_cluster(self):
|
||||
assert get_type_hints(ClusterDict, include_extras=True) == {
|
||||
"id": str | None,
|
||||
"name": str | None,
|
||||
"quorate": bool | None,
|
||||
"nodes": list[ClusterNodeDict],
|
||||
"storages": list[ClusterStorageDict],
|
||||
"networks": list[ClusterNetworkDict],
|
||||
"ha_resources": NotRequired[list[HAResourceDict] | None],
|
||||
"backup_jobs": NotRequired[list[BackupJobDict] | None],
|
||||
"backups": NotRequired[list[BackupDict] | None],
|
||||
}
|
||||
|
||||
def test_cluster_keys_added_later_stay_optional(self):
|
||||
"""A driver written before HA and backups were part of the report stays valid."""
|
||||
assert ClusterDict.__required_keys__ == frozenset(
|
||||
{"id", "name", "quorate", "nodes", "storages", "networks"}
|
||||
)
|
||||
|
||||
def test_node(self):
|
||||
hints = get_type_hints(ClusterNodeDict, include_extras=True)
|
||||
assert hints == {
|
||||
"name": str,
|
||||
"status": str,
|
||||
"local": bool,
|
||||
"maintenance": bool | None,
|
||||
"cpu_threads": int | None,
|
||||
"cpu_cores": int | None,
|
||||
"cpu_model": str | None,
|
||||
"cpu_flags": list[str] | None,
|
||||
"cpu_usage": float | None,
|
||||
"memory_total": int | None,
|
||||
"memory_used": int | None,
|
||||
"votes": NotRequired[int | None],
|
||||
}
|
||||
assert "votes" not in ClusterNodeDict.__required_keys__
|
||||
|
||||
def test_storage(self):
|
||||
hints = get_type_hints(ClusterStorageDict, include_extras=True)
|
||||
assert hints == {
|
||||
"name": str,
|
||||
"type": str,
|
||||
"shared": bool,
|
||||
"content": list[str],
|
||||
"nodes": list[str],
|
||||
"total": int | None,
|
||||
"used": int | None,
|
||||
"backup_server": NotRequired[BackupServerRefDict | None],
|
||||
}
|
||||
# A driver written before backup servers were part of the report stays valid.
|
||||
assert "backup_server" not in ClusterStorageDict.__required_keys__
|
||||
|
||||
def test_network(self):
|
||||
hints = get_type_hints(ClusterNetworkDict, include_extras=True)
|
||||
assert hints == {
|
||||
"name": str,
|
||||
"kind": str,
|
||||
"vlan_aware": bool,
|
||||
"fixed_vlan_tag": NotRequired[int | None],
|
||||
"nodes": list[str],
|
||||
}
|
||||
assert ClusterNetworkDict.__required_keys__ == frozenset(
|
||||
{"name", "kind", "vlan_aware", "nodes"}
|
||||
)
|
||||
|
||||
|
||||
class TestHAAndBackupShapes:
|
||||
def test_ha_resource(self):
|
||||
hints = get_type_hints(HAResourceDict, include_extras=True)
|
||||
assert hints == {
|
||||
"vmid": int,
|
||||
"kind": str,
|
||||
"requested": str,
|
||||
"status": str | None,
|
||||
"node": str | None,
|
||||
"group": NotRequired[str | None],
|
||||
"rules": list[str],
|
||||
}
|
||||
assert HAResourceDict.__required_keys__ == frozenset(
|
||||
{"vmid", "kind", "requested", "status", "node", "rules"}
|
||||
)
|
||||
|
||||
def test_backup_job(self):
|
||||
assert get_type_hints(BackupJobDict) == {
|
||||
"id": str,
|
||||
"enabled": bool,
|
||||
"schedule": str | None,
|
||||
"storage": str | None,
|
||||
"mode": str | None,
|
||||
"all": bool,
|
||||
"vmids": list[int],
|
||||
"exclude": list[int],
|
||||
"pool": str | None,
|
||||
"node": str | None,
|
||||
}
|
||||
|
||||
def test_backup(self):
|
||||
assert get_type_hints(BackupDict) == {
|
||||
"vmid": int,
|
||||
"storage": str,
|
||||
"time": int,
|
||||
"size": int | None,
|
||||
"count": int,
|
||||
}
|
||||
|
||||
|
||||
class TestCreateVmTakesANode:
|
||||
def test_node_is_an_optional_keyword(self):
|
||||
fn = _declared("create_vm_from_cloud_init")
|
||||
kwonly = {
|
||||
arg.arg: default
|
||||
for arg, default in zip(fn.args.kwonlyargs, fn.args.kw_defaults, strict=True)
|
||||
}
|
||||
assert "node" in kwonly
|
||||
default = kwonly["node"]
|
||||
assert isinstance(default, ast.Constant) and default.value is None
|
||||
@@ -0,0 +1,141 @@
|
||||
"""Cron expressions: what they mean, when they fire next, and as systemd ``OnCalendar``.
|
||||
|
||||
One module answers all three, because a job netOrk writes as a timer must fire
|
||||
exactly when the cron expression the user gave says, and the preview must show
|
||||
the same runs the host will make (NetOrk/netork#1933).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import UTC, datetime
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types.cron import is_reboot, next_runs, parse_cron, to_on_calendar
|
||||
|
||||
BERLIN = ZoneInfo("Europe/Berlin")
|
||||
|
||||
|
||||
class TestParse:
|
||||
def test_every_field_is_expanded_to_its_values(self):
|
||||
s = parse_cron("*/15 6-8 1,15 * *")
|
||||
assert s.minutes == frozenset({0, 15, 30, 45})
|
||||
assert s.hours == frozenset({6, 7, 8})
|
||||
assert s.days == frozenset({1, 15})
|
||||
assert s.months == frozenset(range(1, 13))
|
||||
assert s.weekdays == frozenset(range(7))
|
||||
|
||||
def test_a_range_takes_a_step(self):
|
||||
assert parse_cron("1-30/10 * * * *").minutes == frozenset({1, 11, 21})
|
||||
|
||||
def test_names_are_months_and_weekdays_in_any_case(self):
|
||||
s = parse_cron("0 0 * Jan,MAR mon-Fri")
|
||||
assert s.months == frozenset({1, 3})
|
||||
assert s.weekdays == frozenset({1, 2, 3, 4, 5})
|
||||
|
||||
def test_seven_is_sunday_too(self):
|
||||
assert parse_cron("0 0 * * 7").weekdays == frozenset({0})
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("alias", "expanded"),
|
||||
[
|
||||
("@hourly", "0 * * * *"),
|
||||
("@daily", "0 0 * * *"),
|
||||
("@midnight", "0 0 * * *"),
|
||||
("@weekly", "0 0 * * 0"),
|
||||
("@monthly", "0 0 1 * *"),
|
||||
("@yearly", "0 0 1 1 *"),
|
||||
("@annually", "0 0 1 1 *"),
|
||||
],
|
||||
)
|
||||
def test_an_alias_is_its_expression(self, alias, expanded):
|
||||
assert parse_cron(alias) == parse_cron(expanded)
|
||||
|
||||
def test_reboot_is_no_calendar(self):
|
||||
assert is_reboot("@reboot")
|
||||
assert not is_reboot("0 0 * * *")
|
||||
with pytest.raises(ValueError, match="reboot"):
|
||||
parse_cron("@reboot")
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"bad",
|
||||
[
|
||||
"",
|
||||
"* * * *",
|
||||
"* * * * * *",
|
||||
"60 * * * *",
|
||||
"* 24 * * *",
|
||||
"* * 0 * *",
|
||||
"* * * 13 *",
|
||||
"* * * * 8",
|
||||
"*/0 * * * *",
|
||||
"5-1 * * * *",
|
||||
"a * * * *",
|
||||
"@often",
|
||||
"1,,2 * * * *",
|
||||
],
|
||||
)
|
||||
def test_what_cron_would_not_take_is_refused(self, bad):
|
||||
with pytest.raises(ValueError):
|
||||
parse_cron(bad)
|
||||
|
||||
def test_day_of_month_and_weekday_are_or_only_when_both_are_restricted(self):
|
||||
assert parse_cron("0 0 1 * 1").either_day
|
||||
assert not parse_cron("0 0 */2 * 1").either_day
|
||||
assert not parse_cron("0 0 1 * *").either_day
|
||||
|
||||
|
||||
class TestNextRuns:
|
||||
def test_runs_after_the_instant_in_the_hosts_time(self):
|
||||
after = datetime(2026, 11, 3, 5, 50, tzinfo=UTC) # 06:50 in Berlin
|
||||
runs = next_runs("0 7 * * *", after, BERLIN, count=2)
|
||||
assert runs == [
|
||||
datetime(2026, 11, 3, 7, 0, tzinfo=BERLIN),
|
||||
datetime(2026, 11, 4, 7, 0, tzinfo=BERLIN),
|
||||
]
|
||||
|
||||
def test_the_instant_itself_is_not_a_next_run(self):
|
||||
after = datetime(2026, 11, 3, 6, 0, tzinfo=UTC)
|
||||
assert next_runs("0 6 * * *", after, ZoneInfo("UTC"), count=1)[0].day == 4
|
||||
|
||||
def test_either_day_fires_on_both(self):
|
||||
after = datetime(2026, 11, 30, 12, 0, tzinfo=UTC) # a Monday
|
||||
runs = next_runs("0 0 1 * 1", after, ZoneInfo("UTC"), count=3)
|
||||
assert [r.day for r in runs] == [1, 7, 14]
|
||||
|
||||
def test_a_day_that_never_comes_gives_no_runs(self):
|
||||
assert next_runs("0 0 31 2 *", datetime(2026, 1, 1, tzinfo=UTC), BERLIN) == []
|
||||
|
||||
def test_a_time_the_clock_skips_is_left_out(self):
|
||||
# 2026-03-29 02:30 does not exist in Berlin.
|
||||
after = datetime(2026, 3, 28, 12, 0, tzinfo=UTC)
|
||||
runs = next_runs("30 2 * * *", after, BERLIN, count=2)
|
||||
assert [r.day for r in runs] == [30, 31]
|
||||
|
||||
def test_reboot_has_no_next_run(self):
|
||||
assert next_runs("@reboot", datetime(2026, 1, 1, tzinfo=UTC), BERLIN) == []
|
||||
|
||||
|
||||
class TestOnCalendar:
|
||||
def test_every_field_as_its_values(self):
|
||||
assert to_on_calendar("*/15 6 * * *") == ["*-*-* 06:00,15,30,45:00"]
|
||||
|
||||
def test_a_restricted_weekday_leads(self):
|
||||
assert to_on_calendar("30 4 * * 1-5") == ["Mon,Tue,Wed,Thu,Fri *-*-* 04:30:00"]
|
||||
|
||||
def test_months_and_days(self):
|
||||
assert to_on_calendar("0 0 1,15 1,7 *") == ["*-01,07-01,15 00:00:00"]
|
||||
|
||||
def test_either_day_is_two_lines(self):
|
||||
assert to_on_calendar("0 0 1 * 0") == ["*-*-01 00:00:00", "Sun *-*-* 00:00:00"]
|
||||
|
||||
def test_both_days_with_a_star_are_one_line(self):
|
||||
assert to_on_calendar("0 0 */10 * 6") == ["Sat *-*-01,11,21,31 00:00:00"]
|
||||
|
||||
def test_an_alias(self):
|
||||
assert to_on_calendar("@daily") == ["*-*-* 00:00:00"]
|
||||
|
||||
def test_reboot_is_refused(self):
|
||||
with pytest.raises(ValueError, match="reboot"):
|
||||
to_on_calendar("@reboot")
|
||||
@@ -0,0 +1,145 @@
|
||||
"""Crontabs as text: the jobs in them, and changing one line (NetOrk/netork#1933)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types._crontab import (
|
||||
DISABLED_MARKER,
|
||||
marked_lines,
|
||||
parse_anacrontab,
|
||||
parse_crontab,
|
||||
run_script,
|
||||
switched,
|
||||
with_job,
|
||||
without_job,
|
||||
)
|
||||
|
||||
SYSTEM = """\
|
||||
# /etc/crontab: system-wide crontab
|
||||
SHELL=/bin/sh
|
||||
PATH="/usr/local/sbin:/usr/bin:/bin"
|
||||
17 * * * * root cd / && run-parts --report /etc/cron.hourly
|
||||
@reboot root /usr/local/bin/warm-cache # keeps the # in the command
|
||||
#napalm-disabled: 0 4 * * * backup /usr/local/bin/backup
|
||||
garbage line
|
||||
0 4 * * * backup /usr/local/bin/backup
|
||||
"""
|
||||
|
||||
USER = """\
|
||||
MAILTO=ops@example.org
|
||||
*/5 * * * * /usr/bin/php /var/www/cron.php
|
||||
# napalm-scheduled-job: netork-cleanup tidy the cache
|
||||
0 3 * * * find /tmp -mtime +7 -delete
|
||||
"""
|
||||
|
||||
|
||||
class TestParse:
|
||||
def test_a_system_crontab_names_the_user(self):
|
||||
jobs = parse_crontab("/etc/crontab", SYSTEM, system=True)
|
||||
first = jobs[0]
|
||||
assert (first.schedule, first.run_as) == ("17 * * * *", "root")
|
||||
assert first.command == "cd / && run-parts --report /etc/cron.hourly"
|
||||
assert first.env == {"SHELL": "/bin/sh", "PATH": "/usr/local/sbin:/usr/bin:/bin"}
|
||||
assert first.key.startswith("cron:/etc/crontab:")
|
||||
|
||||
def test_an_alias_and_a_hash_inside_the_command(self):
|
||||
reboot = parse_crontab("/etc/crontab", SYSTEM, system=True)[1]
|
||||
assert reboot.schedule == "@reboot"
|
||||
assert reboot.command == "/usr/local/bin/warm-cache # keeps the # in the command"
|
||||
|
||||
def test_a_line_switched_off_is_a_job_that_is_off(self):
|
||||
jobs = parse_crontab("/etc/crontab", SYSTEM, system=True)
|
||||
off = jobs[2]
|
||||
assert not off.enabled
|
||||
assert off.run_as == "backup"
|
||||
|
||||
def test_equal_lines_get_their_own_keys_and_switching_off_keeps_the_key(self):
|
||||
jobs = parse_crontab("/etc/crontab", SYSTEM, system=True)
|
||||
assert jobs[3].key == jobs[2].key + ":2"
|
||||
assert len(jobs) == 4 # the garbage line is left out
|
||||
|
||||
def test_a_user_crontab_runs_as_its_user(self):
|
||||
jobs = parse_crontab("/var/spool/cron/crontabs/www", USER, system=False, user="www-data")
|
||||
assert jobs[0].run_as == "www-data"
|
||||
assert jobs[0].env == {"MAILTO": "ops@example.org"}
|
||||
|
||||
def test_an_applications_job_is_keyed_by_its_name(self):
|
||||
job = parse_crontab("/x", USER, system=False, user="root")[1]
|
||||
assert (job.key, job.name, job.description) == (
|
||||
"job:netork-cleanup",
|
||||
"netork-cleanup",
|
||||
"tidy the cache",
|
||||
)
|
||||
|
||||
def test_an_anacrontab(self):
|
||||
text = "SHELL=/bin/sh\n1\t5\tcron.daily\trun-parts --report /etc/cron.daily\n@monthly 15 cron.monthly run-parts /etc/cron.monthly\nbad line here\n"
|
||||
jobs = parse_anacrontab(text)
|
||||
assert [(j.key, j.schedule, j.run_as) for j in jobs] == [
|
||||
("anacron:cron.daily", "1", "root"),
|
||||
("anacron:cron.monthly", "@monthly", "root"),
|
||||
]
|
||||
assert jobs[0].command == "run-parts --report /etc/cron.daily"
|
||||
|
||||
|
||||
class TestEdit:
|
||||
def test_switching_off_and_on_again_gives_the_text_back(self):
|
||||
jobs = parse_crontab("/etc/crontab", SYSTEM, system=True)
|
||||
off = switched(SYSTEM, jobs[0], enabled=False)
|
||||
assert DISABLED_MARKER + "17 *\t* * *\troot" in off
|
||||
again = parse_crontab("/etc/crontab", off, system=True)
|
||||
assert again[0].key == jobs[0].key and not again[0].enabled
|
||||
assert switched(off, again[0], enabled=True) == SYSTEM.replace(
|
||||
"17 *\t* * *\troot\tcd", "17 *\t* * *\troot\tcd"
|
||||
)
|
||||
|
||||
def test_an_applications_job_is_added_replaced_and_removed(self):
|
||||
lines = marked_lines("netork-x", "a job", "0 1 * * * /bin/true", enabled=True)
|
||||
added = with_job("A=1\n", [], "netork-x", lines)
|
||||
assert added == "A=1\n# napalm-scheduled-job: netork-x a job\n0 1 * * * /bin/true\n"
|
||||
jobs = parse_crontab("/x", added, system=False, user="root")
|
||||
changed = with_job(
|
||||
added,
|
||||
jobs,
|
||||
"netork-x",
|
||||
marked_lines("netork-x", "", "0 2 * * * /bin/true", enabled=False),
|
||||
)
|
||||
assert (
|
||||
changed
|
||||
== "A=1\n# napalm-scheduled-job: netork-x\n#napalm-disabled: 0 2 * * * /bin/true\n"
|
||||
)
|
||||
jobs = parse_crontab("/x", changed, system=False, user="root")
|
||||
assert without_job(changed, jobs, "netork-x") == "A=1\n"
|
||||
|
||||
def test_adding_to_a_text_without_a_last_newline(self):
|
||||
assert with_job("A=1", [], "n", ["# m", "x"]) == "A=1\n# m\nx\n"
|
||||
|
||||
def test_removing_the_only_job_leaves_nothing(self):
|
||||
text = "# napalm-scheduled-job: n\n0 1 * * * x\n"
|
||||
jobs = parse_crontab("/x", text, system=False, user="root")
|
||||
assert without_job(text, jobs, "n") == ""
|
||||
|
||||
|
||||
class TestRunScript:
|
||||
def _run(self, script: str) -> str:
|
||||
return subprocess.run(
|
||||
["sh", "-c", script], capture_output=True, text=True, check=True
|
||||
).stdout
|
||||
|
||||
def test_the_crontabs_variables_are_set(self):
|
||||
assert (
|
||||
self._run(run_script('echo "$GREETING"', {"GREETING": "hi there", "MAILTO": "x"}))
|
||||
== "hi there\n"
|
||||
)
|
||||
|
||||
def test_a_percent_starts_stdin_and_further_ones_are_newlines(self):
|
||||
assert self._run(run_script("cat%one%two", {})) == "one\ntwo\n"
|
||||
|
||||
def test_an_escaped_percent_is_a_percent(self):
|
||||
assert self._run(run_script("date -d @0 -u +\\%Y", {})) == "1970\n"
|
||||
|
||||
@pytest.mark.parametrize("command", ["echo plain", "echo 'quoted'"])
|
||||
def test_without_percent_the_command_is_as_it_is(self, command):
|
||||
assert run_script(command, {}) == command
|
||||
@@ -30,8 +30,7 @@ def test_a_missing_license_may_say_what_the_device_withholds_without_it():
|
||||
# A Supermicro X10 with only SFT-OOB-LIC answers Redfish but keeps its drives and
|
||||
# firmware inventory until DCMS is activated: partly licensed, not unlicensed.
|
||||
hints = get_type_hints(LicenseDict)
|
||||
# Python 3.10 keeps NotRequired[...] around the type; later versions strip it.
|
||||
assert "list[str]" in str(hints["withholds"])
|
||||
assert hints["withholds"] == list[str]
|
||||
assert "withholds" not in LicenseDict.__required_keys__
|
||||
|
||||
|
||||
|
||||
@@ -11,9 +11,11 @@ import pytest
|
||||
|
||||
from napalm_device_types import (
|
||||
AccessPointDriver,
|
||||
BackupServerDriver,
|
||||
EnsurePackageMixin,
|
||||
FirewallDriver,
|
||||
HypervisorDriver,
|
||||
LldpAgentMixin,
|
||||
ManagementControllerDriver,
|
||||
MediaDriver,
|
||||
OSDriver,
|
||||
@@ -27,7 +29,7 @@ from napalm_device_types import (
|
||||
)
|
||||
|
||||
ROLE_BASES = [
|
||||
AccessPointDriver, FirewallDriver, HypervisorDriver, ManagementControllerDriver, MediaDriver,
|
||||
AccessPointDriver, BackupServerDriver, FirewallDriver, HypervisorDriver, ManagementControllerDriver, MediaDriver,
|
||||
OSDriver, PhoneDriver, ResidentialGatewayDriver, StorageDriver, SwitchDriver,
|
||||
] # fmt: skip
|
||||
|
||||
@@ -169,11 +171,91 @@ class TestEnsurePackage:
|
||||
assert "no candidate" in result["output"]
|
||||
|
||||
|
||||
class _Lldp(LldpAgentMixin):
|
||||
"""The generic steps over ensure_package and manage_service; the driver's configure."""
|
||||
|
||||
def __init__(self, refuse: str = ""):
|
||||
self.calls: list = []
|
||||
self.refuse = refuse
|
||||
|
||||
def ensure_package(self, name):
|
||||
self.calls.append(("ensure_package", name))
|
||||
ok = self.refuse != "install"
|
||||
return {"success": ok, "steps": [], "output": "installed" if ok else "E: no candidate"}
|
||||
|
||||
def _write_lldp_agent_config(self, interfaces):
|
||||
self.calls.append(("configure", interfaces))
|
||||
return {"ok": self.refuse != "configure", "output": "interfaces set"}
|
||||
|
||||
def manage_service(self, name, action):
|
||||
self.calls.append((action, name))
|
||||
ok = self.refuse != action
|
||||
return {"success": ok, "output": "" if ok else f"cannot {action}"}
|
||||
|
||||
|
||||
class TestLldpAgent:
|
||||
def test_install_configure_enable_and_restart_on_the_caller_s_interfaces(self):
|
||||
agent = _Lldp()
|
||||
|
||||
result = agent.configure_lldp_agent(["br-lan"])
|
||||
|
||||
assert result["success"] is True
|
||||
assert agent.calls == [
|
||||
("ensure_package", "lldpd"),
|
||||
("configure", ["br-lan"]),
|
||||
("enable", "lldpd"),
|
||||
("restart", "lldpd"),
|
||||
]
|
||||
assert [s["step"] for s in result["steps"]] == ["install", "configure", "start"]
|
||||
|
||||
def test_a_failed_install_configures_nothing(self):
|
||||
agent = _Lldp(refuse="install")
|
||||
|
||||
result = agent.configure_lldp_agent(["br-lan"])
|
||||
|
||||
assert result["success"] is False
|
||||
assert agent.calls == [("ensure_package", "lldpd")]
|
||||
assert "no candidate" in result["output"]
|
||||
|
||||
def test_an_agent_that_will_not_start_at_boot_is_not_restarted(self):
|
||||
agent = _Lldp(refuse="enable")
|
||||
|
||||
result = agent.configure_lldp_agent(["br-lan"])
|
||||
|
||||
assert result["success"] is False
|
||||
assert ("restart", "lldpd") not in agent.calls
|
||||
assert "cannot enable" in result["output"]
|
||||
|
||||
def test_a_failed_restart_fails_the_fix(self):
|
||||
result = _Lldp(refuse="restart").configure_lldp_agent(["br-lan"])
|
||||
|
||||
assert result["success"] is False
|
||||
assert result["steps"][-1]["step"] == "start"
|
||||
|
||||
def test_the_package_and_service_names_are_the_driver_s(self):
|
||||
agent = _Lldp()
|
||||
agent.LLDP_AGENT_PACKAGE = "lldpd-full"
|
||||
agent.LLDP_AGENT_SERVICE = "lldpd.service"
|
||||
|
||||
agent.configure_lldp_agent(["eth0"])
|
||||
|
||||
assert ("ensure_package", "lldpd-full") in agent.calls
|
||||
assert ("restart", "lldpd.service") in agent.calls
|
||||
|
||||
def test_no_interface_is_refused_before_anything_changes(self):
|
||||
agent = _Lldp()
|
||||
|
||||
with pytest.raises(ValueError):
|
||||
agent.configure_lldp_agent([])
|
||||
assert agent.calls == []
|
||||
|
||||
|
||||
@pytest.mark.parametrize("base", ROLE_BASES, ids=lambda b: b.__name__)
|
||||
@pytest.mark.parametrize(
|
||||
"method",
|
||||
[
|
||||
"configure_snmp_agent",
|
||||
"configure_lldp_agent",
|
||||
"ensure_package",
|
||||
"configure_package_proxy",
|
||||
"grant_container_engine_access",
|
||||
|
||||
@@ -0,0 +1,27 @@
|
||||
"""shutdown_host: the contract for shutting the device itself down cleanly.
|
||||
|
||||
netOrk's graph workflows switch a hypervisor off when it has nothing to do
|
||||
(NetOrk/netork#1636): through its own API, never by pulling the power. Like
|
||||
``reboot_host`` it is declared, not implemented, so ``hasattr`` tells netOrk
|
||||
whether a driver can.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import inspect
|
||||
|
||||
from napalm_device_types import DeviceTypeDriver, HostShutdownMixin
|
||||
|
||||
|
||||
def test_every_device_type_driver_carries_the_declaration():
|
||||
assert issubclass(DeviceTypeDriver, HostShutdownMixin)
|
||||
|
||||
|
||||
def test_declared_not_implemented():
|
||||
assert not hasattr(DeviceTypeDriver, "shutdown_host")
|
||||
|
||||
|
||||
def test_declaration_documents_the_contract():
|
||||
source = inspect.getsource(HostShutdownMixin)
|
||||
assert "def shutdown_host(self) -> None" in source
|
||||
assert "RuntimeError" in source
|
||||
@@ -0,0 +1,127 @@
|
||||
"""LLDP rows projected into NAPALM's ``get_lldp_neighbors`` and ``get_lldp_neighbors_detail``.
|
||||
|
||||
A driver reads its neighbours into :class:`LldpNeighborRow` (from ``lldpctl``,
|
||||
an API, a CLI) and the two NAPALM shapes come from those rows the same way for
|
||||
every driver (NAPALM/napalm-device-types#10).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import get_type_hints
|
||||
|
||||
from napalm_device_types import LldpNeighborRow, lldp_neighbors, lldp_neighbors_detail
|
||||
|
||||
|
||||
def _row(local_port: str, name: str, **fields: object) -> LldpNeighborRow:
|
||||
row: LldpNeighborRow = {
|
||||
"local_port": local_port,
|
||||
"chassis_id": "",
|
||||
"chassis_id_type": "",
|
||||
"port_id": "",
|
||||
"port_id_type": "",
|
||||
"port_description": "",
|
||||
"system_name": name,
|
||||
"system_description": "",
|
||||
"management_addresses": [],
|
||||
"capabilities": [],
|
||||
"enabled_capabilities": [],
|
||||
}
|
||||
row.update(fields) # type: ignore[typeddict-item]
|
||||
return row
|
||||
|
||||
|
||||
PHONE = _row(
|
||||
"eth0.1",
|
||||
"desk-phone",
|
||||
chassis_id="00:11:22:33:44:AB",
|
||||
chassis_id_type="mac",
|
||||
port_id="00:11:22:33:44:01",
|
||||
port_id_type="mac",
|
||||
port_description="WAN",
|
||||
system_description="SIP phone",
|
||||
capabilities=["bridge", "telephone"],
|
||||
enabled_capabilities=["telephone"],
|
||||
)
|
||||
PC = _row("eth0.1", "desk-pc", port_id="eno1", port_id_type="ifname")
|
||||
SWITCH = _row("eth1", "access-sw", port_id="gi1/0/7", port_id_type="local")
|
||||
|
||||
|
||||
def test_a_row_names_its_fields():
|
||||
assert set(get_type_hints(LldpNeighborRow)) == {
|
||||
"local_port",
|
||||
"chassis_id",
|
||||
"chassis_id_type",
|
||||
"port_id",
|
||||
"port_id_type",
|
||||
"port_description",
|
||||
"system_name",
|
||||
"system_description",
|
||||
"management_addresses",
|
||||
"capabilities",
|
||||
"enabled_capabilities",
|
||||
}
|
||||
|
||||
|
||||
class TestNeighbors:
|
||||
def test_every_neighbour_of_a_port_is_listed(self):
|
||||
assert lldp_neighbors([PHONE, PC, SWITCH]) == {
|
||||
"eth0.1": [
|
||||
{"hostname": "desk-phone", "port": "00:11:22:33:44:01", "mac": "00:11:22:33:44:ab"},
|
||||
{"hostname": "desk-pc", "port": "eno1"},
|
||||
],
|
||||
"eth1": [{"hostname": "access-sw", "port": "gi1/0/7"}],
|
||||
}
|
||||
|
||||
def test_no_rows_is_no_neighbour(self):
|
||||
assert lldp_neighbors([]) == {}
|
||||
|
||||
def test_a_chassis_mac_is_reported_in_lower_case(self):
|
||||
# netOrk matches a neighbour by this MAC first (NAPALM/napalm-openwrt#31).
|
||||
assert lldp_neighbors([PHONE])["eth0.1"][0]["mac"] == "00:11:22:33:44:ab"
|
||||
|
||||
def test_a_chassis_named_otherwise_has_no_mac(self):
|
||||
switch = _row("eth1", "sw", chassis_id="00:11:22:33:44:55", chassis_id_type="local")
|
||||
|
||||
assert "mac" not in lldp_neighbors([switch])["eth1"][0]
|
||||
|
||||
def test_a_mac_id_that_is_no_mac_address_is_left_out(self):
|
||||
odd = _row("eth1", "sw", chassis_id="00:11", chassis_id_type="mac")
|
||||
|
||||
assert "mac" not in lldp_neighbors([odd])["eth1"][0]
|
||||
|
||||
|
||||
class TestNeighborsDetail:
|
||||
def test_every_field_napalm_names(self):
|
||||
assert lldp_neighbors_detail([PHONE]) == {
|
||||
"eth0.1": [
|
||||
{
|
||||
"parent_interface": "",
|
||||
"remote_port": "00:11:22:33:44:01",
|
||||
"remote_port_description": "WAN",
|
||||
"remote_chassis_id": "00:11:22:33:44:AB",
|
||||
"remote_system_name": "desk-phone",
|
||||
"remote_system_description": "SIP phone",
|
||||
"remote_system_capab": ["bridge", "telephone"],
|
||||
"remote_system_enable_capab": ["telephone"],
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
def test_every_neighbour_of_a_port_is_listed(self):
|
||||
detail = lldp_neighbors_detail([PHONE, PC, SWITCH])
|
||||
|
||||
assert [n["remote_system_name"] for n in detail["eth0.1"]] == ["desk-phone", "desk-pc"]
|
||||
assert [n["remote_system_name"] for n in detail["eth1"]] == ["access-sw"]
|
||||
|
||||
def test_one_interface_only(self):
|
||||
assert list(lldp_neighbors_detail([PHONE, PC, SWITCH], "eth1")) == ["eth1"]
|
||||
|
||||
def test_an_interface_without_neighbours_is_empty(self):
|
||||
assert lldp_neighbors_detail([PHONE], "eth9") == {}
|
||||
|
||||
def test_the_lists_are_the_rows_own_copies(self):
|
||||
detail = lldp_neighbors_detail([PHONE])
|
||||
|
||||
detail["eth0.1"][0]["remote_system_capab"].append("router")
|
||||
|
||||
assert PHONE["capabilities"] == ["bridge", "telephone"]
|
||||
@@ -0,0 +1,308 @@
|
||||
"""``lldpctl -f keyvalue``: one row per LLDP neighbour, for every driver that reads lldpd.
|
||||
|
||||
napalm-linux, napalm-openwrt and napalm-proxmox each parsed this output on their
|
||||
own, and each copy grouped the lines by interface, so two neighbours on one port
|
||||
merged (NAPALM/napalm-openwrt#9, NAPALM/napalm-linux#58), split ``eth0.1`` at its
|
||||
dot (NAPALM/napalm-openwrt#26) and looked for ``chassis.id.value``, a key lldpd
|
||||
never prints. The parse lives here once now.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import LLDPCTL_KEYVALUE_COMMAND, parse_lldpctl_keyvalue
|
||||
|
||||
# The recording napalm-openwrt's tests have used since the driver was written
|
||||
# (tests/unit/test_driver.py, ``LLDPCTL_KV``): one neighbour on eth0. Its two
|
||||
# ``chassis.cap.*`` lines are not lldpd's format (lldpd prints one
|
||||
# ``chassis.<Capability>.enabled=on|off`` line per capability), so they yield none.
|
||||
OPENWRT_ONE_NEIGHBOUR = """\
|
||||
lldp.eth0.via=LLDP
|
||||
lldp.eth0.rid=1
|
||||
lldp.eth0.age=0 day, 01:23:45
|
||||
lldp.eth0.chassis.mac=00:aa:bb:cc:dd:ee
|
||||
lldp.eth0.chassis.name=core-router
|
||||
lldp.eth0.chassis.descr=RouterOS 7.x
|
||||
lldp.eth0.chassis.mgmt-ip=10.0.0.1
|
||||
lldp.eth0.chassis.cap.available=Router, Bridge
|
||||
lldp.eth0.chassis.cap.enabled=Router
|
||||
lldp.eth0.port.ifname=ether1
|
||||
lldp.eth0.port.descr=uplink
|
||||
"""
|
||||
|
||||
# Laid out the way lldpd 1.0's keyvalue writer prints ``show neighbors details``
|
||||
# (src/client/display.c ``display_interface``, ``display_chassis``,
|
||||
# ``display_port``, ``display_vlans``, ``display_med``; src/client/kv_writer.c):
|
||||
# a neighbour opens with the local interface's optional ``alias`` and ``via``,
|
||||
# an id is printed under its type (``chassis.mac``, ``port.local``), a
|
||||
# capability as ``chassis.<Symbol>.enabled``, a management address once per
|
||||
# address. Not recorded from a device.
|
||||
#
|
||||
# On ``eth0.1`` (a VLAN interface, its name holds a dot) a phone and a PC behind
|
||||
# an unmanaged switch; on ``eth1`` a switch that names its port by a local id and
|
||||
# itself by an address.
|
||||
TWO_ON_ONE_DOTTED_PORT = """\
|
||||
lldp.eth0.1.alias=desk
|
||||
lldp.eth0.1.via=LLDP
|
||||
lldp.eth0.1.rid=1
|
||||
lldp.eth0.1.age=0 day, 00:10:00
|
||||
lldp.eth0.1.chassis.mac=00:11:22:33:44:01
|
||||
lldp.eth0.1.chassis.name=desk-phone
|
||||
lldp.eth0.1.chassis.descr=SIP phone, firmware a=1
|
||||
lldp.eth0.1.chassis.mgmt-ip=10.0.0.21
|
||||
lldp.eth0.1.chassis.mgmt-iface=2
|
||||
lldp.eth0.1.chassis.mgmt-ip=fe80::211:22ff:fe33:4401
|
||||
lldp.eth0.1.chassis.mgmt-iface=2
|
||||
lldp.eth0.1.chassis.Bridge.enabled=off
|
||||
lldp.eth0.1.chassis.Tel.enabled=on
|
||||
lldp.eth0.1.port.mac=00:11:22:33:44:01
|
||||
lldp.eth0.1.port.descr=WAN
|
||||
lldp.eth0.1.port.ttl=120
|
||||
lldp.eth0.1.port.auto-negotiation.supported=yes
|
||||
lldp.eth0.1.port.auto-negotiation.enabled=yes
|
||||
lldp.eth0.1.port.auto-negotiation.100Base-TX.hd=yes
|
||||
lldp.eth0.1.port.auto-negotiation.100Base-TX.fd=yes
|
||||
lldp.eth0.1.port.auto-negotiation.current=100BaseTXFD - 2 pair category 5 UTP, full duplex mode
|
||||
lldp.eth0.1.vlan.vlan-id=10
|
||||
lldp.eth0.1.vlan.pvid=yes
|
||||
lldp.eth0.1.vlan=voice
|
||||
lldp.eth0.1.lldp-med.device-type=Communication Device Endpoint (Class III)
|
||||
lldp.eth0.1.lldp-med.Capabilities.available=yes
|
||||
lldp.eth0.1.lldp-med.inventory.software=2.1.4
|
||||
lldp.eth0.1.alias=desk
|
||||
lldp.eth0.1.via=LLDP
|
||||
lldp.eth0.1.rid=2
|
||||
lldp.eth0.1.age=0 day, 00:09:00
|
||||
lldp.eth0.1.chassis.mac=00:11:22:33:44:02
|
||||
lldp.eth0.1.chassis.name=desk-pc
|
||||
lldp.eth0.1.chassis.descr=Debian GNU/Linux 12 (bookworm) Linux 6.1.0-25-amd64
|
||||
lldp.eth0.1.chassis.Bridge.enabled=off
|
||||
lldp.eth0.1.chassis.Router.enabled=off
|
||||
lldp.eth0.1.chassis.Wlan.enabled=off
|
||||
lldp.eth0.1.chassis.Station.enabled=on
|
||||
lldp.eth0.1.port.ifname=eno1
|
||||
lldp.eth0.1.port.ttl=120
|
||||
lldp.eth0.1.unknown-tlvs.unknown-tlv.oui=00,80,C2
|
||||
lldp.eth0.1.unknown-tlvs.unknown-tlv.subtype=9
|
||||
lldp.eth0.1.unknown-tlvs.unknown-tlv.len=1
|
||||
lldp.eth0.1.unknown-tlvs.unknown-tlv=01
|
||||
lldp.eth1.via=LLDP
|
||||
lldp.eth1.rid=3
|
||||
lldp.eth1.age=1 day, 02:00:00
|
||||
lldp.eth1.chassis.ip=192.0.2.7
|
||||
lldp.eth1.chassis.name=access-sw
|
||||
lldp.eth1.chassis.Bridge.enabled=on
|
||||
lldp.eth1.chassis.Router.enabled=off
|
||||
lldp.eth1.chassis.CVLAN.enabled=on
|
||||
lldp.eth1.port.local=gi1/0/7
|
||||
lldp.eth1.port.descr=GigabitEthernet1/0/7
|
||||
lldp.eth1.ppvid.supported=no
|
||||
lldp.eth1.ppvid.enabled=no
|
||||
lldp.eth1.pi=88cc
|
||||
"""
|
||||
|
||||
|
||||
def _rows(output: str = TWO_ON_ONE_DOTTED_PORT) -> list[dict[str, object]]:
|
||||
return [dict(row) for row in parse_lldpctl_keyvalue(output)]
|
||||
|
||||
|
||||
class TestOneRowPerNeighbour:
|
||||
def test_the_openwrt_recording_is_one_neighbour_with_its_fields(self):
|
||||
assert _rows(OPENWRT_ONE_NEIGHBOUR) == [
|
||||
{
|
||||
"local_port": "eth0",
|
||||
"chassis_id": "00:aa:bb:cc:dd:ee",
|
||||
"chassis_id_type": "mac",
|
||||
"port_id": "ether1",
|
||||
"port_id_type": "ifname",
|
||||
"port_description": "uplink",
|
||||
"system_name": "core-router",
|
||||
"system_description": "RouterOS 7.x",
|
||||
"management_addresses": ["10.0.0.1"],
|
||||
"capabilities": [],
|
||||
"enabled_capabilities": [],
|
||||
}
|
||||
]
|
||||
|
||||
def test_two_neighbours_on_one_port_stay_apart(self):
|
||||
rows = _rows()
|
||||
|
||||
assert [(r["local_port"], r["system_name"]) for r in rows] == [
|
||||
("eth0.1", "desk-phone"),
|
||||
("eth0.1", "desk-pc"),
|
||||
("eth1", "access-sw"),
|
||||
]
|
||||
|
||||
def test_a_neighbour_keeps_only_its_own_fields(self):
|
||||
pc = _rows()[1]
|
||||
|
||||
assert pc["chassis_id"] == "00:11:22:33:44:02"
|
||||
assert pc["port_description"] == ""
|
||||
assert pc["management_addresses"] == []
|
||||
|
||||
def test_neighbours_without_alias_are_split_at_via(self):
|
||||
output = OPENWRT_ONE_NEIGHBOUR + OPENWRT_ONE_NEIGHBOUR.replace("core-router", "edge")
|
||||
|
||||
assert [r["system_name"] for r in _rows(output)] == ["core-router", "edge"]
|
||||
|
||||
|
||||
class TestInterfaceName:
|
||||
def test_a_dotted_interface_name_survives(self):
|
||||
assert {r["local_port"] for r in _rows()} == {"eth0.1", "eth1"}
|
||||
|
||||
def test_an_interface_named_like_a_field_survives(self):
|
||||
output = (
|
||||
"lldp.vlan.100.via=LLDP\n"
|
||||
"lldp.vlan.100.chassis.mac=00:aa:bb:cc:dd:01\n"
|
||||
"lldp.vlan.100.port.ifname=port1\n"
|
||||
)
|
||||
|
||||
(row,) = _rows(output)
|
||||
|
||||
assert (row["local_port"], row["chassis_id"], row["port_id"]) == (
|
||||
"vlan.100",
|
||||
"00:aa:bb:cc:dd:01",
|
||||
"port1",
|
||||
)
|
||||
|
||||
def test_without_via_the_name_ends_at_the_first_field(self):
|
||||
output = "lldp.br-lan.10.chassis.name=sw\nlldp.br-lan.10.port.ifname=1\n"
|
||||
|
||||
(row,) = _rows(output)
|
||||
|
||||
assert (row["local_port"], row["system_name"], row["port_id"]) == ("br-lan.10", "sw", "1")
|
||||
|
||||
|
||||
class TestIds:
|
||||
def test_a_port_given_by_its_mac_is_read(self):
|
||||
phone = _rows()[0]
|
||||
|
||||
assert (phone["port_id"], phone["port_id_type"]) == ("00:11:22:33:44:01", "mac")
|
||||
|
||||
def test_a_port_given_by_a_local_id_is_read(self):
|
||||
switch = _rows()[2]
|
||||
|
||||
assert (switch["port_id"], switch["port_id_type"]) == ("gi1/0/7", "local")
|
||||
|
||||
def test_a_chassis_given_by_its_address_is_read(self):
|
||||
switch = _rows()[2]
|
||||
|
||||
assert (switch["chassis_id"], switch["chassis_id_type"]) == ("192.0.2.7", "ip")
|
||||
|
||||
@pytest.mark.parametrize("kind", ["local", "ifname", "ifalias", "unhandled"])
|
||||
def test_every_id_type_lldpd_names_is_read(self, kind):
|
||||
output = f"lldp.eth0.via=LLDP\nlldp.eth0.chassis.{kind}=c1\nlldp.eth0.port.{kind}=p1\n"
|
||||
|
||||
(row,) = _rows(output)
|
||||
|
||||
assert (row["chassis_id"], row["chassis_id_type"]) == ("c1", kind)
|
||||
assert (row["port_id"], row["port_id_type"]) == ("p1", kind)
|
||||
|
||||
def test_a_neighbour_without_ids_has_empty_ones(self):
|
||||
(row,) = _rows("lldp.eth0.via=CDPv2\nlldp.eth0.chassis.name=sw\n")
|
||||
|
||||
assert (row["chassis_id"], row["chassis_id_type"]) == ("", "")
|
||||
assert (row["port_id"], row["port_id_type"]) == ("", "")
|
||||
|
||||
|
||||
class TestOtherFields:
|
||||
def test_every_management_address_is_kept(self):
|
||||
phone = _rows()[0]
|
||||
|
||||
assert phone["management_addresses"] == ["10.0.0.21", "fe80::211:22ff:fe33:4401"]
|
||||
|
||||
def test_a_value_may_hold_an_equals_sign(self):
|
||||
assert _rows()[0]["system_description"] == "SIP phone, firmware a=1"
|
||||
|
||||
def test_capabilities_in_napalm_names_and_which_are_enabled(self):
|
||||
phone, pc, switch = _rows()
|
||||
|
||||
assert phone["capabilities"] == ["bridge", "telephone"]
|
||||
assert phone["enabled_capabilities"] == ["telephone"]
|
||||
assert pc["capabilities"] == ["bridge", "router", "wlan-access-point", "station"]
|
||||
assert pc["enabled_capabilities"] == ["station"]
|
||||
assert switch["capabilities"] == ["bridge", "router", "cvlan"]
|
||||
assert switch["enabled_capabilities"] == ["bridge", "cvlan"]
|
||||
|
||||
def test_every_napalm_capability_name(self):
|
||||
symbols = ["Other", "Repeater", "Bridge", "Router", "Wlan", "Tel", "Docsis", "Station"]
|
||||
output = "lldp.eth0.via=LLDP\n" + "".join(
|
||||
f"lldp.eth0.chassis.{s}.enabled=on\n" for s in symbols
|
||||
)
|
||||
|
||||
(row,) = _rows(output)
|
||||
|
||||
assert row["capabilities"] == [
|
||||
"other",
|
||||
"repeater",
|
||||
"bridge",
|
||||
"router",
|
||||
"wlan-access-point",
|
||||
"telephone",
|
||||
"docsis-cable-device",
|
||||
"station",
|
||||
]
|
||||
assert row["enabled_capabilities"] == row["capabilities"]
|
||||
|
||||
def test_a_capability_line_needs_on_or_off(self):
|
||||
(row,) = _rows(OPENWRT_ONE_NEIGHBOUR)
|
||||
|
||||
assert row["capabilities"] == []
|
||||
|
||||
|
||||
class TestReadOrFailure:
|
||||
"""A failed read raises; ``[]`` only when lldpctl ran and listed nobody."""
|
||||
|
||||
def test_no_output_is_no_neighbour(self):
|
||||
assert parse_lldpctl_keyvalue("") == []
|
||||
|
||||
def test_a_zero_exit_status_without_neighbours_is_no_neighbour(self):
|
||||
assert parse_lldpctl_keyvalue("__LLDPCTL_RC=0\n") == []
|
||||
|
||||
def test_a_missing_lldpctl_raises(self):
|
||||
with pytest.raises(ValueError, match="lldpctl: not found"):
|
||||
parse_lldpctl_keyvalue("sh: lldpctl: not found\n")
|
||||
|
||||
def test_a_non_zero_exit_status_raises_with_what_lldpctl_said(self):
|
||||
output = (
|
||||
"[WARN/control] unable to connect to socket /var/run/lldpd.socket: "
|
||||
"No such file or directory\n"
|
||||
"__LLDPCTL_RC=1\n"
|
||||
)
|
||||
|
||||
with pytest.raises(ValueError, match="unable to connect to socket"):
|
||||
parse_lldpctl_keyvalue(output)
|
||||
|
||||
def test_a_non_zero_exit_status_without_a_message_names_the_status(self):
|
||||
with pytest.raises(ValueError, match="exit status 127"):
|
||||
parse_lldpctl_keyvalue("__LLDPCTL_RC=127\n")
|
||||
|
||||
def test_with_status_zero_a_warning_beside_the_neighbours_is_ignored(self):
|
||||
output = (
|
||||
"lldpctl -f keyvalue 2>&1; echo __LLDPCTL_RC=$?\n"
|
||||
"[WARN/lldpctl] something lldpd wants to say\n"
|
||||
+ OPENWRT_ONE_NEIGHBOUR
|
||||
+ "__LLDPCTL_RC=0\n"
|
||||
)
|
||||
|
||||
assert [r["system_name"] for r in _rows(output)] == ["core-router"]
|
||||
|
||||
def test_a_line_outside_lldp_is_no_neighbour(self):
|
||||
output = "lldpd.config.tx-delay=30\n" + OPENWRT_ONE_NEIGHBOUR
|
||||
|
||||
assert len(_rows(output)) == 1
|
||||
|
||||
def test_a_key_without_a_known_field_is_skipped(self):
|
||||
output = "lldp.summary=1\nlldp.eth9.future-field=x\n" + OPENWRT_ONE_NEIGHBOUR
|
||||
|
||||
assert [r["local_port"] for r in _rows(output)] == ["eth0"]
|
||||
|
||||
|
||||
class TestCommand:
|
||||
def test_the_command_reads_the_neighbours_in_keyvalue_form(self):
|
||||
assert LLDPCTL_KEYVALUE_COMMAND.startswith("lldpctl -f keyvalue ")
|
||||
|
||||
def test_the_command_brings_back_the_error_text_and_the_exit_status(self):
|
||||
assert "2>&1" in LLDPCTL_KEYVALUE_COMMAND
|
||||
assert LLDPCTL_KEYVALUE_COMMAND.endswith("echo __LLDPCTL_RC=$?")
|
||||
@@ -0,0 +1,692 @@
|
||||
"""Personal accounts on a host: reading them, adding one, locking one again.
|
||||
|
||||
The host is a fake that keeps accounts, groups, sudo rules and key files in
|
||||
memory and answers the fixed commands the mixin and a driver send. A driver's
|
||||
own commands (create, open, lock, leave a group) are stand-ins here; the real
|
||||
ones are tested in the drivers.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shlex
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types.channel import CommandResult
|
||||
from napalm_device_types.local_accounts import (
|
||||
LocalAccountsMixin,
|
||||
authorized_keys_paths,
|
||||
sudoers_rule,
|
||||
sudoers_rule_path,
|
||||
)
|
||||
from napalm_device_types.models import FixStepOutcomeDict
|
||||
|
||||
ALICE_KEY = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAlice alice@laptop"
|
||||
OWN_KEY = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQOwn own@desktop"
|
||||
HASH = "$6$salt$" + "x" * 86
|
||||
#: Shell syntax a command for root must not hold: each is one fixed command.
|
||||
SHELL_SYNTAX = (";", "&&", "||", "|", ">", "<", "$(", "`")
|
||||
|
||||
|
||||
@dataclass
|
||||
class _Account:
|
||||
uid: int
|
||||
home: str
|
||||
shell: str = "/bin/sh"
|
||||
groups: list[str] = field(default_factory=list)
|
||||
locked: bool = False
|
||||
password: str | None = None
|
||||
|
||||
|
||||
class _Host:
|
||||
"""What the fake host holds, and the commands it ran."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.login = "netadmin"
|
||||
self.accounts: dict[str, _Account] = {
|
||||
"netadmin": _Account(1000, "/home/netadmin", groups=["netadmin", "wheel"]),
|
||||
"daemon": _Account(1, "/usr/sbin"),
|
||||
}
|
||||
#: Path -> content, for the sudoers drop-ins and the key files.
|
||||
self.files: dict[str, str] = {}
|
||||
#: Users a rule outside the drop-in directory lets run everything.
|
||||
self.other_rules: set[str] = set()
|
||||
#: Whether sudo reads the drop-in directory at all.
|
||||
self.includes_dropins = True
|
||||
#: Commands that fail, by their first words ("as <command>" for one run as
|
||||
#: the person); and commands that succeed without doing anything.
|
||||
self.failing: set[str] = set()
|
||||
self.ignored: set[str] = set()
|
||||
self.ran: list[tuple[str, bool]] = []
|
||||
self.staged = 0
|
||||
|
||||
def is_admin(self, name: str) -> bool:
|
||||
rule = self.files.get(sudoers_rule_path(_Driver.SUDOERS_DIR, name))
|
||||
return name in self.other_rules or (self.includes_dropins and rule is not None)
|
||||
|
||||
def run(self, command: str, privileged: bool, stdin: bytes | None) -> CommandResult:
|
||||
self.ran.append((command, privileged))
|
||||
if stdin is not None:
|
||||
assert not privileged, "a privileged command takes no stdin"
|
||||
return self._stage(command, stdin)
|
||||
words = shlex.split(command)
|
||||
if " ".join(words[:2]) in self.failing or words[0] in self.failing:
|
||||
return CommandResult("", f"{words[0]}: failed", 1)
|
||||
if words[:3] == ["sudo", "-n", "-u"]:
|
||||
assert privileged, "only root changes to another user"
|
||||
return self._as_user(words[3], words[4:])
|
||||
handler = getattr(self, "_" + words[0].replace("-", "_"), None)
|
||||
assert handler is not None, f"unexpected command: {command}"
|
||||
return handler(words[1:], privileged)
|
||||
|
||||
# -- staging, as the login user ---------------------------------------------
|
||||
|
||||
def _stage(self, command: str, stdin: bytes) -> CommandResult:
|
||||
assert "mktemp" in command
|
||||
if "mktemp" in self.failing:
|
||||
return CommandResult("", "mktemp: No space left on device", 1)
|
||||
self.staged += 1
|
||||
path = f"/tmp/tmp.{self.staged}"
|
||||
self.files[path] = stdin.decode()
|
||||
return CommandResult(path, "", 0)
|
||||
|
||||
def _rm(self, args: list[str], privileged: bool) -> CommandResult:
|
||||
for path in args[args.index("--") + 1 :]:
|
||||
if path.startswith("/tmp/"):
|
||||
assert not privileged, "the staged file is the login user's"
|
||||
self.files.pop(path, None)
|
||||
return CommandResult("", "", 0)
|
||||
|
||||
# -- the account database -------------------------------------------------------
|
||||
|
||||
def _getent(self, args: list[str], privileged: bool) -> CommandResult:
|
||||
assert args[0] == "passwd" and not privileged
|
||||
a = self.accounts.get(args[1])
|
||||
if a is None:
|
||||
return CommandResult("", "", 2)
|
||||
return CommandResult(f"{args[1]}:*:{a.uid}:{a.uid}::{a.home}:{a.shell}\n", "", 0)
|
||||
|
||||
def _id(self, args: list[str], privileged: bool) -> CommandResult:
|
||||
if args == ["-un"]:
|
||||
return CommandResult(f"{self.login}\n", "", 0)
|
||||
assert args[0] == "-Gn" and not privileged
|
||||
a = self.accounts.get(args[1])
|
||||
if a is None:
|
||||
return CommandResult("", f"id: {args[1]}: no such user", 1)
|
||||
return CommandResult(" ".join(a.groups) + "\n", "", 0)
|
||||
|
||||
def _sudo(self, args: list[str], privileged: bool) -> CommandResult:
|
||||
assert args[:2] == ["-l", "-U"] and privileged
|
||||
name = args[2]
|
||||
if self.is_admin(name):
|
||||
out = f"User {name} may run the following commands on host:\n (ALL) ALL\n"
|
||||
else:
|
||||
out = f"User {name} is not allowed to run sudo on host.\n"
|
||||
return CommandResult(out, "", 0)
|
||||
|
||||
def _visudo(self, args: list[str], privileged: bool) -> CommandResult:
|
||||
assert args[0] == "-cf" and privileged
|
||||
content = self.files[args[1]]
|
||||
ok = content.endswith(" ALL=(ALL) ALL\n") and "\n" not in content[:-1]
|
||||
return CommandResult("parsed OK" if ok else "syntax error", "", 0 if ok else 1)
|
||||
|
||||
def _install(self, args: list[str], privileged: bool) -> CommandResult:
|
||||
assert privileged
|
||||
source, target = args[args.index("--") + 1 :]
|
||||
assert target.startswith(_Driver.SUDOERS_DIR + "/"), "root installs only sudo rules"
|
||||
assert args[:6] == ["-m", "440", "-o", "root", "-g", _Driver.ROOT_GROUP]
|
||||
self.files[target] = self.files[source]
|
||||
return CommandResult("", "", 0)
|
||||
|
||||
def _test(self, args: list[str], privileged: bool) -> CommandResult:
|
||||
assert args[0] == "-e"
|
||||
return CommandResult("", "", 0 if args[1] in self.files else 1)
|
||||
|
||||
# -- the person's own files, as that person ------------------------------------
|
||||
|
||||
def _as_user(self, name: str, words: list[str]) -> CommandResult:
|
||||
a = self.accounts[name]
|
||||
if a.locked:
|
||||
return CommandResult("", "sudo: account expired", 1)
|
||||
rest = words[words.index("--") + 1 :] if "--" in words else words[1:]
|
||||
if f"as {words[0]}" in self.failing:
|
||||
return CommandResult("", f"{words[0]}: Permission denied", 1)
|
||||
if f"as {words[0]}" in self.ignored:
|
||||
return CommandResult("", "", 0)
|
||||
assert all(p.startswith(a.home + "/") or p.startswith("/tmp/") for p in rest)
|
||||
if words[0] == "cat":
|
||||
return CommandResult(self.files.get(rest[0], ""), "", 0)
|
||||
if words[0] == "mkdir":
|
||||
return CommandResult("", "", 0)
|
||||
if words[0] == "install":
|
||||
assert words[1:3] == ["-m", "600"]
|
||||
self.files[rest[1]] = self.files[rest[0]]
|
||||
return CommandResult("", "", 0)
|
||||
if words[0] == "rm":
|
||||
for path in rest:
|
||||
self.files.pop(path, None)
|
||||
return CommandResult("", "", 0)
|
||||
raise AssertionError(f"unexpected command as {name}: {words}")
|
||||
|
||||
# -- the driver's own commands (stand-ins) -------------------------------------
|
||||
|
||||
def _mkuser(self, args: list[str], privileged: bool) -> CommandResult:
|
||||
assert privileged
|
||||
name, password = args[0], (args[1] if len(args) > 1 else None)
|
||||
uid = 1001 + len(self.accounts)
|
||||
self.accounts[name] = _Account(uid, f"/home/{name}", groups=[name], password=password)
|
||||
return CommandResult("", "", 0)
|
||||
|
||||
def _openuser(self, args: list[str], privileged: bool) -> CommandResult:
|
||||
assert privileged
|
||||
a = self.accounts[args[0]]
|
||||
a.locked = False
|
||||
if len(args) > 1:
|
||||
a.password = args[1]
|
||||
return CommandResult("", "", 0)
|
||||
|
||||
def _lockuser(self, args: list[str], privileged: bool) -> CommandResult:
|
||||
assert privileged
|
||||
self.accounts[args[0]].locked = "lockuser" not in self.ignored
|
||||
return CommandResult("", "", 0)
|
||||
|
||||
def _islocked(self, args: list[str], privileged: bool) -> CommandResult:
|
||||
assert privileged
|
||||
return CommandResult("yes\n" if self.accounts[args[0]].locked else "no\n", "", 0)
|
||||
|
||||
def _leavegroup(self, args: list[str], privileged: bool) -> CommandResult:
|
||||
assert privileged
|
||||
self.accounts[args[0]].groups.remove(args[1])
|
||||
return CommandResult("", "", 0)
|
||||
|
||||
|
||||
class _Driver(LocalAccountsMixin):
|
||||
ACCOUNT_SHELL = "/bin/sh"
|
||||
ACCOUNT_ADMIN_GROUPS = ("wheel", "sudo")
|
||||
SUDOERS_DIR = "/etc/sudoers.d"
|
||||
ROOT_GROUP = "wheel"
|
||||
|
||||
def __init__(self, host: _Host) -> None:
|
||||
self.host = host
|
||||
|
||||
def run_command(
|
||||
self,
|
||||
command: str,
|
||||
*,
|
||||
privileged: bool = False,
|
||||
timeout: float = 60,
|
||||
stdin: bytes | None = None,
|
||||
) -> CommandResult:
|
||||
return self.host.run(command, privileged, stdin)
|
||||
|
||||
def _account_create(self, name: str, password_hash: str | None) -> list[list[str]]:
|
||||
return [["mkuser", name, *([password_hash] if password_hash else [])]]
|
||||
|
||||
def _account_open(
|
||||
self, name: str, password_hash: str | None, *, locked: bool
|
||||
) -> list[list[str]]:
|
||||
return [["openuser", name, *([password_hash] if password_hash else [])]]
|
||||
|
||||
def _account_lock(self, name: str, *, locked: bool) -> list[list[str]]:
|
||||
return [] if locked else [["lockuser", name]]
|
||||
|
||||
def _account_locked(self, name: str, passwd: list[str]) -> bool:
|
||||
result = self.run_command(shlex.join(["islocked", name]), privileged=True)
|
||||
return result.stdout.strip() == "yes"
|
||||
|
||||
def _account_leave_group(self, name: str, group: str, groups: list[str]) -> list[str]:
|
||||
return ["leavegroup", name, group]
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def host() -> _Host:
|
||||
return _Host()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def driver(host: _Host) -> _Driver:
|
||||
return _Driver(host)
|
||||
|
||||
|
||||
def _root_commands(host: _Host) -> list[str]:
|
||||
return [c for c, privileged in host.ran if privileged]
|
||||
|
||||
|
||||
def _add_alice(host: _Host, *, locked: bool = False, keys: str = "") -> None:
|
||||
host.accounts["alice"] = _Account(
|
||||
1001, "/home/alice", groups=["alice", "wheel"], locked=locked, password=HASH
|
||||
)
|
||||
if keys:
|
||||
host.files["/home/alice/.ssh/authorized_keys"] = keys
|
||||
|
||||
|
||||
# -- the helpers ---------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_the_sudo_rule_names_the_account_and_asks_for_its_password() -> None:
|
||||
assert sudoers_rule("alice") == "alice ALL=(ALL) ALL\n"
|
||||
|
||||
|
||||
def test_the_rule_file_has_no_dot_sudo_would_skip() -> None:
|
||||
# sudo ignores a file in an included directory whose name holds a dot.
|
||||
path = sudoers_rule_path("/etc/sudoers.d", "a.lice")
|
||||
assert path.startswith("/etc/sudoers.d/")
|
||||
assert "." not in path.rsplit("/", 1)[1]
|
||||
assert sudoers_rule_path("/etc/sudoers.d", "a.lice") != sudoers_rule_path(
|
||||
"/etc/sudoers.d", "a_lice"
|
||||
)
|
||||
|
||||
|
||||
def test_both_key_files_sshd_reads_by_default() -> None:
|
||||
assert authorized_keys_paths("/home/alice") == (
|
||||
"/home/alice/.ssh/authorized_keys",
|
||||
"/home/alice/.ssh/authorized_keys2",
|
||||
)
|
||||
|
||||
|
||||
# -- reading --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_an_account_is_read_with_its_groups_lock_sudo_and_keys(
|
||||
host: _Host, driver: _Driver
|
||||
) -> None:
|
||||
_add_alice(host, keys=ALICE_KEY + "\n" + OWN_KEY + "\n")
|
||||
host.other_rules.add("alice")
|
||||
|
||||
[account] = driver.get_local_accounts(["alice"])
|
||||
|
||||
assert account == {
|
||||
"name": "alice",
|
||||
"uid": 1001,
|
||||
"home": "/home/alice",
|
||||
"shell": "/bin/sh",
|
||||
"groups": ["alice", "wheel"],
|
||||
"locked": False,
|
||||
"admin": True,
|
||||
"authorized_keys": [ALICE_KEY, OWN_KEY],
|
||||
}
|
||||
|
||||
|
||||
def test_an_account_that_is_not_there_is_left_out(driver: _Driver) -> None:
|
||||
assert driver.get_local_accounts(["bob"]) == []
|
||||
|
||||
|
||||
def test_a_locked_accounts_keys_that_cannot_be_read_are_unknown(
|
||||
host: _Host, driver: _Driver
|
||||
) -> None:
|
||||
_add_alice(host, locked=True, keys=ALICE_KEY + "\n")
|
||||
|
||||
[account] = driver.get_local_accounts(["alice"])
|
||||
|
||||
assert account["locked"] is True
|
||||
assert account["authorized_keys"] is None
|
||||
|
||||
|
||||
def test_an_account_without_a_key_file_has_no_keys(host: _Host, driver: _Driver) -> None:
|
||||
_add_alice(host)
|
||||
|
||||
[account] = driver.get_local_accounts(["alice"])
|
||||
|
||||
assert account["authorized_keys"] == []
|
||||
|
||||
|
||||
def test_a_failing_read_raises_rather_than_report_nothing(host: _Host, driver: _Driver) -> None:
|
||||
_add_alice(host)
|
||||
host.failing.add("sudo -l")
|
||||
|
||||
with pytest.raises(RuntimeError, match="alice"):
|
||||
driver.get_local_accounts(["alice"])
|
||||
|
||||
|
||||
@pytest.mark.parametrize("name", ["", "Alice", "1alice", "al ice", "a;b", "-alice", "a" * 33])
|
||||
def test_a_name_no_guest_takes_is_refused(driver: _Driver, name: str) -> None:
|
||||
with pytest.raises(ValueError, match="account name"):
|
||||
driver.get_local_accounts([name])
|
||||
|
||||
|
||||
# -- adding ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_a_new_account_gets_the_key_and_the_sudo_rule(host: _Host, driver: _Driver) -> None:
|
||||
result = driver.add_local_account("alice", ALICE_KEY, password_hash=HASH, admin=True)
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert [s["step"] for s in result["steps"]] == ["account", "sudo", "keys"]
|
||||
assert host.accounts["alice"].password == HASH
|
||||
assert host.files["/home/alice/.ssh/authorized_keys"] == ALICE_KEY + "\n"
|
||||
assert host.files["/etc/sudoers.d/80-account-alice"] == "alice ALL=(ALL) ALL\n"
|
||||
account = result["account"]
|
||||
assert account is not None
|
||||
assert account["admin"] is True and account["authorized_keys"] == [ALICE_KEY]
|
||||
|
||||
|
||||
def test_every_command_root_runs_is_one_fixed_command(host: _Host, driver: _Driver) -> None:
|
||||
driver.add_local_account("alice", ALICE_KEY, password_hash=HASH, admin=True)
|
||||
driver.lock_local_account("alice")
|
||||
|
||||
for command in _root_commands(host):
|
||||
assert not any(s in command for s in SHELL_SYNTAX), command
|
||||
|
||||
|
||||
def test_the_staged_files_are_removed_again(host: _Host, driver: _Driver) -> None:
|
||||
driver.add_local_account("alice", ALICE_KEY, password_hash=HASH, admin=True)
|
||||
|
||||
assert host.staged == 2
|
||||
assert not [p for p in host.files if p.startswith("/tmp/")]
|
||||
|
||||
|
||||
def test_an_existing_account_keeps_its_own_keys(host: _Host, driver: _Driver) -> None:
|
||||
_add_alice(host, keys=OWN_KEY + "\n")
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY)
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert host.files["/home/alice/.ssh/authorized_keys"] == f"{OWN_KEY}\n{ALICE_KEY}\n"
|
||||
|
||||
|
||||
def test_a_line_without_a_key_is_kept_and_not_taken_for_one(host: _Host, driver: _Driver) -> None:
|
||||
_add_alice(host, keys="# the laptop\n")
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY)
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert host.files["/home/alice/.ssh/authorized_keys"] == f"# the laptop\n{ALICE_KEY}\n"
|
||||
|
||||
|
||||
def test_a_key_already_there_is_not_written_again(host: _Host, driver: _Driver) -> None:
|
||||
# The same key with options in front and another comment is the same key.
|
||||
_add_alice(host, keys=f'from="10.0.0.0/8" {ALICE_KEY.rsplit(" ", 1)[0]} old\n')
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY)
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert host.staged == 0
|
||||
assert "already" in result["steps"][-1]["output"]
|
||||
|
||||
|
||||
def test_a_locked_account_is_opened_again(host: _Host, driver: _Driver) -> None:
|
||||
_add_alice(host, locked=True)
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY, password_hash=HASH)
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert host.accounts["alice"].locked is False
|
||||
assert result["account"] is not None and result["account"]["locked"] is False
|
||||
|
||||
|
||||
def test_without_sudo_the_rule_and_the_admin_groups_go(host: _Host, driver: _Driver) -> None:
|
||||
_add_alice(host)
|
||||
host.files["/etc/sudoers.d/80-account-alice"] = sudoers_rule("alice")
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY, admin=False)
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert "/etc/sudoers.d/80-account-alice" not in host.files
|
||||
assert host.accounts["alice"].groups == ["alice"]
|
||||
|
||||
|
||||
def test_sudo_another_rule_still_grants_fails_the_step(host: _Host, driver: _Driver) -> None:
|
||||
_add_alice(host)
|
||||
host.other_rules.add("alice")
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY, admin=False)
|
||||
|
||||
assert not result["success"]
|
||||
sudo = next(s for s in result["steps"] if s["step"] == "sudo")
|
||||
assert not sudo["ok"] and "another rule" in sudo["output"]
|
||||
# The key still went in: the steps after the account are independent.
|
||||
assert host.files["/home/alice/.ssh/authorized_keys"] == ALICE_KEY + "\n"
|
||||
|
||||
|
||||
def test_a_rule_sudo_does_not_read_fails_the_step(host: _Host, driver: _Driver) -> None:
|
||||
host.includes_dropins = False
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY, password_hash=HASH, admin=True)
|
||||
|
||||
sudo = next(s for s in result["steps"] if s["step"] == "sudo")
|
||||
assert not sudo["ok"] and "/etc/sudoers.d" in sudo["output"]
|
||||
|
||||
|
||||
def test_a_rule_visudo_rejects_is_not_installed(host: _Host, driver: _Driver) -> None:
|
||||
host.failing.add("visudo")
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY, password_hash=HASH, admin=True)
|
||||
|
||||
assert not result["success"]
|
||||
assert "/etc/sudoers.d/80-account-alice" not in host.files
|
||||
assert not [p for p in host.files if p.startswith("/tmp/")]
|
||||
|
||||
|
||||
def test_a_failed_create_stops_before_sudo_and_keys(host: _Host, driver: _Driver) -> None:
|
||||
host.failing.add("mkuser")
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY, admin=True)
|
||||
|
||||
assert not result["success"]
|
||||
assert [s["step"] for s in result["steps"]] == ["account"]
|
||||
assert result["account"] is None
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"key",
|
||||
["", "not a key", ALICE_KEY + "\n" + OWN_KEY, "ssh-ed25519", "ssh-ed25519 AAAA;rm"],
|
||||
)
|
||||
def test_a_public_key_that_is_none_is_refused(driver: _Driver, key: str) -> None:
|
||||
with pytest.raises(ValueError, match="public key"):
|
||||
driver.add_local_account("alice", key)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("password_hash", ["secret", "$6$salt$ab cd", "$2b$10$" + "x" * 53])
|
||||
def test_a_password_hash_the_guest_does_not_take_is_refused(
|
||||
driver: _Driver, password_hash: str
|
||||
) -> None:
|
||||
# This driver takes SHA-512 crypt: neither plain text nor bcrypt.
|
||||
with pytest.raises(ValueError, match="password hash"):
|
||||
driver.add_local_account("alice", ALICE_KEY, password_hash=password_hash)
|
||||
|
||||
|
||||
def test_the_login_users_own_account_is_never_touched(host: _Host, driver: _Driver) -> None:
|
||||
for change in (
|
||||
lambda: driver.add_local_account("netadmin", ALICE_KEY),
|
||||
lambda: driver.lock_local_account("netadmin"),
|
||||
):
|
||||
with pytest.raises(ValueError, match="logs in"):
|
||||
change()
|
||||
assert _root_commands(host) == []
|
||||
|
||||
|
||||
def test_nothing_changes_when_the_login_user_cannot_be_told(host: _Host, driver: _Driver) -> None:
|
||||
_add_alice(host)
|
||||
host.login = ""
|
||||
|
||||
with pytest.raises(RuntimeError, match="logs in"):
|
||||
driver.lock_local_account("alice")
|
||||
assert _root_commands(host) == []
|
||||
|
||||
|
||||
def test_a_system_account_is_never_touched(host: _Host, driver: _Driver) -> None:
|
||||
with pytest.raises(ValueError, match="system account"):
|
||||
driver.lock_local_account("daemon")
|
||||
assert _root_commands(host) == []
|
||||
|
||||
|
||||
# -- locking --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_locking_takes_keys_sudo_and_login_away(host: _Host, driver: _Driver) -> None:
|
||||
_add_alice(host, keys=ALICE_KEY + "\n" + OWN_KEY + "\n")
|
||||
host.files["/home/alice/.ssh/authorized_keys2"] = OWN_KEY + "\n"
|
||||
host.files["/etc/sudoers.d/80-account-alice"] = sudoers_rule("alice")
|
||||
|
||||
result = driver.lock_local_account("alice")
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert [s["step"] for s in result["steps"]] == ["keys", "lock", "sudo"]
|
||||
a = host.accounts["alice"]
|
||||
assert a.locked and a.groups == ["alice"] and a.password == HASH
|
||||
assert not [p for p in host.files if p.startswith("/home/alice/")]
|
||||
assert "/etc/sudoers.d/80-account-alice" not in host.files
|
||||
account = result["account"]
|
||||
assert account is not None and account["locked"] and not account["admin"]
|
||||
|
||||
|
||||
def test_a_rule_left_elsewhere_is_named_but_harmless_once_locked(
|
||||
host: _Host, driver: _Driver
|
||||
) -> None:
|
||||
_add_alice(host)
|
||||
host.other_rules.add("alice")
|
||||
|
||||
result = driver.lock_local_account("alice")
|
||||
|
||||
assert result["success"], result["output"]
|
||||
sudo = next(s for s in result["steps"] if s["step"] == "sudo")
|
||||
assert "another rule" in sudo["output"] and "locked" in sudo["output"]
|
||||
|
||||
|
||||
def test_the_lock_comes_even_when_the_keys_could_not_go(host: _Host, driver: _Driver) -> None:
|
||||
_add_alice(host, locked=True, keys=ALICE_KEY + "\n")
|
||||
|
||||
result = driver.lock_local_account("alice")
|
||||
|
||||
assert not result["success"]
|
||||
keys = next(s for s in result["steps"] if s["step"] == "keys")
|
||||
assert not keys["ok"]
|
||||
assert next(s for s in result["steps"] if s["step"] == "lock")["ok"]
|
||||
assert host.accounts["alice"].locked
|
||||
|
||||
|
||||
def test_locking_an_account_that_is_not_there_fails(driver: _Driver) -> None:
|
||||
result = driver.lock_local_account("bob")
|
||||
|
||||
assert not result["success"]
|
||||
assert "no account" in result["output"]
|
||||
assert result["account"] is None
|
||||
|
||||
|
||||
# -- what fails is said ---------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.mark.parametrize("failing", ["getent", "id -Gn"])
|
||||
def test_an_account_that_cannot_be_read_raises(host: _Host, driver: _Driver, failing: str) -> None:
|
||||
_add_alice(host)
|
||||
host.failing.add(failing)
|
||||
|
||||
with pytest.raises(RuntimeError, match="alice"):
|
||||
driver.get_local_accounts(["alice"])
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("failing", "step", "said"),
|
||||
[
|
||||
("rm -f", "sudo", "80-account-alice"),
|
||||
("leavegroup", "sudo", "out of wheel"),
|
||||
("as cat", "keys", "cannot read"),
|
||||
("as install", "keys", "authorized_keys"),
|
||||
("mktemp", "keys", "cannot stage"),
|
||||
],
|
||||
)
|
||||
def test_a_step_that_fails_says_why(
|
||||
host: _Host, driver: _Driver, failing: str, step: str, said: str
|
||||
) -> None:
|
||||
_add_alice(host, keys=OWN_KEY + "\n")
|
||||
host.failing.add(failing)
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY)
|
||||
|
||||
failed = next(s for s in result["steps"] if s["step"] == step)
|
||||
assert not failed["ok"] and said in failed["output"]
|
||||
assert not result["success"]
|
||||
|
||||
|
||||
def test_a_rule_that_cannot_be_installed_fails_the_step(host: _Host, driver: _Driver) -> None:
|
||||
host.failing.add("install -m")
|
||||
|
||||
result = driver.add_local_account("alice", ALICE_KEY, password_hash=HASH, admin=True)
|
||||
|
||||
sudo = next(s for s in result["steps"] if s["step"] == "sudo")
|
||||
assert not sudo["ok"] and "80-account-alice" in sudo["output"]
|
||||
assert not [p for p in host.files if p.startswith("/tmp/")]
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("failing", "ignored", "step", "said"),
|
||||
[
|
||||
({"lockuser"}, set(), "lock", "lockuser: failed"),
|
||||
(set(), {"lockuser"}, "lock", "still not locked"),
|
||||
({"as rm"}, set(), "keys", "cannot remove"),
|
||||
(set(), {"as rm"}, "keys", "still has keys"),
|
||||
],
|
||||
)
|
||||
def test_a_lock_that_did_not_take_says_so(
|
||||
host: _Host, driver: _Driver, failing: set[str], ignored: set[str], step: str, said: str
|
||||
) -> None:
|
||||
_add_alice(host, keys=ALICE_KEY + "\n")
|
||||
host.failing |= failing
|
||||
host.ignored |= ignored
|
||||
|
||||
result = driver.lock_local_account("alice")
|
||||
|
||||
failed = next(s for s in result["steps"] if s["step"] == step)
|
||||
assert not failed["ok"] and said in failed["output"]
|
||||
assert not result["success"]
|
||||
|
||||
|
||||
def test_an_account_that_cannot_be_read_back_is_said(host: _Host, driver: _Driver) -> None:
|
||||
_add_alice(host)
|
||||
host.failing.add("sudo -l")
|
||||
|
||||
result = driver.lock_local_account("alice")
|
||||
|
||||
assert result["account"] is None
|
||||
assert "[read back] cannot ask sudo about alice" in result["output"]
|
||||
|
||||
|
||||
# -- a key store of the driver's own ---------------------------------------------------
|
||||
|
||||
|
||||
class _OwnKeyStore(_Driver):
|
||||
"""A driver whose host keeps keys in a place of its own."""
|
||||
|
||||
def __init__(self, host: _Host) -> None:
|
||||
super().__init__(host)
|
||||
self.store: dict[str, list[str]] = {}
|
||||
|
||||
def _account_read_keys(self, name: str, home: str) -> list[str] | None:
|
||||
return list(self.store.get(name, []))
|
||||
|
||||
def _account_add_key(self, name: str, home: str, public_key: str) -> FixStepOutcomeDict:
|
||||
self.store.setdefault(name, []).append(f"{home} {public_key}")
|
||||
return {"ok": True, "output": "stored"}
|
||||
|
||||
def _account_remove_keys(self, name: str, home: str) -> FixStepOutcomeDict:
|
||||
self.store.pop(name, None)
|
||||
return {"ok": True, "output": "dropped"}
|
||||
|
||||
|
||||
def test_a_driver_can_keep_the_keys_elsewhere(host: _Host) -> None:
|
||||
driver = _OwnKeyStore(host)
|
||||
|
||||
added = driver.add_local_account("alice", ALICE_KEY)
|
||||
assert added["success"], added["output"]
|
||||
assert driver.store == {"alice": [f"/home/alice {ALICE_KEY}"]}
|
||||
assert added["account"] is not None
|
||||
assert added["account"]["authorized_keys"] == [f"/home/alice {ALICE_KEY}"]
|
||||
assert not [p for p in host.files if p.startswith("/home/")]
|
||||
|
||||
locked = driver.lock_local_account("alice")
|
||||
assert locked["success"], locked["output"]
|
||||
assert driver.store == {}
|
||||
|
||||
|
||||
def test_a_key_file_the_host_adds_is_read_too(host: _Host) -> None:
|
||||
class _ExtraFile(_Driver):
|
||||
def _account_key_files(self, name: str, home: str) -> tuple[str, ...]:
|
||||
return (*super()._account_key_files(name, home), f"/home/{name}/extra")
|
||||
|
||||
_add_alice(host, keys=OWN_KEY + "\n")
|
||||
host.files["/home/alice/extra"] = ALICE_KEY + "\n"
|
||||
|
||||
[account] = _ExtraFile(host).get_local_accounts(["alice"])
|
||||
|
||||
assert account["authorized_keys"] == [OWN_KEY, ALICE_KEY]
|
||||
@@ -0,0 +1,25 @@
|
||||
"""A network target can say which physical NICs it leaves the host through.
|
||||
|
||||
A VM on a VLAN-aware bridge without a VLAN tag only gets an address when the
|
||||
switch port behind the bridge carries an untagged network. To look that port up,
|
||||
the caller needs the bridge's physical NICs: their LLDP neighbours name the
|
||||
switch and the port (NetOrk/netork#1325). They come in groups, one per way out
|
||||
of the bridge: a bond's slaves together, a single NIC alone. A bond has no
|
||||
neighbour of its own, and its slaves are one link, so one slave's neighbour
|
||||
speaks for the others when they report none.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import get_type_hints
|
||||
|
||||
from napalm_device_types.models import NetworkTargetDict
|
||||
|
||||
|
||||
def test_uplinks_is_an_optional_list_of_nic_groups():
|
||||
assert "uplinks" in NetworkTargetDict.__optional_keys__
|
||||
assert get_type_hints(NetworkTargetDict)["uplinks"] == list[list[str]]
|
||||
|
||||
|
||||
def test_contract_core_stays_required():
|
||||
assert {"name", "kind", "vlan_aware"} <= NetworkTargetDict.__required_keys__
|
||||
@@ -176,6 +176,18 @@ __DNF_RC=0
|
||||
with pytest.raises(ValueError, match="exit status 1"):
|
||||
parse_dnf_security(out)
|
||||
|
||||
def test_the_command_reads_only_the_cached_metadata(self, tmp_path):
|
||||
"""An expired cache made dnf download the repositories' metadata inside the
|
||||
poll, which outlasted its whole budget (#76)."""
|
||||
args = tmp_path / "args"
|
||||
dnf = tmp_path / "dnf"
|
||||
dnf.write_text(f"#!/bin/sh\nprintf '%s\\n' \"$@\" > {args}\n")
|
||||
dnf.chmod(0o755)
|
||||
env = {"PATH": f"{tmp_path}:{os.environ.get('PATH', '/usr/bin:/bin')}"}
|
||||
subprocess.run(["sh", "-c", DNF_SECURITY_COMMAND], capture_output=True, env=env)
|
||||
|
||||
assert "--cacheonly" in args.read_text().split()
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("nevra", "name"),
|
||||
[
|
||||
|
||||
@@ -18,6 +18,7 @@ import pytest
|
||||
|
||||
from napalm_device_types import (
|
||||
AccessPointDriver,
|
||||
BackupServerDriver,
|
||||
DeviceTypeDriver,
|
||||
FirewallDriver,
|
||||
HypervisorDriver,
|
||||
@@ -33,6 +34,7 @@ from napalm_device_types.roles import primary_role_of, roles_of
|
||||
|
||||
ROLE_BASES = [
|
||||
AccessPointDriver,
|
||||
BackupServerDriver,
|
||||
FirewallDriver,
|
||||
HypervisorDriver,
|
||||
ManagementControllerDriver,
|
||||
@@ -75,6 +77,8 @@ class TestDeclaredMethodsDoNotExistAtRuntime:
|
||||
@pytest.mark.parametrize(
|
||||
("base", "method"),
|
||||
[
|
||||
(BackupServerDriver, "get_backup_datastores"),
|
||||
(BackupServerDriver, "run_backup_job"),
|
||||
(StorageDriver, "get_disks"),
|
||||
(StorageDriver, "get_volumes"),
|
||||
(HypervisorDriver, "get_vms"),
|
||||
|
||||
@@ -0,0 +1,108 @@
|
||||
"""The scheduled-jobs contract: its digest, the names and specs it takes (NetOrk/netork#1933)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import ScheduledJobsMixin
|
||||
from napalm_device_types.scheduled_jobs import (
|
||||
ACTIONS,
|
||||
ScheduledJobChanged,
|
||||
ScheduledJobRefused,
|
||||
check_job_name,
|
||||
check_spec,
|
||||
job_digest,
|
||||
)
|
||||
|
||||
JOB = {
|
||||
"kind": "cron",
|
||||
"source": "/etc/crontab",
|
||||
"run_as": "root",
|
||||
"schedule": "*/5 * * * *",
|
||||
"command": "/usr/local/bin/backup",
|
||||
"enabled": True,
|
||||
}
|
||||
|
||||
|
||||
class TestDigest:
|
||||
def test_sixteen_hex_characters(self):
|
||||
digest = job_digest(JOB)
|
||||
assert len(digest) == 16
|
||||
int(digest, 16)
|
||||
|
||||
def test_the_same_definition_has_the_same_digest(self):
|
||||
assert job_digest(JOB) == job_digest(dict(JOB))
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"field, value",
|
||||
[
|
||||
("schedule", "0 * * * *"),
|
||||
("command", "/usr/local/bin/other"),
|
||||
("run_as", "nobody"),
|
||||
("enabled", False),
|
||||
("source", "/etc/cron.d/backup"),
|
||||
("kind", "anacron"),
|
||||
],
|
||||
)
|
||||
def test_every_part_of_the_definition_changes_it(self, field, value):
|
||||
assert job_digest({**JOB, field: value}) != job_digest(JOB)
|
||||
|
||||
def test_content_the_definition_points_at_changes_it(self):
|
||||
assert job_digest(JOB, content="12345 678") != job_digest(JOB)
|
||||
|
||||
def test_what_the_host_says_of_its_runs_does_not(self):
|
||||
assert job_digest({**JOB, "last_run": "2026-11-03T06:00:00Z"}) == job_digest(JOB)
|
||||
|
||||
|
||||
class TestNames:
|
||||
@pytest.mark.parametrize("name", ["backup", "netork-backup-db", "a", "x1"])
|
||||
def test_a_good_name(self, name):
|
||||
check_job_name(name)
|
||||
|
||||
@pytest.mark.parametrize("name", ["", "-x", "Backup", "a b", "a/b", "a.timer", "a" * 64])
|
||||
def test_a_name_a_file_or_unit_could_not_carry_safely(self, name):
|
||||
with pytest.raises(ScheduledJobRefused):
|
||||
check_job_name(name)
|
||||
|
||||
|
||||
class TestSpec:
|
||||
SPEC = {
|
||||
"name": "backup",
|
||||
"schedule": "0 3 * * *",
|
||||
"run_as": "root",
|
||||
"command": "/usr/local/bin/backup",
|
||||
"enabled": True,
|
||||
}
|
||||
|
||||
def test_a_good_spec(self):
|
||||
check_spec(self.SPEC)
|
||||
check_spec({**self.SPEC, "schedule": "@reboot"})
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"field, value",
|
||||
[
|
||||
("schedule", "every day"),
|
||||
("run_as", "root; rm"),
|
||||
("run_as", ""),
|
||||
("command", ""),
|
||||
("command", " "),
|
||||
("description", "two\nlines"),
|
||||
],
|
||||
)
|
||||
def test_a_spec_that_cannot_be_written(self, field, value):
|
||||
with pytest.raises(ScheduledJobRefused):
|
||||
check_spec({**self.SPEC, field: value})
|
||||
|
||||
|
||||
class TestContract:
|
||||
def test_the_mixin_declares_nothing_at_runtime(self):
|
||||
driver = ScheduledJobsMixin()
|
||||
for method in ("get_scheduled_jobs", "manage_scheduled_job", "put_scheduled_job"):
|
||||
assert not hasattr(driver, method)
|
||||
|
||||
def test_the_actions(self):
|
||||
assert ACTIONS == ("enable", "disable", "run")
|
||||
|
||||
def test_a_refusal_is_a_value_error_and_a_change_is_not(self):
|
||||
assert issubclass(ScheduledJobRefused, ValueError)
|
||||
assert not issubclass(ScheduledJobChanged, ValueError)
|
||||
@@ -0,0 +1,540 @@
|
||||
"""Scheduled jobs on a POSIX host: the report as root, switching, running, an application's jobs.
|
||||
|
||||
The reading command is run by a real ``sh`` over a directory tree that stands in
|
||||
for the host, with ``systemctl``, ``crontab``, ``dpkg``, ``install`` and
|
||||
``journalctl`` replaced by scripts that keep a log -- so the shell itself is what
|
||||
is tested, not a hand-written copy of its output (NetOrk/netork#1933).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import PosixScheduledJobsMixin
|
||||
from napalm_device_types._scheduled_reading import parse_reading
|
||||
from napalm_device_types.channel import CommandResult
|
||||
from napalm_device_types.scheduled_jobs import ScheduledJobChanged, ScheduledJobRefused
|
||||
|
||||
TIMERS = """\
|
||||
Id=apt-daily.timer
|
||||
Unit=apt-daily.service
|
||||
Description=Daily apt download activities
|
||||
TimersCalendar={ OnCalendar=*-*-* 06,18:00:00 ; next_elapse=@1762192800 }
|
||||
NextElapseUSecRealtime=@1762192800
|
||||
LastTriggerUSec=@1762149600
|
||||
UnitFileState=enabled
|
||||
ActiveState=active
|
||||
FragmentPath=/usr/lib/systemd/system/apt-daily.timer
|
||||
|
||||
Id=fstrim.timer
|
||||
Unit=fstrim.service
|
||||
Description=Discard unused blocks once a week
|
||||
TimersMonotonic={ OnUnitActiveUSec=1w ; next_elapse=n/a }
|
||||
NextElapseUSecRealtime=n/a
|
||||
LastTriggerUSec=n/a
|
||||
UnitFileState=static
|
||||
ActiveState=inactive
|
||||
FragmentPath=/etc/systemd/system/fstrim.timer
|
||||
|
||||
Id=backup.timer
|
||||
Unit=backup.service
|
||||
Description=netork backup
|
||||
TimersCalendar={ OnCalendar=*-*-* 03:00:00 ; next_elapse=n/a }
|
||||
NextElapseUSecRealtime=Tue 2026-11-04 03:00:00 UTC
|
||||
LastTriggerUSec=n/a
|
||||
UnitFileState=enabled
|
||||
ActiveState=active
|
||||
FragmentPath=/etc/systemd/system/backup.timer
|
||||
"""
|
||||
|
||||
UNITS = """\
|
||||
Id=apt-daily.service
|
||||
ExecStart={ path=/usr/lib/apt/apt.systemd.daily ; argv[]=/usr/lib/apt/apt.systemd.daily update ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }
|
||||
User=
|
||||
Result=success
|
||||
FragmentPath=/usr/lib/systemd/system/apt-daily.service
|
||||
|
||||
Id=fstrim.service
|
||||
ExecStart={ path=/sbin/fstrim ; argv[]=/sbin/fstrim --listed-in /etc/fstab ; ignore_errors=no }
|
||||
Result=success
|
||||
FragmentPath=/usr/lib/systemd/system/fstrim.service
|
||||
|
||||
Id=backup.service
|
||||
ExecStart={ path=SCRIPTS/backup ; argv[]=SCRIPTS/backup ; ignore_errors=no }
|
||||
User=backup
|
||||
Result=exit-code
|
||||
FragmentPath=/etc/systemd/system/backup.service
|
||||
"""
|
||||
|
||||
SYSTEMCTL = """#!/bin/sh
|
||||
echo "systemctl $*" >> "$LOG"
|
||||
case "$1" in
|
||||
list-unit-files) printf 'apt-daily.timer enabled enabled\\nfstrim.timer static -\\nbackup.timer enabled enabled\\ngetty@.timer static -\\n' ;;
|
||||
show) case "$*" in *Unit,Description*) sed "s#SCRIPTS#$SCRIPTS#g" "$FIX/timers" ;; *) sed "s#SCRIPTS#$SCRIPTS#g" "$FIX/units" ;; esac ;;
|
||||
esac
|
||||
"""
|
||||
|
||||
CRONTAB = """#!/bin/sh
|
||||
echo "crontab $*" >> "$LOG"
|
||||
[ "$1" = -u ] || exit 2
|
||||
if [ "$3" = -l ]; then
|
||||
[ -f "$SPOOL/$2" ] || { echo "no crontab for $2" >&2; exit 1; }
|
||||
cat "$SPOOL/$2"
|
||||
else
|
||||
cp "$3" "$SPOOL/$2"
|
||||
fi
|
||||
"""
|
||||
|
||||
DPKG = """#!/bin/sh
|
||||
for p in "$@"; do case " $OWNED " in *" $p "*) echo "cron: $p" ;; esac; done
|
||||
"""
|
||||
|
||||
INSTALL = """#!/bin/sh
|
||||
echo "install $*" >> "$LOG"
|
||||
args=""
|
||||
while [ $# -gt 0 ]; do case "$1" in -o|-g) shift 2 ;; *) args="$args '$1'"; shift ;; esac; done
|
||||
eval "/usr/bin/install $args"
|
||||
"""
|
||||
|
||||
ECHO_LOG = """#!/bin/sh
|
||||
echo "$(basename "$0") $*" >> "$LOG"
|
||||
[ "$(basename "$0")" = journalctl ] && echo "backup ran fine"
|
||||
exit 0
|
||||
"""
|
||||
|
||||
|
||||
class Host(PosixScheduledJobsMixin):
|
||||
"""A host in a directory; every command is run by sh, root or not."""
|
||||
|
||||
def __init__(self, root: Path, *, systemd: bool = True, own_cron_d: bool = True) -> None:
|
||||
self.root = root
|
||||
etc = root / "etc"
|
||||
self.CRON_SYSTEM_FILES = (str(etc / "crontab"),) # type: ignore[misc]
|
||||
self.CRON_D_DIRS = (str(etc / "cron.d"),) # type: ignore[misc]
|
||||
self.CRON_SPOOL_DIRS = (str(root / "spool"),) # type: ignore[misc]
|
||||
self.RUN_PARTS_DIRS = (("daily", str(etc / "cron.daily")),) # type: ignore[misc]
|
||||
self.ANACRONTAB = str(etc / "anacrontab") # type: ignore[misc]
|
||||
self.JOB_SCRIPT_DIR = str(root / "scripts") # type: ignore[misc]
|
||||
self.UNIT_DIR = str(root / "units") # type: ignore[misc]
|
||||
self.OWN_CRON_D_DIR = str(etc / "cron.d") if own_cron_d else None # type: ignore[misc]
|
||||
self.ROOT_GROUP = subprocess.run(
|
||||
["id", "-gn"], capture_output=True, text=True
|
||||
).stdout.strip() # type: ignore[misc]
|
||||
self.systemd_dir = root / ("run-systemd" if systemd else "no-systemd")
|
||||
self.commands: list[str] = []
|
||||
|
||||
def _sj_paths(self): # type: ignore[no-untyped-def]
|
||||
paths = super()._sj_paths()
|
||||
from dataclasses import replace
|
||||
|
||||
return replace(paths, systemd_marker=str(self.systemd_dir))
|
||||
|
||||
def run_command(self, command, *, privileged=False, timeout=60, stdin=None): # type: ignore[no-untyped-def]
|
||||
self.commands.append(command)
|
||||
env = {
|
||||
**os.environ,
|
||||
"PATH": f"{self.root / 'bin'}:{os.environ['PATH']}",
|
||||
"LOG": str(self.root / "log"),
|
||||
"SPOOL": str(self.root / "spool"),
|
||||
"FIX": str(self.root / "fix"),
|
||||
"SCRIPTS": str(self.root / "scripts"),
|
||||
"OWNED": str(self.root / "etc" / "crontab"),
|
||||
}
|
||||
done = subprocess.run(
|
||||
["sh", "-c", command], input=stdin, capture_output=True, env=env, timeout=30
|
||||
)
|
||||
return CommandResult(done.stdout.decode(), done.stderr.decode(), done.returncode)
|
||||
|
||||
def log(self) -> str:
|
||||
log = self.root / "log"
|
||||
return log.read_text() if log.exists() else ""
|
||||
|
||||
|
||||
def _tool(root: Path, name: str, body: str) -> None:
|
||||
path = root / "bin" / name
|
||||
path.write_text(body)
|
||||
path.chmod(0o755)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def root(tmp_path: Path) -> Path:
|
||||
for d in (
|
||||
"etc/cron.d",
|
||||
"etc/cron.daily",
|
||||
"spool",
|
||||
"scripts",
|
||||
"units",
|
||||
"bin",
|
||||
"fix",
|
||||
"run-systemd",
|
||||
):
|
||||
(tmp_path / d).mkdir(parents=True)
|
||||
(tmp_path / "etc/crontab").write_text(
|
||||
"SHELL=/bin/sh\nOUT="
|
||||
+ str(tmp_path / "out")
|
||||
+ "\n17 * * * * root cd / && run-parts /etc/cron.hourly\n"
|
||||
'*/5 * * * * root echo "$OUT" > ' + str(tmp_path / "ran") + "\n"
|
||||
)
|
||||
(tmp_path / "etc/cron.d/certbot").write_text("0 */12 * * * root certbot -q renew\n")
|
||||
(tmp_path / "etc/anacrontab").write_text("1 5 cron.daily run-parts /etc/cron.daily\n")
|
||||
(tmp_path / "spool/www-data").write_text("*/5 * * * * php /var/www/cron.php\n")
|
||||
for name, mode in (("logrotate", 0o755), ("off", 0o644), ("x.dpkg-old", 0o755)):
|
||||
script = tmp_path / "etc/cron.daily" / name
|
||||
script.write_text("#!/bin/sh\necho rotated\n")
|
||||
script.chmod(mode)
|
||||
(tmp_path / "scripts/backup").write_text(
|
||||
"#!/bin/sh\n# napalm-scheduled-job: backup nightly backup\n# schedule: 0 3 * * *\n/usr/local/bin/backup --all\n"
|
||||
)
|
||||
(tmp_path / "fix/timers").write_text(TIMERS)
|
||||
(tmp_path / "fix/units").write_text(UNITS)
|
||||
for name, body in (
|
||||
("systemctl", SYSTEMCTL),
|
||||
("crontab", CRONTAB),
|
||||
("dpkg", DPKG),
|
||||
("install", INSTALL),
|
||||
("journalctl", ECHO_LOG),
|
||||
("systemd-analyze", ECHO_LOG),
|
||||
):
|
||||
_tool(tmp_path, name, body)
|
||||
return tmp_path
|
||||
|
||||
|
||||
def _jobs(host: Host) -> dict[str, dict]:
|
||||
return {j["key"]: j for j in host.get_scheduled_jobs()["jobs"]}
|
||||
|
||||
|
||||
class TestReading:
|
||||
def test_every_source_is_read(self, root: Path):
|
||||
jobs = _jobs(Host(root))
|
||||
kinds = sorted({j["kind"] for j in jobs.values()})
|
||||
assert kinds == ["anacron", "cron", "run_parts", "systemd_timer"]
|
||||
users = {j["run_as"] for j in jobs.values() if j["kind"] == "cron"}
|
||||
assert users == {"root", "www-data"}
|
||||
|
||||
def test_a_package_owns_what_it_installed(self, root: Path):
|
||||
jobs = _jobs(Host(root))
|
||||
crontab = [j for j in jobs.values() if j["source"] == str(root / "etc/crontab")]
|
||||
assert {j["managed_by"] for j in crontab} == {"package"}
|
||||
certbot = next(j for j in jobs.values() if j["source"].endswith("certbot"))
|
||||
assert certbot["managed_by"] == "admin"
|
||||
|
||||
def test_run_parts_by_execute_bit_and_the_names_run_parts_runs(self, root: Path):
|
||||
jobs = _jobs(Host(root))
|
||||
daily = root / "etc/cron.daily"
|
||||
assert jobs[f"run-parts:{daily}/logrotate"]["enabled"]
|
||||
assert not jobs[f"run-parts:{daily}/off"]["enabled"]
|
||||
assert f"run-parts:{daily}/x.dpkg-old" not in jobs
|
||||
|
||||
def test_a_changed_script_changes_its_digest(self, root: Path):
|
||||
host = Host(root)
|
||||
key = f"run-parts:{root}/etc/cron.daily/logrotate"
|
||||
before = _jobs(host)[key]["digest"]
|
||||
(root / "etc/cron.daily/logrotate").write_text("#!/bin/sh\necho something else\n")
|
||||
assert _jobs(host)[key]["digest"] != before
|
||||
|
||||
def test_timers_with_their_unit_and_runs(self, root: Path):
|
||||
apt = _jobs(Host(root))["timer:apt-daily.timer"]
|
||||
assert apt["schedule"] == "OnCalendar=*-*-* 06,18:00:00"
|
||||
assert apt["schedule_kind"] == "calendar"
|
||||
assert apt["command"] == "/usr/lib/apt/apt.systemd.daily update"
|
||||
assert (apt["run_as"], apt["managed_by"], apt["enabled"]) == ("root", "package", True)
|
||||
assert apt["last_run"] == "2025-11-03T06:00:00+00:00"
|
||||
assert apt["next_run"] == "2025-11-03T18:00:00+00:00"
|
||||
assert apt["last_result"] == "success"
|
||||
assert apt["actions"] == ["enable", "disable", "run"]
|
||||
|
||||
def test_a_static_timer_only_runs(self, root: Path):
|
||||
fstrim = _jobs(Host(root))["timer:fstrim.timer"]
|
||||
assert fstrim["schedule"] == "OnUnitActiveSec=1w"
|
||||
assert fstrim["schedule_kind"] == "monotonic"
|
||||
assert fstrim["actions"] == ["run"]
|
||||
assert fstrim.get("last_run") is None
|
||||
assert fstrim["managed_by"] == "admin"
|
||||
|
||||
def test_an_applications_timer_by_its_script(self, root: Path):
|
||||
backup = _jobs(Host(root))["job:backup"]
|
||||
assert backup["managed_by"] == "application"
|
||||
assert (backup["name"], backup["description"]) == ("backup", "nightly backup")
|
||||
assert (backup["schedule"], backup["schedule_kind"]) == ("0 3 * * *", "cron")
|
||||
assert backup["command"] == "/usr/local/bin/backup --all"
|
||||
assert backup["run_as"] == "backup"
|
||||
assert "edit" in backup["actions"] and "delete" in backup["actions"]
|
||||
assert backup["next_run"] == "2026-11-04T03:00:00+00:00"
|
||||
assert "last_result" not in backup
|
||||
|
||||
def test_a_file_without_a_last_newline_does_not_swallow_the_next(self, root: Path):
|
||||
(root / "etc/cron.d/certbot").write_text("0 */12 * * * root certbot -q renew")
|
||||
jobs = _jobs(Host(root))
|
||||
assert any(j["source"].endswith("certbot") for j in jobs.values())
|
||||
assert any(j["source"].endswith("spool/www-data") for j in jobs.values())
|
||||
|
||||
def test_without_systemd_no_timers(self, root: Path):
|
||||
jobs = _jobs(Host(root, systemd=False))
|
||||
assert not [j for j in jobs.values() if j["kind"] == "systemd_timer"]
|
||||
|
||||
def test_a_report_cut_short_raises(self, root: Path):
|
||||
host = Host(root)
|
||||
full = host.run_command
|
||||
host.run_command = lambda c, **kw: full(c, **kw)._replace(
|
||||
stdout=full(c, **kw).stdout.split("SJ_END")[0][:-3]
|
||||
) # type: ignore[method-assign]
|
||||
with pytest.raises(ValueError, match="cut short"):
|
||||
host.get_scheduled_jobs()
|
||||
|
||||
|
||||
class TestDriverSettings:
|
||||
def test_what_the_platform_generates(self, root: Path):
|
||||
host = Host(root)
|
||||
host.PLATFORM_SOURCES = ("*/cron.d/cert*",) # type: ignore[misc]
|
||||
certbot = next(j for j in _jobs(host).values() if j["source"].endswith("certbot"))
|
||||
assert certbot["managed_by"] == "platform"
|
||||
|
||||
def test_the_time_zone_and_owners_commands_are_the_drivers(self, root: Path):
|
||||
host = Host(root)
|
||||
host.TIME_ZONE_SH = "echo America/New_York" # type: ignore[misc]
|
||||
host.OWNERS_SH = "cat >/dev/null" # type: ignore[misc]
|
||||
reading = host.get_scheduled_jobs()
|
||||
assert reading["time_zone"] == "America/New_York"
|
||||
assert {j["managed_by"] for j in reading["jobs"] if j["kind"] == "cron"} == {"admin"}
|
||||
|
||||
|
||||
class TestParseReading:
|
||||
def test_the_time_zone_from_either_file(self):
|
||||
out = "SJ_BEGIN\n[tz]\n/usr/share/zoneinfo/Europe/Berlin\nSJ_END\n"
|
||||
assert parse_reading(out).time_zone == "Europe/Berlin"
|
||||
assert parse_reading("SJ_BEGIN\n[tz]\nEtc/UTC\nSJ_END").time_zone == "Etc/UTC"
|
||||
assert parse_reading("SJ_BEGIN\n[tz]\nSJ_END").time_zone is None
|
||||
|
||||
def test_owners_skip_diversions(self):
|
||||
out = "SJ_BEGIN\n[owners]\ndiversion by x from: /etc/crontab\ncron, cron-daemon: /etc/cron.d/x\nSJ_END"
|
||||
reading = parse_reading(out + "")
|
||||
assert reading.jobs == []
|
||||
|
||||
def test_a_time_not_in_utc_is_left_unknown(self):
|
||||
from napalm_device_types._scheduled_reading import _timestamp
|
||||
|
||||
assert _timestamp("Tue 2026-11-04 03:00:00 CET") is None
|
||||
assert _timestamp("@0") is None
|
||||
|
||||
|
||||
class TestSwitching:
|
||||
def _cron(self, host: Host, path: str) -> dict:
|
||||
return next(j for j in _jobs(host).values() if j["source"] == path and j["kind"] == "cron")
|
||||
|
||||
def test_a_system_line_switched_off_and_on(self, root: Path):
|
||||
host = Host(root)
|
||||
path = str(root / "etc/cron.d/certbot")
|
||||
job = self._cron(host, path)
|
||||
mode = os.stat(path).st_mode
|
||||
assert host.manage_scheduled_job(job["key"], "disable", digest=job["digest"])["success"]
|
||||
assert (
|
||||
root / "etc/cron.d/certbot"
|
||||
).read_text() == "#napalm-disabled: 0 */12 * * * root certbot -q renew\n"
|
||||
off = _jobs(host)[job["key"]]
|
||||
assert not off["enabled"]
|
||||
host.manage_scheduled_job(job["key"], "enable", digest=off["digest"])
|
||||
assert (root / "etc/cron.d/certbot").read_text() == "0 */12 * * * root certbot -q renew\n"
|
||||
assert f"install -D -m {mode & 0o777:o}" in host.log()
|
||||
assert os.stat(path).st_mode == mode
|
||||
|
||||
def test_a_linked_file_is_written_through_its_link(self, root: Path):
|
||||
real = root / "etc/vzdump.cron"
|
||||
real.write_text("0 1 * * * root vzdump --all\n")
|
||||
real.chmod(0o640)
|
||||
link = root / "etc/cron.d/vzdump"
|
||||
link.symlink_to(real)
|
||||
host = Host(root)
|
||||
job = self._cron(host, str(link))
|
||||
host.manage_scheduled_job(job["key"], "disable", digest=job["digest"])
|
||||
assert link.is_symlink()
|
||||
assert real.read_text() == "#napalm-disabled: 0 1 * * * root vzdump --all\n"
|
||||
assert real.stat().st_mode & 0o777 == 0o640
|
||||
|
||||
def test_a_users_line_goes_through_crontab(self, root: Path):
|
||||
host = Host(root)
|
||||
job = self._cron(host, str(root / "spool/www-data"))
|
||||
host.manage_scheduled_job(job["key"], "disable", digest=job["digest"])
|
||||
assert (root / "spool/www-data").read_text().startswith("#napalm-disabled: */5")
|
||||
assert "crontab -u www-data /" in host.log()
|
||||
|
||||
def test_a_change_it_did_not_see_is_refused(self, root: Path):
|
||||
host = Host(root)
|
||||
job = self._cron(host, str(root / "etc/cron.d/certbot"))
|
||||
with pytest.raises(ScheduledJobChanged):
|
||||
host.manage_scheduled_job(job["key"], "disable", digest="0" * 16)
|
||||
|
||||
def test_an_unknown_job_and_an_unknown_action(self, root: Path):
|
||||
host = Host(root)
|
||||
with pytest.raises(LookupError):
|
||||
host.manage_scheduled_job("cron:/nowhere:1", "disable", digest="x")
|
||||
with pytest.raises(ScheduledJobRefused):
|
||||
host.manage_scheduled_job("cron:/nowhere:1", "explode", digest="x")
|
||||
|
||||
def test_a_static_timer_is_not_disabled(self, root: Path):
|
||||
host = Host(root)
|
||||
fstrim = _jobs(host)["timer:fstrim.timer"]
|
||||
with pytest.raises(ScheduledJobRefused):
|
||||
host.manage_scheduled_job("timer:fstrim.timer", "disable", digest=fstrim["digest"])
|
||||
|
||||
def test_a_timer_and_a_run_parts_script(self, root: Path):
|
||||
host = Host(root)
|
||||
jobs = _jobs(host)
|
||||
apt = jobs["timer:apt-daily.timer"]
|
||||
host.manage_scheduled_job(apt["key"], "disable", digest=apt["digest"])
|
||||
assert "systemctl disable --now -- apt-daily.timer" in host.log()
|
||||
key = f"run-parts:{root}/etc/cron.daily/off"
|
||||
host.manage_scheduled_job(key, "enable", digest=jobs[key]["digest"])
|
||||
assert os.access(root / "etc/cron.daily/off", os.X_OK)
|
||||
|
||||
def test_an_applications_timer_is_switched_by_its_name(self, root: Path):
|
||||
host = Host(root)
|
||||
backup = _jobs(host)["job:backup"]
|
||||
host.manage_scheduled_job("job:backup", "disable", digest=backup["digest"])
|
||||
assert "systemctl disable --now -- backup.timer" in host.log()
|
||||
|
||||
|
||||
class TestRunning:
|
||||
def test_a_cron_line_runs_as_cron_would(self, root: Path):
|
||||
host = Host(root)
|
||||
job = next(j for j in _jobs(host).values() if "ran" in j["command"])
|
||||
result = host.manage_scheduled_job(job["key"], "run", digest=job["digest"])
|
||||
assert result["success"]
|
||||
assert (root / "ran").read_text() == f"{root / 'out'}\n"
|
||||
assert "timeout 300 sh -c" in host.commands[-1]
|
||||
|
||||
def test_a_timer_runs_its_unit_and_says_what_its_journal_said(self, root: Path):
|
||||
host = Host(root)
|
||||
backup = _jobs(host)["job:backup"]
|
||||
result = host.manage_scheduled_job("job:backup", "run", digest=backup["digest"])
|
||||
assert result == {"success": True, "output": "backup ran fine"}
|
||||
assert "systemctl start -- backup.service" in host.log()
|
||||
|
||||
def test_a_run_parts_script_runs(self, root: Path):
|
||||
host = Host(root)
|
||||
key = f"run-parts:{root}/etc/cron.daily/logrotate"
|
||||
result = host.manage_scheduled_job(key, "run", digest=_jobs(host)[key]["digest"])
|
||||
assert result == {"success": True, "output": "rotated"}
|
||||
|
||||
def test_another_user_runs_through_su(self):
|
||||
assert Host(Path("/x"))._sj_as("www-data", "true") == [
|
||||
"su",
|
||||
"-s",
|
||||
"/bin/sh",
|
||||
"www-data",
|
||||
"-c",
|
||||
"true",
|
||||
]
|
||||
|
||||
|
||||
SPEC = {
|
||||
"name": "netork-tidy",
|
||||
"schedule": "*/30 1-2 * * *",
|
||||
"run_as": "root",
|
||||
"command": "find /tmp -mtime +7 -delete",
|
||||
"enabled": True,
|
||||
"description": "tidy %tmp",
|
||||
}
|
||||
|
||||
|
||||
class TestApplicationJobs:
|
||||
def test_a_timer_on_a_host_with_systemd(self, root: Path):
|
||||
host = Host(root)
|
||||
assert host.put_scheduled_job(SPEC)["output"] == "job:netork-tidy created"
|
||||
timer = (root / "units/netork-tidy.timer").read_text()
|
||||
assert "OnCalendar=*-*-* 01,02:00,30:00" in timer
|
||||
assert "Description=tidy %%tmp" in timer
|
||||
service = (root / "units/netork-tidy.service").read_text()
|
||||
assert f"ExecStart={root}/scripts/netork-tidy" in service and "User=root" in service
|
||||
script = (root / "scripts/netork-tidy").read_text()
|
||||
assert script.splitlines()[1:] == [
|
||||
"# napalm-scheduled-job: netork-tidy tidy %tmp",
|
||||
"# schedule: */30 1-2 * * *",
|
||||
"find /tmp -mtime +7 -delete",
|
||||
]
|
||||
assert "systemctl daemon-reload" in host.log()
|
||||
assert "systemctl enable --now -- netork-tidy.timer" in host.log()
|
||||
|
||||
def test_reboot_is_a_boot_timer(self, root: Path):
|
||||
Host(root).put_scheduled_job({**SPEC, "schedule": "@reboot", "enabled": False})
|
||||
assert "OnBootSec=1min" in (root / "units/netork-tidy.timer").read_text()
|
||||
|
||||
def test_creating_one_that_is_there_and_changing_one_that_is_not(self, root: Path):
|
||||
host = Host(root)
|
||||
with pytest.raises(ScheduledJobChanged):
|
||||
host.put_scheduled_job({**SPEC, "name": "backup"})
|
||||
with pytest.raises(LookupError):
|
||||
host.put_scheduled_job(SPEC, digest="x")
|
||||
with pytest.raises(ScheduledJobChanged):
|
||||
host.put_scheduled_job({**SPEC, "name": "backup"}, digest="x")
|
||||
|
||||
def test_an_unknown_user_is_refused(self, root: Path):
|
||||
with pytest.raises(ScheduledJobRefused, match="no user"):
|
||||
Host(root).put_scheduled_job({**SPEC, "run_as": "no-such-user-here"})
|
||||
|
||||
def test_a_timer_deleted(self, root: Path):
|
||||
host = Host(root)
|
||||
backup = _jobs(host)["job:backup"]
|
||||
host.delete_scheduled_job("job:backup", digest=backup["digest"])
|
||||
assert not (root / "scripts/backup").exists()
|
||||
assert "systemctl disable --now -- backup.timer" in host.log()
|
||||
|
||||
def test_only_an_applications_job_is_deleted(self, root: Path):
|
||||
host = Host(root)
|
||||
apt = _jobs(host)["timer:apt-daily.timer"]
|
||||
with pytest.raises(ScheduledJobRefused):
|
||||
host.delete_scheduled_job(apt["key"], digest=apt["digest"])
|
||||
|
||||
def test_a_cron_d_file_without_systemd(self, root: Path):
|
||||
host = Host(root, systemd=False)
|
||||
host.put_scheduled_job(SPEC)
|
||||
assert (root / "etc/cron.d/netork-tidy").read_text() == (
|
||||
"# napalm-scheduled-job: netork-tidy tidy %tmp\n*/30 1-2 * * * root find /tmp -mtime +7 -delete\n"
|
||||
)
|
||||
job = _jobs(host)["job:netork-tidy"]
|
||||
assert job["managed_by"] == "application" and job["kind"] == "cron"
|
||||
host.delete_scheduled_job("job:netork-tidy", digest=job["digest"])
|
||||
assert not (root / "etc/cron.d/netork-tidy").exists()
|
||||
|
||||
def test_marked_lines_in_the_users_crontab_on_a_busybox_host(self, root: Path):
|
||||
host = Host(root, systemd=False, own_cron_d=False)
|
||||
host.put_scheduled_job({**SPEC, "run_as": "www-data"})
|
||||
text = (root / "spool/www-data").read_text()
|
||||
assert text.endswith(
|
||||
"# napalm-scheduled-job: netork-tidy tidy %tmp\n*/30 1-2 * * * find /tmp -mtime +7 -delete\n"
|
||||
)
|
||||
job = _jobs(host)["job:netork-tidy"]
|
||||
host.put_scheduled_job(
|
||||
{**SPEC, "run_as": "www-data", "enabled": False}, digest=job["digest"]
|
||||
)
|
||||
assert "#napalm-disabled: */30" in (root / "spool/www-data").read_text()
|
||||
job = _jobs(host)["job:netork-tidy"]
|
||||
host.put_scheduled_job({**SPEC, "run_as": "nobody"}, digest=job["digest"])
|
||||
assert "netork-tidy" not in (root / "spool/www-data").read_text()
|
||||
assert "netork-tidy" in (root / "spool/nobody").read_text()
|
||||
job = _jobs(host)["job:netork-tidy"]
|
||||
host.delete_scheduled_job("job:netork-tidy", digest=job["digest"])
|
||||
assert (root / "spool/nobody").read_text() == ""
|
||||
|
||||
def test_a_cron_d_that_is_not_there_is_the_crontab(self, root: Path):
|
||||
host = Host(root, systemd=False)
|
||||
host.OWN_CRON_D_DIR = str(root / "etc/no-cron.d") # type: ignore[misc]
|
||||
host.put_scheduled_job(SPEC)
|
||||
assert "netork-tidy" in (root / "spool/root").read_text()
|
||||
|
||||
def test_a_cron_line_holds_one_line(self, root: Path):
|
||||
with pytest.raises(ScheduledJobRefused, match="one line"):
|
||||
Host(root, systemd=False).put_scheduled_job({**SPEC, "command": "a\nb"})
|
||||
|
||||
def test_a_calendar_systemd_refuses(self, root: Path):
|
||||
_tool(root, "systemd-analyze", "#!/bin/sh\necho 'Failed to parse' >&2\nexit 1\n")
|
||||
with pytest.raises(ScheduledJobRefused, match="systemd does not take"):
|
||||
Host(root).put_scheduled_job(SPEC)
|
||||
|
||||
def test_a_failed_install_says_so(self, root: Path):
|
||||
_tool(root, "install", "#!/bin/sh\necho 'read-only file system' >&2\nexit 1\n")
|
||||
with pytest.raises(RuntimeError, match="read-only"):
|
||||
Host(root).put_scheduled_job(SPEC)
|
||||
@@ -0,0 +1,301 @@
|
||||
"""netOrk's own account on a host: created through the account it logs in with today.
|
||||
|
||||
The fake host of :mod:`tests.test_local_accounts`, taught the rule that lets
|
||||
an account run everything as root without a password.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shlex
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types.channel import CommandResult
|
||||
from napalm_device_types.local_accounts import (
|
||||
service_sudoers_rule,
|
||||
service_sudoers_rule_path,
|
||||
sudoers_rule_path,
|
||||
)
|
||||
from tests.test_local_accounts import (
|
||||
OWN_KEY,
|
||||
SHELL_SYNTAX,
|
||||
_Account,
|
||||
_Driver,
|
||||
_Host,
|
||||
_root_commands,
|
||||
)
|
||||
|
||||
NETORK_KEY = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINetork netork@central"
|
||||
|
||||
|
||||
class _ServiceHost(_Host):
|
||||
"""The fake host, whose sudo also takes the rule without a password."""
|
||||
|
||||
def is_passwordless(self, name: str) -> bool:
|
||||
rule = self.files.get(service_sudoers_rule_path(_Driver.SUDOERS_DIR, name))
|
||||
return self.includes_dropins and rule is not None
|
||||
|
||||
def _sudo(self, args: list[str], privileged: bool) -> CommandResult:
|
||||
name = args[2]
|
||||
if not self.is_passwordless(name):
|
||||
return super()._sudo(args, privileged)
|
||||
out = f"User {name} may run the following commands on host:\n (ALL) NOPASSWD: ALL\n"
|
||||
return CommandResult(out, "", 0)
|
||||
|
||||
def _visudo(self, args: list[str], privileged: bool) -> CommandResult:
|
||||
content = self.files[args[1]]
|
||||
if content.endswith(" ALL=(ALL) NOPASSWD: ALL\n") and "\n" not in content[:-1]:
|
||||
return CommandResult("parsed OK", "", 0)
|
||||
return super()._visudo(args, privileged)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def host() -> _ServiceHost:
|
||||
return _ServiceHost()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def driver(host: _ServiceHost) -> _Driver:
|
||||
return _Driver(host)
|
||||
|
||||
|
||||
def test_the_rule_runs_everything_as_root_without_a_password() -> None:
|
||||
assert service_sudoers_rule("netork") == "netork ALL=(ALL) NOPASSWD: ALL\n"
|
||||
|
||||
|
||||
def test_the_rule_has_a_file_of_its_own_no_personal_rule_shares() -> None:
|
||||
path = service_sudoers_rule_path("/etc/sudoers.d", "net.ork_1")
|
||||
assert path == "/etc/sudoers.d/85-service-net_2eork__1"
|
||||
assert path != sudoers_rule_path("/etc/sudoers.d", "net.ork_1")
|
||||
|
||||
|
||||
def test_a_new_service_account_gets_the_key_and_sudo_without_a_password(
|
||||
host: _ServiceHost, driver: _Driver
|
||||
) -> None:
|
||||
result = driver.add_service_account("netork", NETORK_KEY)
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert [s["step"] for s in result["steps"]] == ["account", "sudo", "keys"]
|
||||
assert host.accounts["netork"].password is None
|
||||
assert host.files["/home/netork/.ssh/authorized_keys"] == NETORK_KEY + "\n"
|
||||
assert host.is_passwordless("netork")
|
||||
account = result["account"]
|
||||
assert account is not None and account["admin"]
|
||||
|
||||
|
||||
def test_every_command_root_runs_is_one_fixed_command(host: _ServiceHost, driver: _Driver) -> None:
|
||||
driver.add_service_account("netork", NETORK_KEY)
|
||||
|
||||
for command in _root_commands(host):
|
||||
assert not any(s in command for s in SHELL_SYNTAX), command
|
||||
|
||||
|
||||
def test_an_existing_account_keeps_its_keys_and_gets_the_rule(
|
||||
host: _ServiceHost, driver: _Driver
|
||||
) -> None:
|
||||
host.accounts["netork"] = _Account(1001, "/home/netork", groups=["netork"])
|
||||
host.files["/home/netork/.ssh/authorized_keys"] = OWN_KEY + "\n"
|
||||
|
||||
result = driver.add_service_account("netork", NETORK_KEY)
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert host.files["/home/netork/.ssh/authorized_keys"] == f"{OWN_KEY}\n{NETORK_KEY}\n"
|
||||
assert host.is_passwordless("netork")
|
||||
|
||||
|
||||
def test_an_existing_system_account_may_become_netorks(host: _ServiceHost, driver: _Driver) -> None:
|
||||
host.accounts["netork"] = _Account(998, "/var/lib/netork", groups=["netork"])
|
||||
|
||||
result = driver.add_service_account("netork", NETORK_KEY)
|
||||
|
||||
assert result["success"], result["output"]
|
||||
|
||||
|
||||
def test_a_locked_account_is_opened_again(host: _ServiceHost, driver: _Driver) -> None:
|
||||
host.accounts["netork"] = _Account(1001, "/home/netork", groups=["netork"], locked=True)
|
||||
|
||||
result = driver.add_service_account("netork", NETORK_KEY)
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert not host.accounts["netork"].locked
|
||||
|
||||
|
||||
def test_the_account_netork_logs_in_with_already_only_gets_what_it_lacks(
|
||||
host: _ServiceHost, driver: _Driver
|
||||
) -> None:
|
||||
host.login = "netork"
|
||||
host.accounts["netork"] = _Account(1001, "/home/netork", groups=["netork"])
|
||||
host.files["/home/netork/.ssh/authorized_keys"] = NETORK_KEY + "\n"
|
||||
|
||||
result = driver.add_service_account("netork", NETORK_KEY)
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert not any(c.startswith("openuser") for c in _root_commands(host))
|
||||
assert "already" in result["output"]
|
||||
|
||||
|
||||
def test_root_is_never_netorks_account(host: _ServiceHost, driver: _Driver) -> None:
|
||||
host.accounts["toor"] = _Account(0, "/root", groups=["wheel"])
|
||||
|
||||
with pytest.raises(ValueError, match="root"):
|
||||
driver.add_service_account("toor", NETORK_KEY)
|
||||
assert not _root_commands(host)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("name", ["Netork", "net ork", "-n", ""])
|
||||
def test_a_name_no_host_takes_is_refused(driver: _Driver, name: str) -> None:
|
||||
with pytest.raises(ValueError):
|
||||
driver.add_service_account(name, NETORK_KEY)
|
||||
|
||||
|
||||
def test_a_key_that_is_none_is_refused(driver: _Driver) -> None:
|
||||
with pytest.raises(ValueError):
|
||||
driver.add_service_account("netork", "not a key")
|
||||
|
||||
|
||||
def test_a_rule_sudo_does_not_read_fails_the_step(host: _ServiceHost, driver: _Driver) -> None:
|
||||
host.includes_dropins = False
|
||||
|
||||
result = driver.add_service_account("netork", NETORK_KEY)
|
||||
|
||||
assert not result["success"]
|
||||
sudo = next(s for s in result["steps"] if s["step"] == "sudo")
|
||||
assert not sudo["ok"] and "does not read it" in sudo["output"]
|
||||
|
||||
|
||||
def test_a_rule_with_a_password_is_not_enough(host: _ServiceHost, driver: _Driver) -> None:
|
||||
host.other_rules.add("netork")
|
||||
host.includes_dropins = False
|
||||
|
||||
result = driver.add_service_account("netork", NETORK_KEY)
|
||||
|
||||
assert not result["success"]
|
||||
|
||||
|
||||
def test_a_failed_create_stops_before_sudo_and_keys(host: _ServiceHost, driver: _Driver) -> None:
|
||||
host.failing.add("mkuser")
|
||||
|
||||
result = driver.add_service_account("netork", NETORK_KEY)
|
||||
|
||||
assert not result["success"]
|
||||
assert [s["step"] for s in result["steps"]] == ["account"]
|
||||
assert result["account"] is None
|
||||
|
||||
|
||||
def test_the_staged_files_are_removed_again(host: _ServiceHost, driver: _Driver) -> None:
|
||||
driver.add_service_account("netork", NETORK_KEY)
|
||||
|
||||
assert not [p for p in host.files if p.startswith("/tmp/")]
|
||||
|
||||
|
||||
def test_a_personal_account_of_the_same_name_keeps_its_own_rule_apart(
|
||||
host: _ServiceHost, driver: _Driver
|
||||
) -> None:
|
||||
driver.add_service_account("netork", NETORK_KEY)
|
||||
|
||||
rules = [c for c in _root_commands(host) if c.startswith("install")]
|
||||
assert [shlex.split(c)[-1] for c in rules] == [
|
||||
service_sudoers_rule_path(_Driver.SUDOERS_DIR, "netork")
|
||||
]
|
||||
|
||||
|
||||
# -- a fixed uid (infrastructure BAUSTANDARD A) ------------------------------------------
|
||||
|
||||
|
||||
class _UidDriver(_Driver):
|
||||
def _account_create_with_uid(self, name: str, uid: int) -> list[list[str]]:
|
||||
return [["mkuseruid", name, str(uid)]]
|
||||
|
||||
|
||||
def _mkuseruid(host: _ServiceHost):
|
||||
def handler(args: list[str], privileged: bool) -> CommandResult:
|
||||
assert privileged
|
||||
name, uid = args[0], int(args[1])
|
||||
host.accounts[name] = _Account(uid, f"/home/{name}", groups=[name])
|
||||
return CommandResult("", "", 0)
|
||||
|
||||
return handler
|
||||
|
||||
|
||||
def _getent_uid(host: _ServiceHost, original):
|
||||
def handler(args: list[str], privileged: bool) -> CommandResult:
|
||||
if args[1].isdigit():
|
||||
owner = next((n for n, a in host.accounts.items() if a.uid == int(args[1])), None)
|
||||
if owner is None:
|
||||
return CommandResult("", "", 2)
|
||||
a = host.accounts[owner]
|
||||
return CommandResult(f"{owner}:*:{a.uid}:{a.uid}::{a.home}:{a.shell}\n", "", 0)
|
||||
return original(args, privileged)
|
||||
|
||||
return handler
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def uid_driver(host: _ServiceHost) -> _UidDriver:
|
||||
host._mkuseruid = _mkuseruid(host) # type: ignore[attr-defined]
|
||||
host._getent = _getent_uid(host, host._getent) # type: ignore[method-assign]
|
||||
return _UidDriver(host)
|
||||
|
||||
|
||||
def test_a_new_account_gets_the_uid_asked_for(host: _ServiceHost, uid_driver: _UidDriver) -> None:
|
||||
result = uid_driver.add_service_account("netork", NETORK_KEY, uid=900)
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert host.accounts["netork"].uid == 900
|
||||
account = result["account"]
|
||||
assert account is not None and account["uid"] == 900
|
||||
|
||||
|
||||
def test_a_uid_another_account_has_stops_before_anything_is_made(
|
||||
host: _ServiceHost, uid_driver: _UidDriver
|
||||
) -> None:
|
||||
host.accounts["fwupd-refresh"] = _Account(900, "/run/fwupd")
|
||||
|
||||
result = uid_driver.add_service_account("netork", NETORK_KEY, uid=900)
|
||||
|
||||
assert not result["success"]
|
||||
assert "fwupd-refresh" in result["output"]
|
||||
assert "netork" not in host.accounts
|
||||
|
||||
|
||||
def test_an_existing_account_keeps_its_uid_and_says_so(
|
||||
host: _ServiceHost, uid_driver: _UidDriver
|
||||
) -> None:
|
||||
host.accounts["netork"] = _Account(1001, "/home/netork", groups=["netork"])
|
||||
|
||||
result = uid_driver.add_service_account("netork", NETORK_KEY, uid=900)
|
||||
|
||||
assert result["success"], result["output"]
|
||||
assert host.accounts["netork"].uid == 1001
|
||||
assert "uid 1001" in result["output"]
|
||||
|
||||
|
||||
def test_a_uid_root_has_is_refused(uid_driver: _UidDriver) -> None:
|
||||
with pytest.raises(ValueError):
|
||||
uid_driver.add_service_account("netork", NETORK_KEY, uid=0)
|
||||
|
||||
|
||||
def test_a_driver_that_cannot_set_a_uid_says_so(host: _ServiceHost, driver: _Driver) -> None:
|
||||
result = driver.add_service_account("netork", NETORK_KEY, uid=900)
|
||||
|
||||
assert not result["success"]
|
||||
assert "uid" in result["output"]
|
||||
|
||||
|
||||
# -- another service's account of the same name (bw-01: the satellite's) ------------------
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"shell", ["/usr/sbin/nologin", "/sbin/nologin", "/bin/false", "/usr/bin/false"]
|
||||
)
|
||||
def test_an_account_that_cannot_log_in_is_another_services_and_left_alone(
|
||||
host: _ServiceHost, driver: _Driver, shell: str
|
||||
) -> None:
|
||||
host.accounts["netork"] = _Account(
|
||||
999, "/home/netork", shell=shell, groups=["netork", "docker"]
|
||||
)
|
||||
|
||||
with pytest.raises(ValueError, match="cannot log in"):
|
||||
driver.add_service_account("netork", NETORK_KEY)
|
||||
assert not _root_commands(host)
|
||||
assert service_sudoers_rule_path(_Driver.SUDOERS_DIR, "netork") not in host.files
|
||||
@@ -22,7 +22,7 @@ from napalm_device_types import (
|
||||
|
||||
def test_a_profile_says_where_each_value_lives():
|
||||
assert set(get_type_hints(SnmpHealthProfile)) == {"uptime", "cpu", "memory", "interfaces"}
|
||||
assert set(get_type_hints(SnmpValueSpec)) == {"oids", "table", "fold", "pattern"}
|
||||
assert set(get_type_hints(SnmpValueSpec)) == {"oids", "table", "fold", "pattern", "unknown"}
|
||||
assert set(get_type_hints(SnmpMemorySpec)) == {"total", "used", "free", "percent", "unit"}
|
||||
assert set(get_type_hints(SnmpInterfacesSpec)) == {"skip", "tx_errors_are_drops"}
|
||||
|
||||
@@ -42,6 +42,24 @@ def test_a_driver_declares_its_profile_as_a_class_attribute():
|
||||
assert Driver.SNMP_HEALTH_PROFILE["cpu"]["oids"][0] == "1.3.6.1.4.1.9.1"
|
||||
|
||||
|
||||
def test_a_value_the_agent_gets_wrong_is_declared_unknown():
|
||||
"""Neither the standard reading nor a default: the application reports none."""
|
||||
assert get_type_hints(SnmpValueSpec)["unknown"] is bool
|
||||
|
||||
class Driver:
|
||||
SNMP_HEALTH_PROFILE: ClassVar[SnmpHealthProfile] = {"cpu": {"unknown": True}}
|
||||
|
||||
assert Driver.SNMP_HEALTH_PROFILE["cpu"].get("unknown") is True
|
||||
|
||||
|
||||
def test_the_shapes_live_in_their_own_module_and_models_still_exports_them():
|
||||
from napalm_device_types import models, snmp_health_profile
|
||||
|
||||
for name in ("SnmpHealthProfile", "SnmpValueSpec", "SnmpMemorySpec", "SnmpInterfacesSpec"):
|
||||
assert getattr(models, name) is getattr(snmp_health_profile, name), name
|
||||
assert getattr(snmp_health_profile, name).__module__ == snmp_health_profile.__name__
|
||||
|
||||
|
||||
def test_the_collector_in_the_base_package_warns_that_it_is_deprecated():
|
||||
async def get(oid):
|
||||
return None
|
||||
|
||||
@@ -0,0 +1,250 @@
|
||||
"""Exempting the managing client from sshd's per-source penalties (NetOrk/netork#1800).
|
||||
|
||||
OpenSSH 9.8 penalises a source whose connections do not authenticate and then
|
||||
drops every connection from it. A security scanner's probes earn the managing
|
||||
client exactly that, and its next session -- a poll, an Ansible run -- is
|
||||
turned away. ``exempt_from_ssh_penalties`` puts the address the host sees the
|
||||
client at into ``PerSourcePenaltyExemptList``, where that directive exists.
|
||||
|
||||
The real outputs this is held to were taken from Debian 13 (OpenSSH 10.0p2) and
|
||||
Debian 12 (OpenSSH 9.2p1, which knows no penalties and rejects the directive).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types.channel import CommandResult
|
||||
from napalm_device_types.ssh_server import (
|
||||
DROPIN_DIR,
|
||||
MAIN_CONFIG,
|
||||
PENALTY_DROPIN_PATH,
|
||||
OpenSshServerMixin,
|
||||
client_address,
|
||||
is_exempt,
|
||||
parse_penalty_exemptions,
|
||||
)
|
||||
|
||||
#: ``sshd -T | grep -i penalt`` on Debian 13, OpenSSH 10.0p2, as installed.
|
||||
DEBIAN_13 = (
|
||||
"persourcepenaltyexemptlist none\n"
|
||||
"persourcepenalties crash:90 authfail:5 noauth:1 grace-exceeded:10 refuseconnection:10"
|
||||
" max:600 min:15 max-sources4:65536 max-sources6:65536 overflow:permissive"
|
||||
" overflow6:permissive\n"
|
||||
)
|
||||
|
||||
|
||||
# ── reading ──────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_a_new_sshd_has_penalties_and_no_exemption():
|
||||
assert parse_penalty_exemptions(DEBIAN_13) == []
|
||||
|
||||
|
||||
def test_the_exemptions_sshd_uses_are_read():
|
||||
text = DEBIAN_13.replace("none", "10.40.100.10,192.0.2.0/24")
|
||||
|
||||
assert parse_penalty_exemptions(text) == ["10.40.100.10", "192.0.2.0/24"]
|
||||
|
||||
|
||||
def test_an_sshd_before_9_8_has_no_penalties():
|
||||
"""It prints neither line -- and refuses the directive, so nothing may be written."""
|
||||
assert parse_penalty_exemptions("port 22\nmacs hmac-sha2-256\n") is None
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("value", "address"),
|
||||
[
|
||||
("10.40.100.10 41866 10.40.100.118 22", "10.40.100.10"),
|
||||
("fd00::10 41866 fd00::118 22", "fd00::10"),
|
||||
("", None),
|
||||
("not-an-address 1 2 3", None),
|
||||
("10.40.100.10 41866", None),
|
||||
],
|
||||
)
|
||||
def test_the_client_address_comes_from_ssh_connection(value, address):
|
||||
assert client_address(value) == address
|
||||
|
||||
|
||||
def test_an_address_inside_an_exempt_network_is_exempt():
|
||||
assert is_exempt("10.40.100.10", ["192.0.2.0/24", "10.40.100.0/24"])
|
||||
assert not is_exempt("10.40.100.10", ["10.40.100.11", "bogus"])
|
||||
|
||||
|
||||
# ── the mixin, against a fake host ───────────────────────────────────────
|
||||
|
||||
|
||||
class PenaltyHost(OpenSshServerMixin):
|
||||
"""An sshd that reads the main config and the drop-in directory, first value wins."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
main: str = f"Include {DROPIN_DIR}/*.conf\nPort 22\n",
|
||||
penalties: bool = True,
|
||||
client: str = "10.40.100.10 41866 10.40.100.118 22",
|
||||
) -> None:
|
||||
self.files: dict[str, str] = {MAIN_CONFIG: main}
|
||||
self.penalties = penalties
|
||||
self.client = client
|
||||
self.syntax_ok = True
|
||||
self.reloads = 0
|
||||
self.ran: list[str] = []
|
||||
|
||||
def _config_lines(self) -> list[str]:
|
||||
lines: list[str] = []
|
||||
for line in self.files[MAIN_CONFIG].splitlines():
|
||||
if re.match(r"\s*include\s", line, re.I):
|
||||
for path in sorted(p for p in self.files if p.startswith(DROPIN_DIR + "/")):
|
||||
lines += self.files[path].splitlines()
|
||||
elif re.match(r"\s*match\s", line, re.I):
|
||||
break
|
||||
else:
|
||||
lines.append(line)
|
||||
return lines
|
||||
|
||||
def exempt(self) -> str:
|
||||
for line in self._config_lines():
|
||||
parts = line.split()
|
||||
if len(parts) == 2 and parts[0].lower() == "persourcepenaltyexemptlist":
|
||||
return parts[1]
|
||||
return "none"
|
||||
|
||||
def run_command(self, command, *, privileged=False, timeout=60, stdin=None):
|
||||
self.ran.append(command)
|
||||
if "SSH_CONNECTION" in command:
|
||||
assert not privileged, "it is the login session's, not sudo's"
|
||||
return CommandResult(self.client, "", 0)
|
||||
if "mktemp" in command:
|
||||
path = f"/tmp/netork-ssh.{len(self.files)}"
|
||||
self.files[path] = (stdin or b"").decode()
|
||||
return CommandResult(path, "", 0)
|
||||
assert privileged, command
|
||||
m = re.search(r"install -m \"\$m\" -- '?([^' ]+)'? '?([^' ]+)'?", command)
|
||||
if m:
|
||||
target = re.search(r"mv -f -- '?[^' ]+'? '?([^' ;]+)'?", command).group(1)
|
||||
self.files[target] = self.files.pop(m.group(1))
|
||||
return CommandResult("", "", 0)
|
||||
if "sshd -T" in command:
|
||||
if not self.penalties:
|
||||
return CommandResult("port 22\n", "", 0)
|
||||
return CommandResult(DEBIAN_13.replace("none", self.exempt()), "", 0)
|
||||
if "sshd -t" in command:
|
||||
return CommandResult(
|
||||
"", "" if self.syntax_ok else "bad line 1", 0 if self.syntax_ok else 255
|
||||
)
|
||||
m = re.match(r"cat -- '?([^' ]+)'?$", command)
|
||||
if m:
|
||||
path = m.group(1)
|
||||
if path in self.files:
|
||||
return CommandResult(self.files[path], "", 0)
|
||||
return CommandResult("", "No such file", 1)
|
||||
m = re.match(r"rm -f -- '?([^' ]+)'?$", command)
|
||||
if m:
|
||||
self.files.pop(m.group(1), None)
|
||||
return CommandResult("", "", 0)
|
||||
if "reload" in command or "HUP" in command:
|
||||
self.reloads += 1
|
||||
return CommandResult("", "", 0)
|
||||
if command.startswith("mkdir"):
|
||||
return CommandResult("", "", 0)
|
||||
raise AssertionError(f"unexpected command: {command}")
|
||||
|
||||
|
||||
def test_the_client_address_is_exempted_in_a_dropin():
|
||||
host = PenaltyHost()
|
||||
lines: list[str] = []
|
||||
|
||||
result = host.exempt_from_ssh_penalties(log=lines.append)
|
||||
|
||||
assert result.applied and not result.rolled_back
|
||||
assert result.address == "10.40.100.10"
|
||||
assert result.path == PENALTY_DROPIN_PATH
|
||||
assert host.exempt() == "10.40.100.10"
|
||||
assert host.reloads == 1
|
||||
assert lines
|
||||
|
||||
|
||||
def test_the_hosts_own_exemptions_stay():
|
||||
host = PenaltyHost(
|
||||
main=f"Include {DROPIN_DIR}/*.conf\nPerSourcePenaltyExemptList 192.0.2.0/24\n"
|
||||
)
|
||||
|
||||
result = host.exempt_from_ssh_penalties()
|
||||
|
||||
assert result.applied
|
||||
assert host.exempt() == "192.0.2.0/24,10.40.100.10"
|
||||
|
||||
|
||||
def test_nothing_is_written_to_an_sshd_without_penalties():
|
||||
"""OpenSSH before 9.8 does not start with the directive in its config."""
|
||||
host = PenaltyHost(penalties=False)
|
||||
|
||||
result = host.exempt_from_ssh_penalties()
|
||||
|
||||
assert not result.applied and not result.rolled_back
|
||||
assert PENALTY_DROPIN_PATH not in host.files
|
||||
assert host.reloads == 0
|
||||
assert "9.8" in result.message
|
||||
|
||||
|
||||
def test_nothing_changes_when_the_address_is_already_exempt():
|
||||
host = PenaltyHost(
|
||||
main=f"Include {DROPIN_DIR}/*.conf\nPerSourcePenaltyExemptList 10.40.100.0/24\n"
|
||||
)
|
||||
|
||||
result = host.exempt_from_ssh_penalties()
|
||||
|
||||
assert not result.applied
|
||||
assert PENALTY_DROPIN_PATH not in host.files
|
||||
assert "already" in result.message
|
||||
|
||||
|
||||
def test_an_sshd_config_without_the_include_is_left_alone():
|
||||
host = PenaltyHost(main="Port 22\n")
|
||||
|
||||
result = host.exempt_from_ssh_penalties()
|
||||
|
||||
assert not result.applied
|
||||
assert host.files[MAIN_CONFIG] == "Port 22\n"
|
||||
assert DROPIN_DIR in result.message
|
||||
|
||||
|
||||
def test_a_rejected_config_is_taken_back():
|
||||
host = PenaltyHost()
|
||||
host.syntax_ok = False
|
||||
|
||||
result = host.exempt_from_ssh_penalties()
|
||||
|
||||
assert result.rolled_back and not result.applied
|
||||
assert PENALTY_DROPIN_PATH not in host.files
|
||||
|
||||
|
||||
def test_an_exemption_sshd_does_not_use_is_taken_back():
|
||||
"""A drop-in read before ours sets the directive; ours is valid and ignored."""
|
||||
host = PenaltyHost()
|
||||
host.files[f"{DROPIN_DIR}/00-aaa.conf"] = "PerSourcePenaltyExemptList 192.0.2.1\n"
|
||||
|
||||
result = host.exempt_from_ssh_penalties()
|
||||
|
||||
assert result.rolled_back
|
||||
assert PENALTY_DROPIN_PATH not in host.files
|
||||
|
||||
|
||||
def test_it_is_taken_back_when_a_new_session_cannot_get_in():
|
||||
host = PenaltyHost()
|
||||
|
||||
result = host.exempt_from_ssh_penalties(verify=lambda: False)
|
||||
|
||||
assert result.rolled_back
|
||||
assert PENALTY_DROPIN_PATH not in host.files
|
||||
assert host.reloads == 2
|
||||
|
||||
|
||||
def test_a_session_without_a_client_address_raises():
|
||||
host = PenaltyHost(client="")
|
||||
|
||||
with pytest.raises(RuntimeError):
|
||||
host.exempt_from_ssh_penalties()
|
||||
@@ -0,0 +1,168 @@
|
||||
"""A connection sshd turns away before the SSH banner is tried again (NetOrk/netork#1800).
|
||||
|
||||
OpenSSH 9.8+ drops every connection from a source it has just penalised
|
||||
(``PerSourcePenalties``): it accepts the TCP connection and closes it at once,
|
||||
and paramiko reports "Error reading SSH protocol banner". The penalty lasts
|
||||
seconds to minutes, so a connection that waits a little gets in.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import socket
|
||||
import threading
|
||||
|
||||
import paramiko
|
||||
import pytest
|
||||
from napalm.base.exceptions import ConnectionException
|
||||
|
||||
from napalm_device_types import ssh_host_keys
|
||||
from napalm_device_types.ssh_host_keys import (
|
||||
REFUSED_RETRY_WAITS,
|
||||
HostKeyCheck,
|
||||
refused_before_banner,
|
||||
retry_refused,
|
||||
)
|
||||
|
||||
|
||||
def _banner_error() -> paramiko.SSHException:
|
||||
try:
|
||||
try:
|
||||
raise EOFError()
|
||||
except EOFError as eof:
|
||||
raise paramiko.SSHException("Error reading SSH protocol banner") from eof
|
||||
except paramiko.SSHException as exc:
|
||||
return exc
|
||||
|
||||
|
||||
class TestWhatCountsAsRefused:
|
||||
def test_paramikos_banner_error(self):
|
||||
assert refused_before_banner(_banner_error())
|
||||
|
||||
def test_wrapped_by_a_driver(self):
|
||||
try:
|
||||
try:
|
||||
raise _banner_error()
|
||||
except paramiko.SSHException as exc:
|
||||
raise ConnectionException(f"cannot connect: {exc}") from exc
|
||||
except ConnectionException as wrapped:
|
||||
assert refused_before_banner(wrapped)
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"exc",
|
||||
[
|
||||
paramiko.AuthenticationException("Authentication failed."),
|
||||
TimeoutError("timed out"),
|
||||
ConnectionRefusedError(111, "Connection refused"),
|
||||
ValueError("anything else"),
|
||||
],
|
||||
)
|
||||
def test_not_a_failed_login_a_silent_host_or_a_closed_port(self, exc):
|
||||
"""Those do not end by waiting: retrying them only delays the error."""
|
||||
assert not refused_before_banner(exc)
|
||||
|
||||
|
||||
class TestRetry:
|
||||
def test_it_waits_and_tries_again(self):
|
||||
slept: list[float] = []
|
||||
attempts = iter([_banner_error(), _banner_error(), "connected"])
|
||||
|
||||
def attempt():
|
||||
value = next(attempts)
|
||||
if isinstance(value, BaseException):
|
||||
raise value
|
||||
return value
|
||||
|
||||
assert retry_refused(attempt, sleep=slept.append) == "connected"
|
||||
assert slept == list(REFUSED_RETRY_WAITS[:2])
|
||||
|
||||
def test_it_gives_up_with_the_last_error(self):
|
||||
slept: list[float] = []
|
||||
|
||||
def attempt():
|
||||
raise _banner_error()
|
||||
|
||||
with pytest.raises(paramiko.SSHException):
|
||||
retry_refused(attempt, sleep=slept.append)
|
||||
assert slept == list(REFUSED_RETRY_WAITS)
|
||||
|
||||
def test_other_errors_are_not_retried(self):
|
||||
slept: list[float] = []
|
||||
|
||||
def attempt():
|
||||
raise paramiko.AuthenticationException("Authentication failed.")
|
||||
|
||||
with pytest.raises(paramiko.AuthenticationException):
|
||||
retry_refused(attempt, sleep=slept.append)
|
||||
assert slept == []
|
||||
|
||||
def test_the_waits_outlast_the_penalty_seen(self):
|
||||
"""16 s on a new Ubuntu 26.04 VM on .50."""
|
||||
assert sum(REFUSED_RETRY_WAITS) > 16
|
||||
|
||||
|
||||
class _PenalisingServer:
|
||||
"""A TCP server that closes the first *drops* connections at once, as a penalised
|
||||
source's are, and then speaks SSH with any password."""
|
||||
|
||||
def __init__(self, drops: int) -> None:
|
||||
self.drops = drops
|
||||
self.listener = socket.create_server(("127.0.0.1", 0))
|
||||
self.port = self.listener.getsockname()[1]
|
||||
self.key = paramiko.ECDSAKey.generate()
|
||||
self._thread = threading.Thread(target=self._serve, daemon=True)
|
||||
self._thread.start()
|
||||
|
||||
def _serve(self) -> None:
|
||||
while True:
|
||||
try:
|
||||
conn, _ = self.listener.accept()
|
||||
except OSError:
|
||||
return
|
||||
if self.drops:
|
||||
self.drops -= 1
|
||||
conn.close()
|
||||
continue
|
||||
transport = paramiko.Transport(conn)
|
||||
transport.add_server_key(self.key)
|
||||
|
||||
class _Any(paramiko.ServerInterface):
|
||||
def get_allowed_auths(self, username):
|
||||
return "password"
|
||||
|
||||
def check_auth_password(self, username, password):
|
||||
return paramiko.AUTH_SUCCESSFUL
|
||||
|
||||
try:
|
||||
transport.start_server(server=_Any())
|
||||
except Exception:
|
||||
pass
|
||||
self.transport = transport
|
||||
return
|
||||
|
||||
def close(self) -> None:
|
||||
self.listener.close()
|
||||
transport = getattr(self, "transport", None)
|
||||
if transport:
|
||||
transport.close()
|
||||
|
||||
|
||||
def test_host_key_check_connects_once_the_penalty_is_over(monkeypatch):
|
||||
server = _PenalisingServer(drops=2)
|
||||
slept: list[float] = []
|
||||
monkeypatch.setattr(ssh_host_keys.time, "sleep", slept.append)
|
||||
client = paramiko.SSHClient()
|
||||
try:
|
||||
HostKeyCheck({}, "127.0.0.1", server.port).connect(
|
||||
client,
|
||||
username="admin",
|
||||
password="s3cret",
|
||||
look_for_keys=False,
|
||||
allow_agent=False,
|
||||
timeout=5,
|
||||
banner_timeout=5,
|
||||
)
|
||||
assert client.get_transport().is_authenticated()
|
||||
assert slept == list(REFUSED_RETRY_WAITS[:2])
|
||||
finally:
|
||||
client.close()
|
||||
server.close()
|
||||
@@ -14,6 +14,7 @@ import pytest
|
||||
|
||||
from napalm_device_types.channel import CommandResult
|
||||
from napalm_device_types.ssh_server import (
|
||||
DROPIN_DIR,
|
||||
DROPIN_PATH,
|
||||
MAIN_CONFIG,
|
||||
OpenSshServerMixin,
|
||||
@@ -401,6 +402,32 @@ def test_mixin_falls_back_to_sshd_config_without_an_include():
|
||||
assert host._effective()["mac"] == ["hmac-sha2-256"]
|
||||
|
||||
|
||||
class GeneratedConfigHost(FakeHost):
|
||||
"""A platform that writes sshd_config from its own database (OpenMediaVault)."""
|
||||
|
||||
SSHD_CONFIG_GENERATED = True
|
||||
|
||||
|
||||
def test_generated_sshd_config_still_gets_the_dropin():
|
||||
host = GeneratedConfigHost()
|
||||
result = host.harden_ssh_server({"mac": ["hmac-sha1"]})
|
||||
assert result.applied
|
||||
assert result.path == DROPIN_PATH
|
||||
assert host.files[MAIN_CONFIG] == "Include /etc/ssh/sshd_config.d/*.conf\nPort 22\n"
|
||||
|
||||
|
||||
def test_generated_sshd_config_without_an_include_is_left_alone():
|
||||
"""The platform would write its own file over the change on its next apply."""
|
||||
host = GeneratedConfigHost(main="Port 22\nMACs hmac-sha1,hmac-sha2-256\n")
|
||||
result = host.harden_ssh_server({"mac": ["hmac-sha1"]})
|
||||
assert not result.applied and not result.rolled_back
|
||||
assert result.path is None
|
||||
assert host.files[MAIN_CONFIG] == "Port 22\nMACs hmac-sha1,hmac-sha2-256\n"
|
||||
assert list(host.files) == [MAIN_CONFIG]
|
||||
assert host.reloads == 0
|
||||
assert DROPIN_DIR in result.message
|
||||
|
||||
|
||||
def test_mixin_restores_when_sshd_rejects_the_config():
|
||||
host = FakeHost()
|
||||
host.syntax_ok = False
|
||||
|
||||
@@ -0,0 +1,395 @@
|
||||
"""The sudo password an earlier driver left in shell histories, and taking it back.
|
||||
|
||||
The scripts are run for real, with ``sh``, against a home directory in a temp
|
||||
directory: what they count, delete and report is what a host would see. Root's
|
||||
part -- giving files back to root -- runs with a ``chown`` that only records,
|
||||
so the test needs no root.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import getpass
|
||||
import os
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types.channel import CommandResult
|
||||
from napalm_device_types.sudo_history import HISTORY_BACKUP_SUFFIX, SudoHistoryMixin
|
||||
|
||||
MARKER = "| sudo -S -p "
|
||||
|
||||
LEAKED = "echo 'hunter2' | sudo -S -p \"\" apt-get update\n"
|
||||
OTHER = "ls -la\n"
|
||||
|
||||
|
||||
class _Host(SudoHistoryMixin):
|
||||
"""A driver whose host is this machine, with *home* as the login user's home."""
|
||||
|
||||
SUDO_HISTORY_MARKER = MARKER
|
||||
|
||||
def __init__(self, home: Path, bin_dir: Path, *, root: bool = False) -> None:
|
||||
self.home = home
|
||||
self.bin_dir = bin_dir
|
||||
self.root = root
|
||||
#: Who ``id -un`` says the login user is; None for this machine's user name.
|
||||
self.user: str | None = None
|
||||
self.ran: list[tuple[str, bool]] = []
|
||||
|
||||
def run_command(
|
||||
self,
|
||||
command: str,
|
||||
*,
|
||||
privileged: bool = False,
|
||||
timeout: float = 60,
|
||||
stdin: bytes | None = None,
|
||||
) -> CommandResult:
|
||||
# The password goes ahead of a privileged command's stdin: there must be none.
|
||||
assert not (privileged and stdin is not None)
|
||||
self.ran.append((command, privileged))
|
||||
env = {
|
||||
**os.environ,
|
||||
"HOME": str(self.home),
|
||||
"PATH": f"{self.bin_dir}:{os.environ['PATH']}",
|
||||
}
|
||||
if command.strip() == "id -u; id -un" and self.root:
|
||||
return CommandResult("0\nroot\n", "", 0)
|
||||
if command.strip() == "id -u; id -un":
|
||||
# Never this machine's uid: CI runs as root, which the owner step skips.
|
||||
return CommandResult(f"4242\n{self.user or getpass.getuser()}\n", "", 0)
|
||||
done = subprocess.run( # noqa: S603 - the test's own scripts
|
||||
["sh", "-c", command], capture_output=True, text=True, env=env, timeout=timeout
|
||||
)
|
||||
return CommandResult(done.stdout, done.stderr, done.returncode)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def bin_dir(tmp_path: Path) -> Path:
|
||||
"""A ``chown`` that records its arguments instead of changing anything."""
|
||||
d = tmp_path / "bin"
|
||||
d.mkdir()
|
||||
chown = d / "chown"
|
||||
chown.write_text(f'#!/bin/sh\nprintf "%s\\n" "$*" >> {tmp_path / "chowned"}\n')
|
||||
chown.chmod(0o755)
|
||||
return d
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def home(tmp_path: Path) -> Path:
|
||||
h = tmp_path / "home"
|
||||
h.mkdir()
|
||||
return h
|
||||
|
||||
|
||||
def _chowned(tmp_path: Path) -> list[str]:
|
||||
log = tmp_path / "chowned"
|
||||
return log.read_text().splitlines() if log.exists() else []
|
||||
|
||||
|
||||
class TestCounting:
|
||||
def test_counts_the_lines_per_history_file_without_reading_them_out(self, home, bin_dir):
|
||||
(home / ".bash_history").write_text(OTHER + LEAKED * 3)
|
||||
(home / ".zsh_history").write_text(": 1700000000:0;" + LEAKED)
|
||||
(home / ".sh_history").write_text(OTHER)
|
||||
host = _Host(home, bin_dir)
|
||||
|
||||
found = host.get_history_sudo_lines()
|
||||
|
||||
assert found == [
|
||||
{"file": "~/.bash_history", "lines": 3},
|
||||
{"file": "~/.zsh_history", "lines": 1},
|
||||
]
|
||||
# Counted as the login user, and nothing of the content came back.
|
||||
assert all(not privileged for _cmd, privileged in host.ran)
|
||||
assert "hunter2" not in repr(found)
|
||||
|
||||
def test_no_history_no_lines(self, home, bin_dir):
|
||||
assert _Host(home, bin_dir).get_history_sudo_lines() == []
|
||||
|
||||
def test_the_pattern_is_a_fixed_string(self, home, bin_dir):
|
||||
# "-S" is no regex trouble, but "|" would be an alternation in an ERE.
|
||||
(home / ".bash_history").write_text("sudo -S -p\n| sudo\n")
|
||||
assert _Host(home, bin_dir).get_history_sudo_lines() == []
|
||||
|
||||
def test_a_driver_that_never_typed_the_password_counts_nothing(self, home, bin_dir):
|
||||
(home / ".bash_history").write_text(LEAKED)
|
||||
host = _Host(home, bin_dir)
|
||||
host.SUDO_HISTORY_MARKER = None
|
||||
|
||||
assert host.get_history_sudo_lines() == []
|
||||
assert host.ran == []
|
||||
|
||||
def test_an_unreadable_history_raises_rather_than_reporting_none(self, home, bin_dir):
|
||||
if os.geteuid() == 0:
|
||||
pytest.skip("root reads every file")
|
||||
history = home / ".bash_history"
|
||||
history.write_text(LEAKED)
|
||||
history.chmod(0)
|
||||
try:
|
||||
with pytest.raises(RuntimeError):
|
||||
_Host(home, bin_dir).get_history_sudo_lines()
|
||||
finally:
|
||||
history.chmod(0o600)
|
||||
|
||||
|
||||
class TestRemoving:
|
||||
def test_deletes_exactly_those_lines_and_says_how_many(self, home, bin_dir):
|
||||
(home / ".bash_history").write_text(OTHER + LEAKED + "cd /etc\n" + LEAKED)
|
||||
(home / ".ash_history").write_text(LEAKED)
|
||||
host = _Host(home, bin_dir)
|
||||
|
||||
result = host.remove_history_sudo_lines()
|
||||
|
||||
assert result["success"] is True
|
||||
assert (home / ".bash_history").read_text() == OTHER + "cd /etc\n"
|
||||
assert (home / ".ash_history").read_text() == ""
|
||||
assert result["removed"] == 3
|
||||
assert result["files"] == [
|
||||
{"file": "~/.bash_history", "before": 2, "after": 0},
|
||||
{"file": "~/.ash_history", "before": 1, "after": 0},
|
||||
]
|
||||
assert "~/.bash_history: 2 lines removed (2 before, 0 after)" in result["output"]
|
||||
|
||||
def test_no_backup_with_the_password_is_left_behind(self, home, bin_dir):
|
||||
(home / ".bash_history").write_text(LEAKED)
|
||||
|
||||
_Host(home, bin_dir).remove_history_sudo_lines()
|
||||
|
||||
assert sorted(p.name for p in home.iterdir()) == [".bash_history"]
|
||||
assert not (home / f".bash_history{HISTORY_BACKUP_SUFFIX}").exists()
|
||||
|
||||
def test_the_history_runs_as_the_login_user(self, home, bin_dir):
|
||||
(home / ".bash_history").write_text(LEAKED)
|
||||
host = _Host(home, bin_dir)
|
||||
|
||||
host.remove_history_sudo_lines()
|
||||
|
||||
history_scripts = [(c, p) for c, p in host.ran if ".bash_history" in c]
|
||||
assert history_scripts and all(not p for _c, p in history_scripts)
|
||||
|
||||
def test_a_clean_history_is_left_as_it_is(self, home, bin_dir):
|
||||
history = home / ".bash_history"
|
||||
history.write_text(OTHER)
|
||||
before = history.stat().st_ino
|
||||
|
||||
result = _Host(home, bin_dir).remove_history_sudo_lines()
|
||||
|
||||
assert result["success"] is True
|
||||
assert result["removed"] == 0
|
||||
assert result["files"] == [{"file": "~/.bash_history", "before": 0, "after": 0}]
|
||||
assert history.stat().st_ino == before # not rewritten
|
||||
|
||||
def test_a_history_that_is_a_link_is_left_alone_and_the_fix_fails(
|
||||
self, home, bin_dir, tmp_path
|
||||
):
|
||||
target = tmp_path / "elsewhere"
|
||||
target.write_text(LEAKED)
|
||||
(home / ".bash_history").symlink_to(target)
|
||||
|
||||
result = _Host(home, bin_dir).remove_history_sudo_lines()
|
||||
|
||||
assert result["success"] is False
|
||||
assert target.read_text() == LEAKED
|
||||
assert (home / ".bash_history").is_symlink()
|
||||
assert "~/.bash_history: a symbolic link, left alone (1 line)" in result["output"]
|
||||
|
||||
def test_the_lines_of_a_driver_that_never_typed_the_password_are_not_looked_for(
|
||||
self, home, bin_dir
|
||||
):
|
||||
host = _Host(home, bin_dir)
|
||||
host.SUDO_HISTORY_MARKER = None
|
||||
|
||||
result = host.remove_history_sudo_lines()
|
||||
|
||||
assert result["success"] is True
|
||||
assert result["removed"] == 0
|
||||
|
||||
def test_an_unreadable_history_fails_the_step(self, home, bin_dir):
|
||||
if os.geteuid() == 0:
|
||||
pytest.skip("root reads every file")
|
||||
history = home / ".bash_history"
|
||||
history.write_text(LEAKED)
|
||||
history.chmod(0)
|
||||
try:
|
||||
result = _Host(home, bin_dir).remove_history_sudo_lines()
|
||||
finally:
|
||||
history.chmod(0o600)
|
||||
|
||||
assert result["success"] is False
|
||||
assert "~/.bash_history: cannot be read" in result["output"]
|
||||
|
||||
def test_a_sed_that_fails_keeps_the_lines_and_leaves_no_backup(self, home, bin_dir):
|
||||
(bin_dir / "sed").write_text(
|
||||
'#!/bin/sh\nfor a; do f="$a"; done\ncp "$f" "$f.napalm-bak"\nexit 4\n'
|
||||
)
|
||||
(bin_dir / "sed").chmod(0o755)
|
||||
(home / ".bash_history").write_text(LEAKED * 2)
|
||||
|
||||
result = _Host(home, bin_dir).remove_history_sudo_lines()
|
||||
|
||||
assert result["success"] is False
|
||||
assert "~/.bash_history: could not be rewritten (2 lines kept)" in result["output"]
|
||||
assert sorted(p.name for p in home.iterdir()) == [".bash_history"]
|
||||
|
||||
def test_a_sed_that_lost_the_file_gets_it_back(self, home, bin_dir):
|
||||
(bin_dir / "sed").write_text(
|
||||
'#!/bin/sh\nfor a; do f="$a"; done\nmv "$f" "$f.napalm-bak"\nexit 4\n'
|
||||
)
|
||||
(bin_dir / "sed").chmod(0o755)
|
||||
(home / ".bash_history").write_text(OTHER + LEAKED)
|
||||
|
||||
result = _Host(home, bin_dir).remove_history_sudo_lines()
|
||||
|
||||
assert result["success"] is False
|
||||
assert (home / ".bash_history").read_text() == OTHER + LEAKED
|
||||
assert sorted(p.name for p in home.iterdir()) == [".bash_history"]
|
||||
|
||||
def test_a_script_that_dies_says_why(self, home, bin_dir):
|
||||
host = _Host(home, bin_dir)
|
||||
host.run_command = lambda command, **kw: CommandResult("", "sh: killed", 137) # type: ignore[method-assign]
|
||||
|
||||
result = host.remove_history_sudo_lines()
|
||||
|
||||
assert result["success"] is False
|
||||
assert "sh: killed" in result["output"]
|
||||
|
||||
def test_a_line_it_does_not_know_is_ignored(self):
|
||||
files: list = []
|
||||
said: list[str] = []
|
||||
|
||||
assert SudoHistoryMixin._history_line(["what", "ever"], files, said) is True
|
||||
assert SudoHistoryMixin._history_line([], files, said) is True
|
||||
assert files == [] and said == []
|
||||
|
||||
|
||||
class TestOwners:
|
||||
def _host(self, home, bin_dir, files):
|
||||
host = _Host(home, bin_dir)
|
||||
host.SUDO_ROOT_FILES = tuple(str(f) for f in files)
|
||||
return host
|
||||
|
||||
def test_a_file_the_login_user_owns_goes_back_to_root(self, home, bin_dir, tmp_path):
|
||||
conf = tmp_path / "snmpd.conf"
|
||||
conf.write_text("x")
|
||||
|
||||
result = self._host(home, bin_dir, [conf]).remove_history_sudo_lines()
|
||||
|
||||
assert _chowned(tmp_path) == [f"-h 0:0 {conf}"]
|
||||
assert result["owners_restored"] == [str(conf)]
|
||||
assert result["success"] is True
|
||||
|
||||
def test_only_as_root_and_without_stdin(self, home, bin_dir, tmp_path):
|
||||
conf = tmp_path / "snmpd.conf"
|
||||
conf.write_text("x")
|
||||
host = self._host(home, bin_dir, [conf])
|
||||
|
||||
host.remove_history_sudo_lines()
|
||||
|
||||
privileged = [c for c, p in host.ran if p]
|
||||
assert len(privileged) == 1 and "chown" in privileged[0]
|
||||
# The login user's name is quoted into root's script, never taken from stdin.
|
||||
assert getpass.getuser() in privileged[0]
|
||||
|
||||
def test_a_link_or_a_missing_file_is_not_touched(self, home, bin_dir, tmp_path):
|
||||
real = tmp_path / "real"
|
||||
real.write_text("x")
|
||||
link = tmp_path / "00proxy"
|
||||
link.symlink_to(real)
|
||||
|
||||
result = self._host(home, bin_dir, [link, tmp_path / "missing"]).remove_history_sudo_lines()
|
||||
|
||||
assert _chowned(tmp_path) == []
|
||||
assert result["owners_restored"] == []
|
||||
assert result["success"] is True
|
||||
|
||||
def test_a_file_someone_else_owns_is_not_touched(self, home, bin_dir, tmp_path):
|
||||
conf = tmp_path / "snmpd.conf"
|
||||
conf.write_text("x")
|
||||
host = self._host(home, bin_dir, [conf])
|
||||
host.user = "nobody-here-4242"
|
||||
|
||||
result = host.remove_history_sudo_lines()
|
||||
|
||||
assert _chowned(tmp_path) == []
|
||||
assert result["owners_restored"] == []
|
||||
|
||||
def test_a_root_login_has_nothing_to_give_back(self, home, bin_dir, tmp_path):
|
||||
conf = tmp_path / "snmpd.conf"
|
||||
conf.write_text("x")
|
||||
host = self._host(home, bin_dir, [conf])
|
||||
host.root = True
|
||||
|
||||
result = host.remove_history_sudo_lines()
|
||||
|
||||
assert _chowned(tmp_path) == []
|
||||
assert result["success"] is True
|
||||
|
||||
def test_the_owners_are_set_even_when_the_history_could_not_be_cleaned(
|
||||
self, home, bin_dir, tmp_path
|
||||
):
|
||||
target = tmp_path / "elsewhere"
|
||||
target.write_text(LEAKED)
|
||||
(home / ".bash_history").symlink_to(target)
|
||||
conf = tmp_path / "snmpd.conf"
|
||||
conf.write_text("x")
|
||||
|
||||
result = self._host(home, bin_dir, [conf]).remove_history_sudo_lines()
|
||||
|
||||
assert result["success"] is False
|
||||
assert [s["step"] for s in result["steps"]] == ["history", "owners"]
|
||||
assert result["owners_restored"] == [str(conf)]
|
||||
|
||||
def test_without_knowing_the_login_user_nothing_changes_owner(self, home, bin_dir, tmp_path):
|
||||
conf = tmp_path / "snmpd.conf"
|
||||
conf.write_text("x")
|
||||
host = self._host(home, bin_dir, [conf])
|
||||
real = host.run_command
|
||||
|
||||
def run(command, **kw):
|
||||
if command.startswith("id "):
|
||||
return CommandResult("", "id: not found", 127)
|
||||
return real(command, **kw)
|
||||
|
||||
host.run_command = run # type: ignore[method-assign]
|
||||
|
||||
result = host.remove_history_sudo_lines()
|
||||
|
||||
assert result["success"] is False
|
||||
assert "cannot tell who the login user is" in result["output"]
|
||||
assert _chowned(tmp_path) == []
|
||||
|
||||
def test_a_sudo_that_refuses_fails_the_step(self, home, bin_dir, tmp_path):
|
||||
conf = tmp_path / "snmpd.conf"
|
||||
conf.write_text("x")
|
||||
host = self._host(home, bin_dir, [conf])
|
||||
real = host.run_command
|
||||
|
||||
def run(command, *, privileged=False, **kw):
|
||||
if privileged:
|
||||
return CommandResult("", "sudo: a password is required", 1)
|
||||
return real(command, privileged=privileged, **kw)
|
||||
|
||||
host.run_command = run # type: ignore[method-assign]
|
||||
|
||||
result = host.remove_history_sudo_lines()
|
||||
|
||||
assert result["success"] is False
|
||||
assert "sudo: a password is required" in result["output"]
|
||||
|
||||
def test_a_refused_chown_fails_the_step(self, home, bin_dir, tmp_path):
|
||||
(bin_dir / "chown").write_text("#!/bin/sh\necho 'Operation not permitted' >&2\nexit 1\n")
|
||||
conf = tmp_path / "snmpd.conf"
|
||||
conf.write_text("x")
|
||||
|
||||
result = self._host(home, bin_dir, [conf]).remove_history_sudo_lines()
|
||||
|
||||
assert result["success"] is False
|
||||
assert result["owners_restored"] == []
|
||||
assert f"{conf}: could not be given back to root" in result["output"]
|
||||
|
||||
|
||||
def test_exported_from_the_package():
|
||||
import napalm_device_types
|
||||
|
||||
assert napalm_device_types.SudoHistoryMixin is SudoHistoryMixin
|
||||
@@ -0,0 +1,285 @@
|
||||
"""TLS certificates: trust on first use by fingerprint, a changed one refused before any login.
|
||||
|
||||
The layer above keeps the store (NetOrk/netork#1167, NAPALM/napalm-hpe-aruba-procurve#17).
|
||||
A driver gets the fingerprint it should expect through ``optional_args["tls_fingerprints"]``
|
||||
(by port) and reports every certificate it sees through
|
||||
``optional_args["tls_certificate_observer"]``. The handshake tests run real TLS against an
|
||||
in-process server on a socket pair: what matters is the certificate the server presents in
|
||||
a real handshake, which a fake could not show.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime
|
||||
import hashlib
|
||||
import socket
|
||||
import ssl
|
||||
import threading
|
||||
|
||||
import pytest
|
||||
import requests
|
||||
from cryptography import x509
|
||||
from cryptography.hazmat.primitives import hashes, serialization
|
||||
from cryptography.hazmat.primitives.asymmetric import ec
|
||||
from cryptography.x509.oid import NameOID
|
||||
from napalm.base.exceptions import ConnectionException
|
||||
|
||||
from napalm_device_types.tls_certificates import (
|
||||
KNOWN_FINGERPRINTS_ARG,
|
||||
OBSERVER_ARG,
|
||||
CertificateCheck,
|
||||
TlsCertificateMismatch,
|
||||
fetch_certificate,
|
||||
fingerprint,
|
||||
normalise_fingerprint,
|
||||
)
|
||||
|
||||
DER = b"not really a certificate, but bytes are bytes"
|
||||
DER_HEX = hashlib.sha256(DER).hexdigest().upper()
|
||||
DER_FP = "SHA256:" + ":".join(DER_HEX[i : i + 2] for i in range(0, 64, 2))
|
||||
OTHER_FP = "SHA256:" + ":".join(["AB"] * 32)
|
||||
|
||||
|
||||
def _self_signed(tmp_path, name: str = "switch") -> tuple[bytes, str, str]:
|
||||
"""A self-signed certificate, as a ProCurve ships one: (DER, cert file, key file)."""
|
||||
key = ec.generate_private_key(ec.SECP256R1())
|
||||
subject = x509.Name([x509.NameAttribute(NameOID.COMMON_NAME, name)])
|
||||
now = datetime.datetime.now(datetime.UTC)
|
||||
cert = (
|
||||
x509.CertificateBuilder()
|
||||
.subject_name(subject)
|
||||
.issuer_name(subject)
|
||||
.public_key(key.public_key())
|
||||
.serial_number(x509.random_serial_number())
|
||||
.not_valid_before(now - datetime.timedelta(days=1))
|
||||
.not_valid_after(now + datetime.timedelta(days=1))
|
||||
.sign(key, hashes.SHA256())
|
||||
)
|
||||
cert_file, key_file = tmp_path / f"{name}.crt", tmp_path / f"{name}.key"
|
||||
cert_file.write_bytes(cert.public_bytes(serialization.Encoding.PEM))
|
||||
key_file.write_bytes(
|
||||
key.private_bytes(
|
||||
serialization.Encoding.PEM,
|
||||
serialization.PrivateFormat.PKCS8,
|
||||
serialization.NoEncryption(),
|
||||
)
|
||||
)
|
||||
return cert.public_bytes(serialization.Encoding.DER), str(cert_file), str(key_file)
|
||||
|
||||
|
||||
class _TlsServer:
|
||||
"""A TLS server on one end of a socket pair; it handshakes and then waits."""
|
||||
|
||||
def __init__(self, cert_file: str, key_file: str) -> None:
|
||||
self.client_sock, server_sock = socket.socketpair()
|
||||
context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
|
||||
context.load_cert_chain(cert_file, key_file)
|
||||
self._server_sock = server_sock
|
||||
self._context = context
|
||||
self.received = b""
|
||||
self._thread = threading.Thread(target=self._serve, daemon=True)
|
||||
self._thread.start()
|
||||
|
||||
def _serve(self) -> None:
|
||||
try:
|
||||
tls = self._context.wrap_socket(self._server_sock, server_side=True)
|
||||
self.received = tls.recv(1024)
|
||||
except (OSError, ssl.SSLError): # the client hung up after the handshake
|
||||
pass
|
||||
|
||||
def connect(self, address, timeout=None):
|
||||
return self.client_sock
|
||||
|
||||
def close(self) -> None:
|
||||
self.client_sock.close()
|
||||
self._server_sock.close()
|
||||
self._thread.join(timeout=5)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def certificate(tmp_path):
|
||||
return _self_signed(tmp_path)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def server(certificate):
|
||||
_der, cert_file, key_file = certificate
|
||||
s = _TlsServer(cert_file, key_file)
|
||||
yield s
|
||||
s.close()
|
||||
|
||||
|
||||
def _check(expected: str | None, seen: list, port: int = 443) -> CertificateCheck:
|
||||
args: dict = {OBSERVER_ARG: seen.append}
|
||||
if expected is not None:
|
||||
args[KNOWN_FINGERPRINTS_ARG] = {str(port): expected}
|
||||
return CertificateCheck(args, "10.0.0.5", port)
|
||||
|
||||
|
||||
class TestFingerprint:
|
||||
def test_is_sha256_of_the_der_certificate_in_colon_hex(self):
|
||||
assert fingerprint(DER) == DER_FP
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"value",
|
||||
[
|
||||
DER_FP,
|
||||
DER_FP.lower(),
|
||||
DER_HEX,
|
||||
DER_HEX.lower(),
|
||||
DER_FP.removeprefix("SHA256:"),
|
||||
" ".join(DER_HEX[i : i + 2] for i in range(0, 64, 2)),
|
||||
"sha256:" + DER_HEX,
|
||||
],
|
||||
)
|
||||
def test_a_pin_may_be_written_as_openssl_or_a_browser_shows_it(self, value):
|
||||
assert normalise_fingerprint(value) == DER_FP
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"value", ["", "AB:CD", "SHA1:" + DER_HEX, "ZZ" * 32, DER_HEX + "AB", 42]
|
||||
)
|
||||
def test_anything_else_is_refused(self, value):
|
||||
with pytest.raises(ValueError):
|
||||
normalise_fingerprint(value)
|
||||
|
||||
|
||||
class TestKnownFingerprints:
|
||||
def test_the_pin_is_the_one_for_the_port_connected_to(self):
|
||||
args = {KNOWN_FINGERPRINTS_ARG: {"443": DER_HEX}}
|
||||
assert CertificateCheck(args, "10.0.0.5", 443).expected == DER_FP
|
||||
assert CertificateCheck(args, "10.0.0.5", 8443).expected is None
|
||||
|
||||
def test_a_port_may_be_given_as_a_number(self):
|
||||
args = {KNOWN_FINGERPRINTS_ARG: {8443: DER_FP}}
|
||||
assert CertificateCheck(args, "10.0.0.5", 8443).expected == DER_FP
|
||||
|
||||
def test_a_malformed_mapping_is_refused(self):
|
||||
with pytest.raises(ValueError):
|
||||
CertificateCheck({KNOWN_FINGERPRINTS_ARG: [DER_FP]}, "10.0.0.5", 443)
|
||||
|
||||
def test_without_optional_args_it_is_first_use(self):
|
||||
assert CertificateCheck(None, "10.0.0.5", 443).expected is None
|
||||
|
||||
|
||||
class TestVerify:
|
||||
def test_first_use_accepts_and_reports_only_once_observed(self):
|
||||
seen: list = []
|
||||
check = _check(None, seen)
|
||||
observation = check.verify(DER)
|
||||
assert observation.accepted is True
|
||||
assert seen == []
|
||||
check.observe()
|
||||
assert seen == [
|
||||
{
|
||||
"host": "10.0.0.5",
|
||||
"port": 443,
|
||||
"seen": {"fingerprint": DER_FP},
|
||||
"expected": None,
|
||||
"accepted": True,
|
||||
}
|
||||
]
|
||||
|
||||
def test_the_pinned_certificate_is_accepted(self):
|
||||
seen: list = []
|
||||
check = _check(DER_HEX.lower(), seen)
|
||||
assert check.verify(DER).accepted is True
|
||||
check.observe()
|
||||
assert seen[0]["expected"] == {"fingerprint": DER_FP}
|
||||
assert seen[0]["accepted"] is True
|
||||
|
||||
def test_another_certificate_is_refused_reported_and_named(self):
|
||||
seen: list = []
|
||||
with pytest.raises(TlsCertificateMismatch) as caught:
|
||||
_check(OTHER_FP, seen).verify(DER)
|
||||
assert caught.value.observation.seen == DER_FP
|
||||
assert caught.value.observation.expected == OTHER_FP
|
||||
assert seen == [
|
||||
{
|
||||
"host": "10.0.0.5",
|
||||
"port": 443,
|
||||
"seen": {"fingerprint": DER_FP},
|
||||
"expected": {"fingerprint": OTHER_FP},
|
||||
"accepted": False,
|
||||
}
|
||||
]
|
||||
|
||||
def test_the_mismatch_is_a_connection_failure_that_names_both(self):
|
||||
with pytest.raises(ConnectionException) as caught:
|
||||
_check(OTHER_FP, []).verify(DER)
|
||||
message = str(caught.value)
|
||||
assert "10.0.0.5:443" in message
|
||||
assert DER_FP in message
|
||||
assert OTHER_FP in message
|
||||
|
||||
def test_a_failing_observer_does_not_break_the_connection(self):
|
||||
def boom(_observation):
|
||||
raise RuntimeError("store down")
|
||||
|
||||
check = CertificateCheck({OBSERVER_ARG: boom}, "10.0.0.5", 443)
|
||||
check.verify(DER)
|
||||
check.observe()
|
||||
|
||||
def test_observe_before_verify_is_a_mistake(self):
|
||||
with pytest.raises(RuntimeError):
|
||||
_check(None, []).observe()
|
||||
|
||||
|
||||
class TestHandshake:
|
||||
def test_fetch_returns_the_certificate_the_server_presents(self, server, certificate):
|
||||
der, _cert, _key = certificate
|
||||
assert fetch_certificate("10.0.0.5", 443, 5, connect=server.connect) == der
|
||||
|
||||
def test_verify_server_refuses_another_certificate_before_anything_is_sent(
|
||||
self, server, tmp_path
|
||||
):
|
||||
other_der, _c, _k = _self_signed(tmp_path, "impostor")
|
||||
seen: list = []
|
||||
check = _check(fingerprint(other_der), seen)
|
||||
with pytest.raises(TlsCertificateMismatch):
|
||||
check.verify_server(5, connect=server.connect)
|
||||
server.close()
|
||||
assert server.received == b""
|
||||
assert seen[0]["accepted"] is False
|
||||
|
||||
def test_verify_server_accepts_on_first_use(self, server, certificate):
|
||||
der, _cert, _key = certificate
|
||||
observation = _check(None, []).verify_server(5, connect=server.connect)
|
||||
assert observation.seen == fingerprint(der)
|
||||
|
||||
|
||||
class TestRequests:
|
||||
def test_pin_makes_every_connection_present_the_verified_certificate(self):
|
||||
check = _check(None, [])
|
||||
check.verify(DER)
|
||||
session = requests.Session()
|
||||
check.pin(session)
|
||||
adapter = session.get_adapter("https://10.0.0.5/rest/v7/")
|
||||
assert adapter.poolmanager.connection_pool_kw["assert_fingerprint"] == DER_HEX.lower()
|
||||
proxy = adapter.proxy_manager_for("http://proxy.example:3128")
|
||||
assert proxy.connection_pool_kw["assert_fingerprint"] == DER_HEX.lower()
|
||||
|
||||
def test_pin_before_verify_is_a_mistake(self):
|
||||
with pytest.raises(RuntimeError):
|
||||
_check(None, []).pin(requests.Session())
|
||||
|
||||
def test_a_changed_certificate_on_a_pinned_session_is_a_mismatch(self):
|
||||
from urllib3.util.ssl_ import assert_fingerprint
|
||||
|
||||
seen: list = []
|
||||
check = _check(DER_FP, seen)
|
||||
check.verify(DER)
|
||||
impostor = b"another certificate"
|
||||
with pytest.raises(TlsCertificateMismatch) as caught, check.refusing():
|
||||
try:
|
||||
assert_fingerprint(impostor, DER_HEX)
|
||||
except Exception as exc:
|
||||
raise requests.exceptions.SSLError(exc) from exc
|
||||
assert caught.value.observation.seen == fingerprint(impostor)
|
||||
assert caught.value.observation.expected == DER_FP
|
||||
assert seen[-1]["accepted"] is False
|
||||
|
||||
def test_refusing_lets_other_errors_through(self):
|
||||
check = _check(None, [])
|
||||
check.verify(DER)
|
||||
with pytest.raises(requests.exceptions.ConnectionError), check.refusing():
|
||||
raise requests.exceptions.ConnectionError("no route to host")
|
||||
@@ -4,6 +4,7 @@ import pytest
|
||||
|
||||
from napalm_device_types import (
|
||||
AccessPointDriver,
|
||||
BackupServerDriver,
|
||||
FirewallDriver,
|
||||
HypervisorDriver,
|
||||
ManagementControllerDriver,
|
||||
@@ -15,6 +16,7 @@ from napalm_device_types import (
|
||||
|
||||
BASE_CLASSES = [
|
||||
(AccessPointDriver, "Access Point"),
|
||||
(BackupServerDriver, "Backup Server"),
|
||||
(FirewallDriver, "Firewall"),
|
||||
(HypervisorDriver, "Hypervisor"),
|
||||
(ManagementControllerDriver, "Management Controller"),
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
"""Moving a guest to another node of its cluster, and a node's HA maintenance mode.
|
||||
|
||||
``get_vm_migration_preflight`` says where a guest could go and what holds it
|
||||
back, ``migrate_vm`` moves it and waits, ``set_node_maintenance`` switches a
|
||||
node's HA maintenance mode. Declared only, under ``TYPE_CHECKING``: netOrk
|
||||
probes each with ``hasattr`` and hides what a driver lacks (VMware has none of
|
||||
them yet).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import ast
|
||||
import inspect
|
||||
from typing import get_type_hints
|
||||
|
||||
import pytest
|
||||
|
||||
import napalm_device_types.hypervisor as hypervisor_module
|
||||
from napalm_device_types import HypervisorDriver, MigrationPreflightDict
|
||||
|
||||
|
||||
def _declared(name: str) -> ast.FunctionDef:
|
||||
tree = ast.parse(inspect.getsource(hypervisor_module))
|
||||
for node in ast.walk(tree):
|
||||
if isinstance(node, ast.FunctionDef) and node.name == name:
|
||||
return node
|
||||
raise AssertionError(f"HypervisorDriver does not declare {name}()")
|
||||
|
||||
|
||||
def _signature(name: str) -> str:
|
||||
fn = _declared(name)
|
||||
return f"({ast.unparse(fn.args)}) -> {ast.unparse(fn.returns) if fn.returns else None}"
|
||||
|
||||
|
||||
class TestDeclarations:
|
||||
@pytest.mark.parametrize(
|
||||
"method", ["get_vm_migration_preflight", "migrate_vm", "set_node_maintenance"]
|
||||
)
|
||||
def test_absent_until_a_driver_implements_it(self, method):
|
||||
assert not hasattr(HypervisorDriver, method)
|
||||
|
||||
def test_preflight(self):
|
||||
assert _signature("get_vm_migration_preflight") == (
|
||||
"(self, name: str) -> MigrationPreflightDict"
|
||||
)
|
||||
|
||||
def test_migrate(self):
|
||||
assert _signature("migrate_vm") == (
|
||||
"(self, name: str, target_node: str, online: bool=True, "
|
||||
"target_storage: str | None=None, timeout: int=3600) -> None"
|
||||
)
|
||||
|
||||
def test_maintenance(self):
|
||||
assert _signature("set_node_maintenance") == "(self, node: str, enabled: bool) -> None"
|
||||
|
||||
|
||||
class TestPreflightShape:
|
||||
def test_keys(self):
|
||||
assert get_type_hints(MigrationPreflightDict) == {
|
||||
"running": bool,
|
||||
"allowed_nodes": list[str],
|
||||
"refused_nodes": dict[str, str],
|
||||
"local_disks": list[str],
|
||||
"local_resources": list[str],
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
"""A provisioning that failed after the hypervisor gave the VM its id says
|
||||
which VM it left behind, so the caller can remove it (NAPALM/napalm-proxmox#28)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pickle
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import VMProvisionIncomplete
|
||||
from napalm_device_types.provisioning import VMProvisionIncomplete as from_module
|
||||
|
||||
|
||||
def _raise_from(cause: Exception) -> VMProvisionIncomplete:
|
||||
try:
|
||||
try:
|
||||
raise cause
|
||||
except Exception as exc:
|
||||
raise VMProvisionIncomplete(str(exc), vmid="101", node="pve1") from exc
|
||||
except VMProvisionIncomplete as caught:
|
||||
return caught
|
||||
|
||||
|
||||
def test_it_is_exported_from_the_package_root():
|
||||
assert VMProvisionIncomplete is from_module
|
||||
|
||||
|
||||
def test_it_names_the_vm_left_behind():
|
||||
caught = _raise_from(RuntimeError("disk import failed"))
|
||||
assert (caught.vmid, caught.node) == ("101", "pve1")
|
||||
|
||||
|
||||
def test_it_keeps_the_failure_and_its_words():
|
||||
cause = RuntimeError("disk import failed")
|
||||
caught = _raise_from(cause)
|
||||
assert caught.__cause__ is cause
|
||||
assert str(caught) == "disk import failed"
|
||||
|
||||
|
||||
def test_a_caller_catching_runtime_error_still_catches_it():
|
||||
# The contract has create_vm_from_cloud_init raise RuntimeError on failure.
|
||||
with pytest.raises(RuntimeError):
|
||||
raise VMProvisionIncomplete("start failed", vmid="7", node="n")
|
||||
|
||||
|
||||
def test_vmid_and_node_are_keyword_only():
|
||||
with pytest.raises(TypeError):
|
||||
VMProvisionIncomplete("failed", "101", "pve1") # type: ignore[misc]
|
||||
|
||||
|
||||
def test_it_survives_pickling_as_a_task_result_would():
|
||||
caught = pickle.loads(pickle.dumps(VMProvisionIncomplete("failed", vmid="101", node="pve1")))
|
||||
assert (str(caught), caught.vmid, caught.node) == ("failed", "101", "pve1")
|
||||
@@ -0,0 +1,28 @@
|
||||
"""begin_stop_vm: ask for a VM's clean shutdown and return at once.
|
||||
|
||||
``stop_vm`` blocks until the VM is stopped, with a limit of the driver's own.
|
||||
A caller that keeps its waits as timers (netOrk's graph workflows,
|
||||
NetOrk/netork#1636) asks with ``begin_stop_vm`` and reads ``get_vm_status``
|
||||
until the VM is stopped. A driver mixes it in when it can; no role demands it.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import inspect
|
||||
|
||||
from napalm_device_types import HypervisorDriver, VmStopRequestMixin
|
||||
|
||||
|
||||
def test_no_role_demands_it():
|
||||
assert not issubclass(HypervisorDriver, VmStopRequestMixin)
|
||||
|
||||
|
||||
def test_declared_not_implemented():
|
||||
assert not hasattr(VmStopRequestMixin, "begin_stop_vm")
|
||||
|
||||
|
||||
def test_declaration_documents_the_contract():
|
||||
source = inspect.getsource(VmStopRequestMixin)
|
||||
assert "def begin_stop_vm(self, name: str) -> None" in source
|
||||
assert "ValueError" in source
|
||||
assert "RuntimeError" in source
|
||||
Reference in new issue
Block a user