Compare commits
172
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
dacd10fcce | ||
|
|
8a54674e40 | ||
|
|
d45455adf1 | ||
|
|
ac49e2b5d4 | ||
|
|
c967635cba | ||
|
|
c470686305 | ||
|
|
1edf458feb | ||
|
|
00452d0d37 | ||
|
|
4e40f95bc1 | ||
|
|
70e1c9b218 | ||
|
|
a70501ff60 | ||
|
|
c6d119eb83 | ||
|
|
8e9d21c368 | ||
|
|
25f28297e4 | ||
|
|
847c902193 | ||
|
|
ffbb194300 | ||
|
|
edef6cf07c | ||
|
|
df476a12f2 | ||
|
|
834cb2e0ea | ||
|
|
f573fa53d2 | ||
|
|
fdf2f89115 | ||
|
|
d63abff467 | ||
|
|
9d6a1a65ed | ||
|
|
95e3aa1e26 | ||
|
|
a0901e3277 | ||
|
|
060e210130 | ||
|
|
31973d2d5f | ||
|
|
abda0973e7 | ||
|
|
4eeaaef236 | ||
|
|
3490d8b62a | ||
|
|
2542081c6f | ||
|
|
4427be35cc | ||
|
|
ad9ec0fb4d | ||
|
|
8dd7e8ab9a | ||
|
|
935fad2a2d | ||
|
|
f337ae7fb8 | ||
|
|
e2aebeadeb | ||
|
|
36cf117b95 | ||
|
|
02d461036b | ||
|
|
63039ce758 | ||
|
|
516fb18c0e | ||
|
|
8d32e70cd7 | ||
|
|
51e8c849bb | ||
|
|
6aadcc3d91 | ||
|
|
e98d4763f3 | ||
|
|
24f4be3b7e | ||
|
|
f4354f498e | ||
|
|
73771aba00 | ||
|
|
c9571d59f1 | ||
|
|
cf862af026 | ||
|
|
83ea13f88e | ||
|
|
4938056d20 | ||
|
|
9c3be46d8a | ||
|
|
a6289cb24a | ||
|
|
46c1c9dc4e | ||
|
|
bc17862a44 | ||
|
|
9f6ffa8a25 | ||
|
|
4150570a94 | ||
|
|
a809e6c02f | ||
|
|
5b894913e1 | ||
|
|
00aa5efef0 | ||
|
|
97cb8779c5 | ||
|
|
47751d12d6 | ||
|
|
3f59023f46 | ||
|
|
666be091e3 | ||
|
|
25cc264306 | ||
|
|
86959b7599 | ||
|
|
8664accbc4 | ||
|
|
449a6b13ca | ||
|
|
c71ddd7fe6 | ||
|
|
3b6c5f9585 | ||
|
|
1c71fe6896 | ||
|
|
9159106837 | ||
|
|
28ff14589b | ||
|
|
5fffb3b886 | ||
|
|
2846824a21 | ||
|
|
0545d1d814 | ||
|
|
49c65cbd5d | ||
|
|
888a7229dc | ||
|
|
0f0064c2d7 | ||
|
|
cb90fa9835 | ||
|
|
551ed0a29d | ||
|
|
0dc1a598e2 | ||
|
|
131346c228 | ||
|
|
a5b7b540c6 | ||
|
|
874c6c5a92 | ||
|
|
145c2c668b | ||
|
|
1cef3385e8 | ||
|
|
fbd3cf0129 | ||
|
|
e400c4359c | ||
|
|
4d3574cd60 | ||
|
|
87dbdfac66 | ||
|
|
a07ced2df6 | ||
|
|
22e4377c2b | ||
|
|
49830671c9 | ||
|
|
d7ec558933 | ||
|
|
0475184684 | ||
|
|
b7350a9b2c | ||
|
|
08296d0d2e | ||
|
|
ac8e7c4c11 | ||
|
|
f77091c489 | ||
|
|
be3958e561 | ||
|
|
a21e365c4e | ||
|
|
cce26eff44 | ||
|
|
888a11ca6a | ||
|
|
94bf399d34 | ||
|
|
bb2229fd9e | ||
|
|
680500dd2e | ||
|
|
a0e70d1686 | ||
|
|
2bd7bcc77c | ||
|
|
83cd7db761 | ||
|
|
1dd448e1d3 | ||
|
|
753ddc5eda | ||
|
|
59fb1273e8 | ||
|
|
4641063db6 | ||
|
|
634c142477 | ||
|
|
2d1c40c87d | ||
|
|
87c9b06bf3 | ||
|
|
2805f911c4 | ||
|
|
6551008ad3 | ||
|
|
321537ff0a | ||
|
|
9324a5b8f1 | ||
|
|
b4c4ec0b0b | ||
|
|
07b50f6a48 | ||
|
|
90367c1370 | ||
|
|
2739f10ecd | ||
|
|
0330f17a67 | ||
|
|
d3034bc8ea | ||
|
|
4f3b88db0e | ||
|
|
c7ea15dbb4 | ||
|
|
01fe680456 | ||
|
|
fde6d717a8 | ||
|
|
a1da0337e2 | ||
|
|
7afa590c0c | ||
|
|
155b5bbbe4 | ||
|
|
e509445572 | ||
|
|
dd8977750d | ||
|
|
e58b65a3db | ||
|
|
7ec427c743 | ||
|
|
d43e15e8db | ||
|
|
23c352aa8a | ||
|
|
a710a85de6 | ||
|
|
c947250179 | ||
|
|
ff6dbe0206 | ||
|
|
7aa731ad34 | ||
|
|
72679ff4f8 | ||
|
|
7b44f689bc | ||
|
|
ba2f18ea7d | ||
|
|
37c09d0fa0 | ||
|
|
8111c4cde3 | ||
|
|
e1ce72cc5c | ||
|
|
7ba84acad3 | ||
|
|
8eb7dcd871 | ||
|
|
95d9031c8d | ||
|
|
885c7e1f53 | ||
|
|
eb80d5cb0d | ||
|
|
e50e497939 | ||
|
|
3aed0b48d7 | ||
|
|
735b683028 | ||
|
|
08ec32e93a | ||
|
|
bd43bd75fa | ||
|
|
1112191aec | ||
|
|
b8b89acee1 | ||
|
|
f833e23422 | ||
|
|
b97ec654a0 | ||
|
|
c2d8d4a0d2 | ||
|
|
18676a573f | ||
|
|
4071f35050 | ||
|
|
17d8dabb4d | ||
|
|
d55b036a8e | ||
|
|
536ffcf6e1 | ||
|
|
97e7ede131 |
No files matched your search
@@ -0,0 +1,15 @@
|
||||
name: CI
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
tags: ["v*"]
|
||||
pull_request:
|
||||
|
||||
jobs:
|
||||
driver:
|
||||
uses: NAPALM/driver-standard/.gitea/workflows/driver.yml@main
|
||||
with:
|
||||
standard-ref: main
|
||||
secrets:
|
||||
PYPI_TOKEN: ${{ secrets.PYPI_TOKEN }}
|
||||
+285
@@ -0,0 +1,285 @@
|
||||
# Changelog
|
||||
|
||||
All notable changes to napalm-device-types. The format follows
|
||||
[Keep a Changelog](https://keepachangelog.com/), versions follow
|
||||
[Semantic Versioning](https://semver.org/). Versions up to 4.21.0 were not
|
||||
recorded here; their history is the git log.
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [4.29.0] - 2026-11-04
|
||||
|
||||
### Added
|
||||
- `remove_local_account_keys(name, keys, *, clear_password=False)`: with `clear_password`
|
||||
the account's password goes along with the keys, and so do its sudo rule and admin
|
||||
groups, as sudo asks for that password. For a person who deleted their key and whose
|
||||
password went with it. The account stays open; the steps are `keys`, `password` and
|
||||
`sudo`, each run when another failed (NetOrk/netork#2033).
|
||||
|
||||
## [4.28.0] - 2026-11-04
|
||||
|
||||
### Added
|
||||
- `LocalAccountsMixin.add_local_account(..., replaces=[...])`: the keys the new one takes
|
||||
the place of go from the account, matched by their base64 whatever a line's options or
|
||||
comment. Keys the person added on the host stay. The step is `old keys`; it runs even
|
||||
when the new key could not be added (NetOrk/netork#2013).
|
||||
- `add_local_account(..., clear_password=True)`: the account's password goes, and it logs
|
||||
in with a key only. Step `password`. It cannot be combined with a `password_hash` or
|
||||
with `admin=True`, as the sudo rule asks for that password. A driver supplies
|
||||
`_account_clear_password`; one that does not fails the step and says so.
|
||||
- `LocalAccountsMixin.remove_local_account_keys(name, keys)`: takes the named keys off an
|
||||
account and leaves it open, its password and sudo as they are -- for a person who
|
||||
deleted a key and has no new one. Step `keys`.
|
||||
- A driver whose host keeps keys elsewhere overrides `_account_drop_keys` next to
|
||||
`_account_add_key` and `_account_remove_keys`.
|
||||
|
||||
## [4.27.0] - 2026-11-03
|
||||
|
||||
### Added
|
||||
- `ScheduledJobsMixin`: what a host runs on its own schedule as one shape,
|
||||
`ScheduledJobDict` — cron lines, run-parts scripts, anacron, systemd timers, `periodic`,
|
||||
a platform's own jobs, Windows tasks — with who owns each definition (`managed_by`) and
|
||||
what the driver can do to it (`actions`). `get_scheduled_jobs` reads them with the host's
|
||||
time zone; `manage_scheduled_job` enables, disables or runs one now; `put_scheduled_job`
|
||||
and `delete_scheduled_job` create, change and delete a job an application writes. Every
|
||||
write names the `digest` the caller saw (`job_digest`) and raises `ScheduledJobChanged`
|
||||
when the host's definition differs; `ScheduledJobRefused` for a job that cannot take a
|
||||
verb (NetOrk/netork#1933, #86).
|
||||
- `PosixScheduledJobsMixin`: the implementation over the command channel. One framed report
|
||||
as root reads `/etc/crontab`, `/etc/cron.d`, the user crontabs, the run-parts directories,
|
||||
`/etc/anacrontab` and every systemd timer with the unit it starts, and which package owns
|
||||
each definition. A cron line is switched off behind `#napalm-disabled: `; an
|
||||
application's job is a timer where systemd runs, else a file in `/etc/cron.d` or marked
|
||||
lines in the user's crontab.
|
||||
- `napalm_device_types.cron`: cron expressions as Vixie cron reads them, their next runs in
|
||||
a time zone, and as systemd `OnCalendar` lines that fire on exactly the same minutes.
|
||||
|
||||
### Removed
|
||||
- `OSDriver.get_cron_jobs` and `CronJobDict`. No application called them; the readers in
|
||||
napalm-linux and napalm-bsd go with them. `get_scheduled_jobs` replaces them.
|
||||
|
||||
## [4.26.0] - 2026-11-03
|
||||
|
||||
### Added
|
||||
- `OpenSshServerMixin.SSHD_CONFIG_GENERATED`: a driver whose platform writes sshd_config
|
||||
from its own database, but includes `/etc/ssh/sshd_config.d` (OpenMediaVault), sets it
|
||||
and keeps `harden_ssh_server`. The hardening then goes only into the drop-in; a
|
||||
sshd_config without that include is left alone and the result says why, since the
|
||||
platform would write over a change to it on its next apply (#88, NetOrk/netork#1950).
|
||||
|
||||
## [4.25.0] - 2026-11-02
|
||||
|
||||
### Added
|
||||
- `BackupServerDriver` (`ROLE = "backup_server"`), a role base for backup servers such as
|
||||
Proxmox Backup Server (#82). It declares `get_backup_datastores`, `get_backup_groups`
|
||||
(each group with its newest snapshot), `get_backup_snapshots`, `get_backup_jobs`
|
||||
(garbage collection, verify, prune, sync and tape in one list) and `get_backup_remotes`,
|
||||
and the actions `start_garbage_collection`, `run_backup_job`, `verify_backup_snapshot`,
|
||||
`set_backup_snapshot_protected` and `delete_backup_snapshot`, which return a task to
|
||||
follow with `get_backup_task`. The shapes live in `napalm_device_types.backups`. A
|
||||
server that manages its own disks lists `StorageDriver` as its second role.
|
||||
- `ClusterStorageDict.backup_server` (optional): which backup server, datastore and
|
||||
namespace a hypervisor's backup storage writes to, with the certificate fingerprint it
|
||||
pins, so an application can tie a server's backup groups to a cluster's guests.
|
||||
|
||||
## [4.24.0] - 2026-11-02
|
||||
|
||||
### Added
|
||||
- `parse_lldpctl_keyvalue(output)` and `LLDPCTL_KEYVALUE_COMMAND`: lldpd's neighbours
|
||||
(`lldpctl -f keyvalue`) as one `LldpNeighborRow` per neighbour, for every driver of a
|
||||
host that runs lldpd. Several neighbours on one port (a phone and the PC behind it, an
|
||||
unmanaged switch) stay apart, an interface name with dots (`eth0.1`) stays whole, the
|
||||
chassis and port ids are read whatever their type (MAC, address, interface name, local
|
||||
id), with management addresses and capabilities. A failed lldpctl (lldpd not running,
|
||||
lldpctl missing) raises instead of reading as no neighbours.
|
||||
- `lldp_neighbors(rows)` and `lldp_neighbors_detail(rows, interface="")`: NAPALM's
|
||||
`get_lldp_neighbors` and `get_lldp_neighbors_detail` from those rows, so a driver no
|
||||
longer builds the two shapes itself. A neighbour that names its chassis by a MAC
|
||||
address carries it in `get_lldp_neighbors` as `mac`, in lower case
|
||||
(`LldpNeighborDict`), so an application can match it by MAC.
|
||||
|
||||
### Fixed
|
||||
- `DNF_SECURITY_COMMAND` reads the cached metadata only (`dnf --cacheonly`), as
|
||||
`APT_UPGRADABLE_COMMAND` reads apt's local index (#76). With an expired cache dnf
|
||||
downloaded the repositories' metadata first, and on an Oracle Linux host on OCI that
|
||||
outlasted netOrk's whole poll budget. Refreshing the index stays a step of its own
|
||||
(`dnf makecache`). Without a cache dnf fails at once, which already reads as
|
||||
"security unknown".
|
||||
|
||||
## [4.23.0] - 2026-11-02
|
||||
|
||||
### Changed
|
||||
- Python 3.12 is the floor (`requires-python = ">=3.12"`): netOrk runs every driver
|
||||
on 3.12, so 3.10 and 3.11 are no longer tested (NAPALM/driver-standard#6).
|
||||
- The floor on napalm is 5.1: older releases import `pkg_resources`, which a Python
|
||||
3.12 venv no longer has.
|
||||
- `requests` is a declared dependency (floor 2.25.0), for `tls_certificates`; it came
|
||||
with napalm before.
|
||||
- The SNMP health profile shapes (`SnmpHealthProfile`, `SnmpValueSpec`,
|
||||
`SnmpMemorySpec`, `SnmpInterfacesSpec`) are defined in
|
||||
`napalm_device_types.snmp_health_profile`. The package and `models` still export
|
||||
them, so an existing import keeps working.
|
||||
|
||||
### Added
|
||||
- `HypervisorDriver.get_vm_migration_preflight(name)`, `migrate_vm(name, target_node,
|
||||
online=True, target_storage=None, timeout=3600)` and `set_node_maintenance(node,
|
||||
enabled)`: which nodes of its cluster a guest could move to and what blocks it
|
||||
(`MigrationPreflightDict`: `allowed_nodes`, `refused_nodes` with the platform's
|
||||
reason, `local_disks` the migration copies, `local_resources` that block it), the
|
||||
move itself, blocking until the platform's task ended and raising with the
|
||||
platform's own cause, and a node's HA maintenance mode. Declared only; netOrk
|
||||
probes each with `hasattr`.
|
||||
- `get_clusters()` may report HA, backups and quorum votes: `ClusterDict.ha_resources`
|
||||
(`HAResourceDict`: requested and current state, node, PVE 8 group, PVE 9 rules),
|
||||
`backup_jobs` (`BackupJobDict`: schedule, storage, mode, the guests a job covers)
|
||||
and `backups` (`BackupDict`: per guest and storage the newest backup, its size
|
||||
and how many there are), and `ClusterNodeDict.votes`. All `NotRequired`, so
|
||||
drivers written before stay valid; None means could not be read, `[]` read and
|
||||
empty.
|
||||
- `napalm_device_types.tls_certificates`: TLS certificates by trust on first use, the
|
||||
counterpart of `ssh_host_keys` for a driver's HTTPS API. A driver takes the
|
||||
fingerprint to expect from `optional_args["tls_fingerprints"]` (by port, the SHA-256
|
||||
of the DER certificate, `SHA256:AB:CD:…`) and refuses any other certificate with
|
||||
`TlsCertificateMismatch` (a `ConnectionException` naming both fingerprints) before a
|
||||
password is sent; no CA is consulted, so a self-signed certificate verifies. Without
|
||||
a fingerprint the first certificate is accepted. Every certificate seen goes to
|
||||
`optional_args["tls_certificate_observer"]`. `CertificateCheck.verify_server()`
|
||||
checks the certificate in a handshake of its own, `pin(session)` makes every later
|
||||
connection of a `requests` session present the same one, `refusing()` turns a refusal
|
||||
there into the mismatch, `observe()` reports it once the login stands
|
||||
(NAPALM/napalm-hpe-aruba-procurve#17).
|
||||
- `LldpAgentMixin.configure_lldp_agent(interfaces)` (`napalm_device_types.fixes`): run
|
||||
an LLDP agent on the interfaces the caller names. The template installs the agent
|
||||
(`ensure_package(LLDP_AGENT_PACKAGE)`), has the driver write the interface list
|
||||
(`_write_lldp_agent_config`), then enables and restarts the service
|
||||
(`manage_service(LLDP_AGENT_SERVICE, …)`); both names default to `lldpd`. No
|
||||
interface is a ValueError before anything changes. It is the action for what LLDP
|
||||
getters used to do on their own (NAPALM/napalm-openwrt#8).
|
||||
- `SnmpValueSpec.unknown`: a driver whose device's agent answers a value wrongly says
|
||||
so in its `SNMP_HEALTH_PROFILE` (`"cpu": {"unknown": True}`), and the application
|
||||
reports no value for it -- not the standard reading, not 0. OpenBSD's net-snmp,
|
||||
which reports the CPU as never idle, is the case (NAPALM/napalm-bsd#7).
|
||||
- `VMProvisionIncomplete` (`napalm_device_types.provisioning`): what
|
||||
`create_vm_from_cloud_init` raises when it fails after the VM was created. A
|
||||
`RuntimeError` with the failure as its cause and the half-built VM's `vmid` and
|
||||
`node`, so the caller can remove it with `destroy_vm` (NAPALM/napalm-proxmox#28).
|
||||
- `HypervisorDriver.get_clusters()` and its shapes `ClusterDict`, `ClusterNodeDict`,
|
||||
`ClusterStorageDict`, `ClusterNetworkDict`: every cluster the connection can see,
|
||||
with its nodes (offline ones included, `local` on the node the connection is),
|
||||
their CPU and memory, and the storages and networks with the nodes that see each.
|
||||
A standalone host is a cluster of one with no name; a manager reports each cluster
|
||||
and standalone host it runs. Declared only (NetOrk/netork#484, #1821).
|
||||
- `create_vm_from_cloud_init(node=…)`: the cluster node to create the VM on. None
|
||||
leaves it to the driver; a driver that can only create on its own node raises
|
||||
ValueError for another (NetOrk/netork#1822).
|
||||
- `LocalAccountsMixin.add_service_account(name, public_key)`: netOrk's own account,
|
||||
created through the account the driver logs in with today. It takes a key and no
|
||||
password, and a rule of its own in `85-service-<name>` lets it run everything as
|
||||
root without a password (`NOPASSWD: ALL`, checked with `visudo` and `sudo -l`). An
|
||||
account that is there is opened again and keeps its other keys; root is refused
|
||||
(NetOrk/netork#1817). With `uid=`, a new account gets that uid once no other account
|
||||
has it; one that is there keeps its own. A driver supplies
|
||||
`_account_create_with_uid`. An account that is there and cannot log in (`nologin`,
|
||||
`false`) is another service's, such as a netOrk satellite's `netork`, and is refused
|
||||
before anything changes.
|
||||
- `LocalAccountsMixin` (`napalm_device_types.local_accounts`): personal login accounts
|
||||
on a host. `get_local_accounts(names)` reads them back (groups, lock, sudo, key
|
||||
files), `add_local_account(name, public_key, password_hash=, admin=)` creates or
|
||||
reopens one, `lock_local_account(name)` takes its keys and sudo away and refuses
|
||||
every login while keeping its home. Every command root runs is one fixed command,
|
||||
the key files are read and written as the account itself, and the sudo rule is a
|
||||
drop-in of its own, checked with `visudo` and `sudo -l`. A driver supplies the
|
||||
commands that create, open and lock an account and take it out of a group, and may
|
||||
replace where the keys are kept (`_account_key_files`, `_account_read_keys`,
|
||||
`_account_add_key`, `_account_remove_keys`; default `~/.ssh/authorized_keys`)
|
||||
(NetOrk/netork#1793).
|
||||
- `OpenSshServerMixin.exempt_from_ssh_penalties`: puts the address the host sees
|
||||
the managing client at (`$SSH_CONNECTION`) into `PerSourcePenaltyExemptList`,
|
||||
in `/etc/ssh/sshd_config.d/00-netork-penalty-exempt.conf`, next to the host's
|
||||
own exemptions. Only where sshd has per-source penalties (OpenSSH 9.8+; an older
|
||||
one refuses the directive) and sshd_config includes the drop-in directory;
|
||||
checked with `sshd -t` and `sshd -T`, then a new session, and taken back on any
|
||||
failure. Helpers: `parse_penalty_exemptions`, `client_address`, `is_exempt`
|
||||
(NetOrk/netork#1800).
|
||||
- `retry_refused` and `refused_before_banner` (`napalm_device_types.ssh_host_keys`):
|
||||
a connection sshd closes before its banner -- what OpenSSH 9.8+ does to a source
|
||||
it has just penalised -- is tried again after 1, 3, 7 and 15 s.
|
||||
`HostKeyCheck.connect` does so itself (NetOrk/netork#1800).
|
||||
- `ARP_AGE_UNKNOWN` (`napalm_device_types.arp`, `-1.0`): the `age` a driver reports
|
||||
in `get_arp_table` and `get_ipv6_neighbors_table` when the device does not say how
|
||||
old an entry is. `0.0` means "confirmed just now" and is no stand-in for it
|
||||
(NetOrk/netork#1327).
|
||||
- `NetworkTargetDict.uplinks` (optional): the physical NICs a bridge leaves the
|
||||
host through, one group per way out -- a bond's slaves together, a single NIC
|
||||
alone (`[["eno1", "eno2"]]`). A missing key means unknown, an empty list a
|
||||
bridge without uplink. NetOrk follows their LLDP neighbours to the switch port
|
||||
and its untagged VLAN; one slave's neighbour speaks for its bond
|
||||
(NetOrk/netork#1325).
|
||||
- `HostShutdownMixin.shutdown_host()`, on `DeviceTypeDriver`: shut the device itself
|
||||
down cleanly through its own means; declared, so `hasattr` says whether a driver
|
||||
can. No forced variant (NetOrk/netork#1636).
|
||||
- `VmStopRequestMixin.begin_stop_vm(name)`: ask for a VM's clean shutdown and return
|
||||
at once, for callers that wait on a timer instead of in `stop_vm`. No role demands
|
||||
it (NetOrk/netork#1636).
|
||||
- `SudoHistoryMixin` (`napalm_device_types.sudo_history`): the sudo password an earlier
|
||||
driver typed into an interactive shell (`echo '<password>' | sudo -S -p "" …`) and
|
||||
left in the login user's shell history, for a driver to mix in (NetOrk/netork#1090).
|
||||
The driver says what its lines carry (`SUDO_HISTORY_MARKER`) and which files its
|
||||
partial sudo left to the login user (`SUDO_ROOT_FILES`), and supplies `run_command`.
|
||||
- `get_history_sudo_lines()` counts the lines per history file (`~/.bash_history`,
|
||||
`~/.zsh_history`, `~/.ash_history`, `~/.sh_history`) with `grep -c` as the login
|
||||
user. No line is read back; a file it cannot read raises.
|
||||
- `remove_history_sudo_lines()` deletes exactly those lines as the login user (`sed
|
||||
-i`, its backup overwritten with `shred -u`), and as root gives each of
|
||||
`SUDO_ROOT_FILES` back to `root:root` where it is a regular file, not a symbolic
|
||||
link, owned by the login user. It reports every file's lines before and after,
|
||||
how many were removed and which files are root's again. Nothing goes to stdin.
|
||||
|
||||
### Fixed
|
||||
- `AptProxyMixin` knows that apt's https sources take the http proxy too: apt's https
|
||||
method asks `Acquire::https::Proxy` and then `Acquire::http::Proxy`, for the host and
|
||||
then for all. An `00proxy` with the http line alone sent download.docker.com and
|
||||
every other https source to apt-cacher-ng, which answers `403 CONNECT denied` without
|
||||
a `PassThroughPattern`, and the whole upgrade failed (NetOrk/netork#1811).
|
||||
- The report carries `https_proxy`, `https_per_host` and `https_settings`;
|
||||
`https_proxy_for(report, host)` is the proxy apt takes for one https host.
|
||||
- `check_https_source()` fetches one https source's `InRelease` the way apt does;
|
||||
None where no https source goes through a proxy.
|
||||
- `configure_package_proxy` gains an `https` step: one https source through the new
|
||||
proxy. Where the proxy refuses it, `00proxy` also says `Acquire::https::Proxy
|
||||
"DIRECT";`, later https lines to the same proxy are commented out, and the source
|
||||
is fetched direct. An HTTP error from the mirror counts as let through.
|
||||
- `apt_fetch_command` takes https URLs; `direct` sets the host's https setting too.
|
||||
- `comment_out_command(..., scheme="https")`.
|
||||
|
||||
## [4.22.0] - 2026-10-23
|
||||
|
||||
### Added
|
||||
- `AptProxyMixin` (`napalm_device_types.apt_proxy`): apt's proxy on a
|
||||
Debian-family host, for a driver to mix in (NetOrk/netork#1397).
|
||||
- `get_package_proxy()` is the proxy apt really fetches through, as
|
||||
`apt-config dump` says, and no longer the content of one file. A proxy the
|
||||
Debian or Ubuntu installer set elsewhere wins over `00proxy`, and was missed.
|
||||
- `get_package_proxy_report()` adds where it is set: every line of apt's
|
||||
configuration that sets it, in the order apt reads them, the per-host
|
||||
proxies, and the sources fetched over http and over https.
|
||||
- `check_package_proxy(url, expect)` fetches a page from the host with apt's
|
||||
own downloader, past any proxy, and says whether the host reached it and
|
||||
whether the page says what you expected.
|
||||
- `configure_package_proxy(url)` writes the proxy, comments out every other
|
||||
line that sets a different one, reads the result back and fetches one
|
||||
source through the proxy. It fails when apt still fetches elsewhere.
|
||||
- `same_proxy_url()` compares two proxy URLs by scheme, host and port.
|
||||
|
||||
## [4.21.1] - 2026-10-22
|
||||
|
||||
The first release on PyPI since 0.1.0.
|
||||
|
||||
### Changed
|
||||
- The package follows the NAPALM driver standard
|
||||
(https://git.netork.io/NAPALM/driver-standard): CI on Gitea runs ruff, strict
|
||||
mypy, the tests on Python 3.10–3.13 and at the lowest declared versions,
|
||||
bandit, pip-audit and a build check, and a release tag publishes to PyPI.
|
||||
- Type annotations use builtin generics (`list[str]`, `X | None`) instead of
|
||||
`typing.List` and `Optional`. They mean the same to a type checker.
|
||||
- The project links point at git.netork.io instead of GitHub.
|
||||
@@ -6,14 +6,14 @@ Instead of inheriting directly from `napalm.base.NetworkDriver`, a driver can in
|
||||
|
||||
```python
|
||||
# without napalm-device-types
|
||||
class OpenWrtDriver(NetworkDriver):
|
||||
...
|
||||
class OpenWrtDriver(NetworkDriver): ...
|
||||
|
||||
|
||||
# with napalm-device-types
|
||||
from napalm_device_types import AccessPointDriver
|
||||
|
||||
class OpenWrtDriver(AccessPointDriver):
|
||||
...
|
||||
|
||||
class OpenWrtDriver(AccessPointDriver): ...
|
||||
```
|
||||
|
||||
## Why?
|
||||
@@ -29,13 +29,36 @@ OpenMediaVault box is a NAS built on Debian. So a driver inherits **one role bas
|
||||
role its device fills**, and **the order it lists them in is the ranking**:
|
||||
|
||||
```python
|
||||
class QnapQtsDriver(StorageDriver, HypervisorDriver, LinuxDriver):
|
||||
... # primary_role_of(...) == "storage"
|
||||
class QnapQtsDriver(
|
||||
StorageDriver, HypervisorDriver, LinuxDriver
|
||||
): ... # primary_role_of(...) == "storage"
|
||||
```
|
||||
|
||||
`roles_of(cls)`, `role_keys_of(cls)` and `primary_role_of(cls)` read that back. Nothing
|
||||
restates the ranking: there is no precedence table and no attribute to override.
|
||||
|
||||
A server's baseboard management controller (Supermicro BMC, Dell iDRAC, HPE iLO,
|
||||
Lenovo XCC) is a role of its own, `ManagementControllerDriver` (`management_controller`),
|
||||
reached over Redfish or IPMI (`USES_SSH = False`). It is not the server it manages:
|
||||
`reboot_host` restarts the BMC (Redfish `Manager.Reset`), while `set_power_state`
|
||||
(`On`, `GracefulShutdown`, `ForceOff`, `GracefulRestart`, `ForceRestart`, `PowerCycle`)
|
||||
acts on the server. `get_hardware_inventory` carries the server's serial number, system
|
||||
UUID and NIC MACs -- what ties the BMC to the host it manages. Its sensors come through
|
||||
NAPALM's `get_environment`, with `cpu` and `memory` empty, since a BMC does not know the
|
||||
host OS's load; it has no `HealthMetricsMixin` (SNMP) and no command channel.
|
||||
|
||||
A backup server (Proxmox Backup Server) is a role of its own too, `BackupServerDriver`
|
||||
(`backup_server`). Its datastores, the backup groups in them (each with its newest
|
||||
snapshot), its jobs (garbage collection, verify, prune, sync, tape) and its remotes have
|
||||
no counterpart in `StorageDriver`, so they are not bent into pools, volumes and
|
||||
replication jobs. A server that also manages its own disks and pools lists
|
||||
`StorageDriver` second: `class ProxmoxBackupDriver(BackupServerDriver, StorageDriver, ...)`.
|
||||
Its actions (garbage collection, running a job, verifying, protecting or deleting a
|
||||
snapshot) start a task and return its id at once; `get_backup_task` follows it. On the
|
||||
hypervisor side, a cluster storage that writes to a backup server says which one
|
||||
(`ClusterStorageDict.backup_server`: server, port, datastore, namespace, fingerprint), so
|
||||
an application can tie the server's groups to the cluster's guests.
|
||||
|
||||
### Role bases declare; they never implement
|
||||
|
||||
**A role base must not contain a single runtime method — not even a
|
||||
@@ -44,10 +67,12 @@ restates the ranking: there is no precedence table and no attribute to override.
|
||||
```python
|
||||
class StorageDriver(DeviceTypeDriver):
|
||||
"""Contract, not code."""
|
||||
|
||||
ROLE: str = "storage"
|
||||
TYPE_LABEL: str = "Storage"
|
||||
|
||||
if TYPE_CHECKING: # nothing exists at runtime
|
||||
if TYPE_CHECKING: # nothing exists at runtime
|
||||
|
||||
def get_disks(self) -> List[PhysicalDiskDict]: ...
|
||||
```
|
||||
|
||||
@@ -64,14 +89,252 @@ Two things follow, and both are improvements:
|
||||
- **A method that was never implemented raises `AttributeError`, not
|
||||
`NotImplementedError`.** Ask before calling.
|
||||
|
||||
The rule covers this package's methods, not NAPALM's. `NetworkDriver` still carries
|
||||
a runtime `NotImplementedError` stub for each of its getters (`get_interfaces`,
|
||||
`get_environment`, `get_users`, …), and every role base inherits it. So a NAPALM getter
|
||||
must come from the driver class itself or from a mixin listed **before** the first role
|
||||
base; a mixin listed after it loses the MRO to NAPALM's stub. For the same reason
|
||||
`hasattr` says nothing about NAPALM's getters, and the OS role's account listing is
|
||||
`get_os_users`, not `get_users` -- NAPALM's `get_users` has a different shape.
|
||||
|
||||
### NAPALM's getters: an unknown ARP age is `-1.0`
|
||||
|
||||
NAPALM requires an `age` on every entry of `get_arp_table` and
|
||||
`get_ipv6_neighbors_table`: seconds since the device last confirmed that neighbour.
|
||||
Many devices do not report one (`ip neigh`, `/proc/net/arp`, a router's host list).
|
||||
A driver that cannot read the age reports `ARP_AGE_UNKNOWN` (`-1.0`, as NAPALM's IOS
|
||||
driver does), never `0.0`: `0.0` means "confirmed just now", and an application that
|
||||
ranks sightings by freshness takes it at its word (NetOrk/netork#1327). An expiry
|
||||
countdown (FreeBSD's `expires in 1180 seconds`) is not an age either. A driver whose
|
||||
device does report the age keeps reporting it.
|
||||
|
||||
## Function classes: what a device *can do*
|
||||
|
||||
Behaviour shared across roles lives in a function class, exactly once, and a role base
|
||||
is a thin bundle over them — `PackageManagementMixin`, `HealthMetricsMixin`,
|
||||
`ServiceControlMixin`, `UpdateMixin`, `NatVpnMixin`, `MacAclMixin`, `FirewallRuleMixin`,
|
||||
`DhcpServerMixin`, `PingSweepMixin`, `ConfigLifecycleMixin`, `InterfaceFilterMixin`.
|
||||
`HostRebootMixin` (`reboot_host`) is mixed into `DeviceTypeDriver` itself, since any
|
||||
device may be restartable; like the others it only declares.
|
||||
`HostRebootMixin` (`reboot_host`) and `HostShutdownMixin` (`shutdown_host`) are mixed
|
||||
into `DeviceTypeDriver` itself, since any device may be restartable or able to shut itself
|
||||
down; like the others they only declare. `VmStopRequestMixin` (`begin_stop_vm`) asks for a
|
||||
VM's clean shutdown without waiting for it — a hypervisor driver mixes it in when it can.
|
||||
|
||||
**Fixes are templates** (`napalm_device_types.fixes`, since 4.13). A change the
|
||||
application wants made on a device runs as fixed steps the driver implements, with values
|
||||
the caller supplies: `SnmpAgentMixin.configure_snmp_agent(settings)` (install, configure,
|
||||
firewall, start), `LldpAgentMixin.configure_lldp_agent(interfaces)` (install, write the
|
||||
interfaces, enable and restart; the driver writes only the interfaces),
|
||||
`EnsurePackageMixin.ensure_package(name)`, `PackageProxyMixin.configure_package_proxy(url)` and
|
||||
`ContainerEngineMixin.grant_container_engine_access(user)`. A driver whose device takes the
|
||||
change in one call overrides the template method. A driver mixes in the fixes its device
|
||||
can take; none is on a role base. They replace `run_device_action`, which is deprecated.
|
||||
|
||||
**SNMP is the application's** (`docs/STRUCTURE.md`). `HealthMetricsMixin` is deprecated
|
||||
since 4.12 and goes in 5.0. A driver does not collect over SNMP; a driver whose device
|
||||
publishes its health values under OIDs of its own declares them as data:
|
||||
|
||||
```python
|
||||
class MySwitchDriver(SwitchDriver):
|
||||
SNMP_HEALTH_PROFILE: ClassVar[SnmpHealthProfile] = {
|
||||
"uptime": {"oids": ["1.3.6.1.2.1.1.3.0"]},
|
||||
"cpu": {"oids": ["<vendor cpu oid>", "<older firmware's>"]},
|
||||
"memory": {"total": {"oids": ["<oid>"]}, "free": {"oids": ["<oid>"]}, "unit": "KiB"},
|
||||
}
|
||||
```
|
||||
|
||||
A value is `oids` (the first that answers wins) or a `table` folded by `first`, `average`
|
||||
or `max`, with an optional `pattern` for an agent that answers with text. What the profile
|
||||
does not name, the application reads from the standard UCD-MIB and IF-MIB.
|
||||
A value the agent gets wrong is `{"unknown": True}`: the application then reports no value
|
||||
for it, rather than the standard reading or a default (OpenBSD's net-snmp reports its CPU
|
||||
as never idle).
|
||||
|
||||
`KernelFactsMixin` (`get_kernel_facts`) is the exception that is mixed in by a driver
|
||||
rather than by a role base: what a Linux kernel has built and loaded is read the same way
|
||||
everywhere, so the command and its parse are concrete here and a driver supplies only
|
||||
`_run_kernel_facts_command`. `OSDriver` does not carry it — a Windows host is an OS driver
|
||||
too, and `hasattr(driver, "get_kernel_facts")` has to stay truthful.
|
||||
|
||||
`ServicePackagesMixin` (`get_service_packages`) is mixed in the same way: which installed
|
||||
packages each systemd service is made of — the one shipping its unit file and the one
|
||||
shipping the binary its `ExecStart` runs, by `dpkg -S` or `rpm -qf`, not the libraries
|
||||
that binary links against. An application needs it to let a statement about a service
|
||||
(the postgresql unit) reach the findings on its packages. A driver supplies
|
||||
`_run_service_packages_command`.
|
||||
|
||||
`AptProxyMixin` (`napalm_device_types.apt_proxy`, since 4.22) is `PackageProxyMixin` for a
|
||||
Debian-family host, mixed in the same way. What apt fetches through is what `apt-config dump`
|
||||
says, not one file: apt reads `apt.conf.d` in order and `apt.conf` last, and the last value
|
||||
wins (`get_package_proxy`, `get_package_proxy_report` with every line that sets it).
|
||||
`check_package_proxy(url, expect)` fetches a page with apt's own downloader, past any proxy.
|
||||
`configure_package_proxy(url)` writes `00proxy`, comments out every other line that sets a
|
||||
different proxy, reads the result back and fetches one http source's `InRelease` through it.
|
||||
A driver supplies `_run_apt_command`, `_run_apt_root_command` and `_write_apt_file`.
|
||||
|
||||
`HostStatusMixin` (`get_host_status`) is mixed in the same way: whether a Linux host needs
|
||||
a reboot to finish an update (`/var/run/reboot-required`, `needs-restarting -r`, or a newer
|
||||
kernel of the running flavour installed), whether it patches itself (unattended-upgrades,
|
||||
dnf-automatic) and when that last succeeded (apt's `upgrade-stamp`), and whether dpkg was
|
||||
left halfway (a journal in `/var/lib/dpkg/updates` or a half-configured package; unknown
|
||||
while a package manager is running), and how full `/`, `/boot` and `/var` are (`df -Pk`,
|
||||
each filesystem once, in bytes; how full is too full is the caller's decision). `package_updates` holds the shared apt and dnf parsers: apt's suites become
|
||||
an update's `origin`, a `-security` suite makes it a security update, and dnf's security
|
||||
advisories do the same.
|
||||
|
||||
`ListeningSocketsMixin` (`get_listening_sockets`) is mixed in the same way: every listening
|
||||
TCP and bound UDP socket from `ss -lntup`, with the systemd service or container behind it
|
||||
from `/proc/<pid>/cgroup`, in one round trip. A driver supplies
|
||||
`_run_listening_sockets_command(command, privileged=)`; the command arrives as one `sh -c`
|
||||
argument, so a `sudo -n` prefix covers all of it. Without root `ss` names only the login
|
||||
user's processes, and the reading says so (`attributed: false`) instead of failing. A host
|
||||
without `ss` is read with `netstat -lntup` (OpenWrt's busybox, old net-tools); on OpenWrt the
|
||||
cgroup names the procd service (`/services/<name>/<instance>`). A host with neither raises
|
||||
`ListeningSocketsUnavailable`.
|
||||
|
||||
LLDP neighbours are rows: `LldpNeighborRow` is one neighbour as its local port heard it
|
||||
(chassis and port id with their type, name, description, management addresses,
|
||||
capabilities in NAPALM's names), and `lldp_neighbors(rows)` / `lldp_neighbors_detail(rows,
|
||||
interface)` are NAPALM's two getters projected from them, a port with two neighbours
|
||||
keeping both; a neighbour named by its chassis MAC carries it as `mac` (lower case). A driver for a host that runs lldpd sends `LLDPCTL_KEYVALUE_COMMAND` and
|
||||
hands its output to `parse_lldpctl_keyvalue`: one row per neighbour block, an interface
|
||||
name with dots (`eth0.1`) kept whole, ids read under the type lldpd prints them with
|
||||
(`chassis.mac`, `port.local`, ...). When lldpctl fails (lldpd not running, no `lldpctl`)
|
||||
the parse raises instead of reading as no neighbours.
|
||||
|
||||
`HostFirewallMixin` (`get_host_firewall`) reads a Linux host's own netfilter rules the same
|
||||
way: `nft -j list ruleset`, with the tables iptables-nft manages taken from `iptables-save`
|
||||
instead, because nft shows their extensions (`-m conntrack`, `-m set`) only as an opaque `xt`.
|
||||
A legacy iptables is a ruleset of its own and is kept beside nft's. Every chain says its family,
|
||||
table, hook, priority and policy; every rule is a list of matches (`iifname`, `saddr`, `dport`,
|
||||
`ct_state`, ...) and a verdict. A condition it does not model is an `unknown` match with its
|
||||
text, never left out. A driver supplies `_run_host_firewall_command(command, privileged=)`; both
|
||||
tools need root, so a host where neither could be read raises `HostFirewallUnavailable` rather
|
||||
than reading as one without rules. Whether the rules keep a service unreachable is the
|
||||
consumer's call.
|
||||
|
||||
`VirtualIpsMixin` (`get_virtual_ips`) reads the addresses a host's HA configuration lets
|
||||
float between machines: vip-manager (5.x's `--config` YAML, 1.x's `-ip=${VIP_IP}` resolved
|
||||
through the unit's environment) and keepalived (`virtual_ipaddress` of every
|
||||
`vrrp_instance`, `include` followed four levels deep). It reports what is *declared*, not
|
||||
what the host holds right now -- that is in its interface addresses. The same files hold
|
||||
the etcd password and `auth_pass`, so an awk program filters **on the host** and prints
|
||||
allowlisted fields only; a secret never reaches the caller. Root first, then without it.
|
||||
A mechanism it could not read completely is `None` ("did not look"), a host without it
|
||||
`[]`. The command is about 4 kB, so a driver sends it on an exec channel
|
||||
(`_run_virtual_ips_command(command, privileged=)`), not typed into a shell.
|
||||
|
||||
`ServiceClustersMixin` (`get_service_clusters`) reads the configuration that proves two
|
||||
hosts run one service together -- not merely that they answer at one address: Patroni
|
||||
(`scope`, `name` and `restapi.connect_address` from its YAML, then the members and roles
|
||||
its REST API's `/cluster` lists from the DCS), adguardhome-sync (`ORIGIN_URL` and
|
||||
`REPLICA*_URL` of its container: the origin pushes to each replica) and BunkerWeb (a
|
||||
worker's `API_WHITELIST_IP` names the manager it takes its configuration from, a manager's
|
||||
`BUNKERWEB_INSTANCES` its workers). Each member carries its role as the mechanism words it
|
||||
and the index of the member it takes its configuration from (`config_from`). The same
|
||||
configuration holds the superuser password, the sync's logins and BunkerWeb's database URI,
|
||||
so awk programs filter **on the host**: allowlisted fields only, a URL without its
|
||||
`user:password@`, and the DCS endpoints stay on the host. Root first (Patroni's YAML is the
|
||||
postgres user's), then without it. `None` per mechanism is "did not look", `[]` "has none".
|
||||
The container engine is asked through `_container_engine_binary` where the driver has one.
|
||||
A driver sends the command on an exec channel (`_run_service_clusters_command(command,
|
||||
privileged=)`).
|
||||
|
||||
`get_dhcp_ha` (declared on `DhcpServerMixin`, optional) is what a DHCP server's own HA
|
||||
configuration says: whether HA is on, which peer this server is, each peer's name, role --
|
||||
as the server words it, not a shared vocabulary -- and, where the server has one, the URL
|
||||
its HA listener answers on. It is configuration, not state: whether the pair is in hot
|
||||
standby right now is not in it. `ConfigSyncMixin` (`get_config_sync`) is where a host
|
||||
pushes its configuration (`peer`, None for no one) and for which services, by the names
|
||||
`get_services` reports; sync items the driver has no service name for stay visible under
|
||||
`unmapped`. A driver mixes it in -- no role promises it. Both getters raise when they could
|
||||
not read; neither answers "off" for "could not ask". What the facts mean for a pair (hot
|
||||
standby, which side to write to) is the caller's to work out.
|
||||
|
||||
A firewall's CARP addresses are a third mechanism, `carp`: a firewall driver reads them from
|
||||
its own API and reports them under the reading's `carp` key (`vip_manager` and `keepalived`
|
||||
then None). What a CARP state means is protocol knowledge and stays in that driver. The
|
||||
contract only carries the result: unlike the other two, a CARP entry says who holds the
|
||||
address (`holds`), because a firewall's interface list need not show it. `holds` None
|
||||
leaves the holder to the interfaces.
|
||||
|
||||
`SystemIdentityMixin` (`get_system_identity`) reads a host's SMBIOS identity -- system UUID,
|
||||
product serial and board serial from `/sys/class/dmi/id` -- which is what ties a server to
|
||||
its BMC (`ManagementControllerDriver.get_hardware_inventory` reports the same values from
|
||||
the other side). Vendor placeholders (`0123456789` on Supermicro boards, `To be filled by
|
||||
O.E.M.`, `Default string`, all-zero or the common `03000200-…-000700080009` UUID) read as
|
||||
empty: matching on them would join unrelated machines. `is_placeholder_identity` is the
|
||||
same filter for a BMC driver's side. `product_uuid` and `product_serial` are root's, so a
|
||||
driver supplies `_run_system_identity_command(command, privileged=)`.
|
||||
|
||||
`OpenSshServerMixin` (`harden_ssh_server`, `get_ssh_server_algorithms`) takes algorithms
|
||||
out of what an OpenSSH server offers -- the ones ssh-audit flags. It reads the lists sshd
|
||||
really uses (`sshd -T`), takes out what was asked and writes the rest to
|
||||
`/etc/ssh/sshd_config.d/00-netork-ssh-hardening.conf`; only where sshd_config has no such
|
||||
`Include` is sshd_config itself rewritten, after a copy. It never removes the last algorithm
|
||||
of a class, nor -- given the managing client's lists -- the last one that client speaks.
|
||||
`sshd -t` must accept the result and `sshd -T` must show it; after the reload the caller's
|
||||
`verify` opens a new session, and if that fails the still-open one puts the old state back.
|
||||
A driver mixes it in and supplies `run_command`; a platform that regenerates sshd_config
|
||||
itself (an appliance with its own database) must not, or opts out with
|
||||
`harden_ssh_server = None`.
|
||||
|
||||
`SudoHistoryMixin` (`get_history_sudo_lines`, `remove_history_sudo_lines`) deals with what
|
||||
a driver left behind when it typed `echo '<password>' | sudo -S -p "" …` into an interactive
|
||||
shell: that line, password and all, in the login user's shell history, and files its partial
|
||||
sudo wrote as the login user. The driver names the marker its lines carry
|
||||
(`SUDO_HISTORY_MARKER`) and those files (`SUDO_ROOT_FILES`). The getter counts the lines per
|
||||
history file with `grep -c`, as the login user; the fix deletes them with `sed -i` and
|
||||
overwrites the backup with `shred -u`, then gives the files back to root (regular files the
|
||||
login user owns, never a symbolic link). The password is never read, sent or written. A
|
||||
driver mixes it in and supplies `run_command`.
|
||||
|
||||
**Update readers raise when they cannot read.** `get_available_updates` returns an empty
|
||||
list only when nothing is pending; netOrk keeps "pending since" per package, and an empty
|
||||
list for "don't know" would reset it.
|
||||
|
||||
`SystemdServicesMixin` (`get_services`, `manage_service`) is mixed in the same way, by
|
||||
the drivers whose host runs systemd. Listing the services, checking a unit name and
|
||||
reading an action's exit status are the same on every such host, so they are concrete
|
||||
here, and a driver supplies only `_run_service_command(command, *, privileged, timeout)`
|
||||
— how a command reaches its host and how it gains root there. The listing is one round
|
||||
trip (`list-unit-files` plus one `systemctl show` over every loaded unit) instead of an
|
||||
`is-enabled` and a `show` per unit. A host without systemd raises `SystemdUnavailable`,
|
||||
a `NotImplementedError`, so a driver can fall back to another init system.
|
||||
|
||||
`ScheduledJobsMixin` (`get_scheduled_jobs`, `manage_scheduled_job`, `put_scheduled_job`,
|
||||
`delete_scheduled_job`) is what a host runs on its own schedule — cron lines, run-parts
|
||||
scripts, anacron, systemd timers, `periodic`, a platform's own jobs, Windows tasks — as one
|
||||
shape, `ScheduledJobDict`. Every job says who owns its definition (`managed_by`: package,
|
||||
admin, platform, application) and what the driver can do to it (`actions`): every job a
|
||||
driver can switch takes `enable`, `disable` and `run`; only a job an application wrote
|
||||
through `put_scheduled_job` takes `edit` and `delete`. Every write names the `digest` the
|
||||
caller saw and raises `ScheduledJobChanged` when the host's definition is no longer that.
|
||||
`PosixScheduledJobsMixin` implements it over `run_command`: one framed report as root,
|
||||
a cron line switched off behind `#napalm-disabled: `, a timer by `systemctl`, an
|
||||
application's job as a timer where systemd runs, else in `/etc/cron.d` or the user's
|
||||
crontab. A driver sets the paths its host keeps jobs in. `napalm_device_types.cron` parses
|
||||
cron expressions, gives their next runs and writes them as `OnCalendar` lines.
|
||||
|
||||
### Access channels: why there is no container model here
|
||||
|
||||
`CommandChannelMixin` declares a driver's public channel to its host:
|
||||
- `run_command(command, *, privileged=False, timeout=60, stdin=None)` returns a `CommandResult(stdout, stderr, exit_code)`;
|
||||
- `open_stream(command, *, privileged=False)` returns a `ByteStream` to the command's stdin and stdout.
|
||||
|
||||
The mixin sits in `DeviceTypeDriver` and only declares, so `hasattr(driver, "run_command")` is true exactly where a driver implements it. For SSH drivers, `channel.run_on_transport` and `channel.open_stream_on_transport` are the implementation over a paramiko exec channel: no PTY, stderr kept apart, a real exit code. How a command gains root stays with the driver.
|
||||
|
||||
**`ContainerEngineMixin` is deliberately different from `SystemdServicesMixin`.** The systemd mixin owns its command and its parse; this one owns neither.
|
||||
- `container_engines()` says which engines the host offers (`[{"engine": "docker", "api": "docker-engine"}]`).
|
||||
- `open_container_engine(engine)` returns a `ContainerEngineConnection`:
|
||||
- `open_api()` is a stream to the engine's API (`docker system dial-stdio`);
|
||||
- `run_cli(args)` and `stream_cli(args)` run the engine's CLI with arguments the caller chooses. With `privileged=True` the driver runs it the way `run_command` gains root, for an engine that refuses the login user.
|
||||
- The driver decides only *how* the engine is reached. Its hook `_container_engine_binary(engine)` returns, for QNAP, the Container Station path, and the caller never sees it.
|
||||
|
||||
**What runs on the engine, and what to do with it, is netOrk's** (NetOrk/netork#765): the container model, the Engine API requests, compose, updates. Above the connection everything is specific to the service, so this package abstracts the connection and nothing more.
|
||||
|
||||
A driver mixes `ContainerEngineMixin` in itself; `OSDriver` does not carry it. A Windows host is an OS driver too, and has no `dial-stdio` to offer over WinRM.
|
||||
|
||||
**Never half-close early.** `ByteStream.write` does not close anything; only `close_write` does. `dial-stdio` hands the daemon a half-close as "client gone", and the daemon then answers an unfinished request with HTTP 499.
|
||||
|
||||
A function class may use the **template form** — public method concrete, the
|
||||
device-specific part a `_hook` declared under `if TYPE_CHECKING` — *when the base
|
||||
@@ -84,6 +347,47 @@ NAPALM's own getters (`get_facts`, `get_interfaces`, `ping`, `get_config`) are n
|
||||
wrapped in a template — they belong to NAPALM, and code outside this repo relies on
|
||||
their contract.
|
||||
|
||||
### SSH host keys: trust on first use
|
||||
|
||||
A driver that accepts any host key logs in with the device's credentials at whatever
|
||||
answers on the device's address (NetOrk/netork#1167). The store of trusted keys is the
|
||||
caller's; `ssh_host_keys.HostKeyCheck` is how every SSH driver verifies against it:
|
||||
|
||||
- **In:** `optional_args["ssh_host_keys"]` = `{"22": {"key_type": "ssh-ed25519", "key": "<base64>"}}`,
|
||||
by port. The driver takes the key for the port it connects to. With one, any other key is
|
||||
refused *before* authentication and the connection raises `SshHostKeyMismatch` (a
|
||||
`ConnectionException`). Without one, the first key is accepted.
|
||||
- **Out:** `optional_args["ssh_host_key_observer"]`, a callable, gets one dict per
|
||||
connection: `{"host", "port", "seen": {"key_type", "key", "fingerprint"}, "expected", "accepted"}`.
|
||||
`accepted` is False only for a refused key. A failing observer is logged, never raised.
|
||||
- **paramiko:** `HostKeyCheck(optional_args, host, port).connect(client, **connect_kwargs)`.
|
||||
- **netmiko:** `with check.netmiko(self.netmiko_optional_args) as kwargs: ConnectHandler(..., **kwargs)`,
|
||||
then `check.observe(conn.remote_conn_pre.get_transport())`. The known key reaches netmiko as a
|
||||
`known_hosts` file that exists only inside the block, with `ssh_strict=True`.
|
||||
|
||||
A caller that passes neither key gets first use on every connection, as before; a driver
|
||||
that ignores both still connects. Old and new versions on either side keep working.
|
||||
|
||||
### TLS certificates: trust on first use
|
||||
|
||||
The same for a driver's HTTPS API. Devices ship self-signed certificates, so a CA check
|
||||
fails on all of them and turning verification off trusts whatever answers;
|
||||
`tls_certificates.CertificateCheck` pins the certificate by its fingerprint instead
|
||||
(NAPALM/napalm-hpe-aruba-procurve#17). No CA is consulted.
|
||||
|
||||
- **In:** `optional_args["tls_fingerprints"]` = `{"443": "SHA256:AB:CD:…"}`, by port: the
|
||||
SHA-256 of the DER certificate as `openssl x509 -fingerprint -sha256` prints it (prefix,
|
||||
colons and case optional). With one, any other certificate is refused *before* a request
|
||||
carries credentials and the connection raises `TlsCertificateMismatch` (a
|
||||
`ConnectionException` naming both fingerprints). Without one, the first certificate is
|
||||
accepted. A driver does not fall back to another transport on a mismatch.
|
||||
- **Out:** `optional_args["tls_certificate_observer"]`, a callable, gets one dict per
|
||||
connection: `{"host", "port", "seen": {"fingerprint"}, "expected", "accepted"}`.
|
||||
- **requests:** `check = CertificateCheck(optional_args, host, port)`;
|
||||
`check.verify_server(timeout)` before the login, `check.pin(session)` so every later
|
||||
connection of the session must present that certificate, the login inside
|
||||
`with check.refusing():`, then `check.observe()`.
|
||||
|
||||
## Design principle: generic vs. device-specific logic
|
||||
|
||||
When adding behavior to a device-type base class, split it along one line: **would
|
||||
@@ -100,22 +404,30 @@ Concretely: matching/comparison/reconciliation algorithms, orchestration flows,
|
||||
generic data shapes belong here. Only the actual device communication belongs in
|
||||
`vendor/napalm-<name>`.
|
||||
|
||||
**Worked example — firewall rule diff/apply** (`FirewallDriver`):
|
||||
**Worked example — firewall rule diff/apply** (`FirewallRuleMixin`, in `FirewallDriver`):
|
||||
|
||||
```python
|
||||
class FirewallDriver(DeviceTypeDriver):
|
||||
# Abstract — every driver implements its own device communication.
|
||||
def get_firewall_rules(self) -> List[FirewallRuleDict]: raise NotImplementedError
|
||||
def apply_firewall_rule(self, rule: FirewallRuleDict, *, uuid: Optional[str] = None) -> Dict[str, Any]: raise NotImplementedError
|
||||
def commit_firewall_rules(self) -> Dict[str, Any]: raise NotImplementedError
|
||||
class FirewallRuleMixin: # mixed into FirewallDriver
|
||||
# Declared only — every driver implements its own device communication.
|
||||
if TYPE_CHECKING:
|
||||
|
||||
def get_firewall_rules(self) -> List[FirewallRuleDict]: ...
|
||||
def apply_firewall_rule(
|
||||
self, rule: FirewallRuleDict, *, uuid: Optional[str] = None
|
||||
) -> Dict[str, Any]: ...
|
||||
def commit_firewall_rules(self) -> Dict[str, Any]: ...
|
||||
|
||||
# Concrete — the matching/comparison/orchestration algorithm is identical
|
||||
# for every firewall vendor, so it lives here once.
|
||||
def diff_firewall_rules(self, desired: List[FirewallRuleDict]) -> FirewallRuleDiffDict:
|
||||
... # matches self.get_firewall_rules() against `desired` by description
|
||||
def diff_firewall_rules(
|
||||
self, desired: List[FirewallRuleDict]
|
||||
) -> (
|
||||
FirewallRuleDiffDict
|
||||
): ... # matches self.get_firewall_rules() against `desired` by description
|
||||
|
||||
def apply_firewall_ruleset(self, desired: List[FirewallRuleDict]):
|
||||
... # computes the diff, calls apply_firewall_rule() per change, commits
|
||||
def apply_firewall_ruleset(
|
||||
self, desired: List[FirewallRuleDict]
|
||||
): ... # computes the diff, calls apply_firewall_rule() per change, commits
|
||||
```
|
||||
|
||||
The same split applies to `DhcpServerMixin`: `get_dhcp_reservations`/
|
||||
@@ -159,7 +471,7 @@ detection vs. presentation there.
|
||||
pip install napalm-device-types
|
||||
```
|
||||
|
||||
Requires Python ≥ 3.9 and NAPALM ≥ 4.0.
|
||||
Requires Python ≥ 3.12 (tested up to 3.13) and NAPALM ≥ 4.0.
|
||||
|
||||
## Available base classes
|
||||
|
||||
@@ -172,12 +484,38 @@ Requires Python ≥ 3.9 and NAPALM ≥ 4.0.
|
||||
| `OSDriver` | General-purpose operating systems | Linux, BSD, macOS |
|
||||
| `StorageDriver` | Storage appliances & NAS/SAN | TrueNAS, Synology DSM, QNAP QTS |
|
||||
| `ResidentialGatewayDriver` | Router + firewall + AP in one box | OpenWrt, FritzBox |
|
||||
| `PhoneDriver` | IP telephony endpoints | Yealink |
|
||||
| `MediaDriver` | Speakers, streamers, media renderers | Sonos, Chromecast |
|
||||
| `ManagementControllerDriver` | Server baseboard management controllers | Supermicro BMC, Dell iDRAC, HPE iLO, Lenovo XCC |
|
||||
| `BackupServerDriver` | Backup servers: datastores, backup groups, GC/verify/prune/sync/tape jobs | Proxmox Backup Server |
|
||||
|
||||
Mixins mixed into the classes above rather than used on their own:
|
||||
`ConfigLifecycleMixin` (config load/compare/commit/rollback), `PingSweepMixin`
|
||||
(subnet sweeps), and `DhcpServerMixin` (static DHCP reservations — mixed into
|
||||
`FirewallDriver` and `ResidentialGatewayDriver`, since both commonly run the
|
||||
DHCP server for their networks).
|
||||
Mixins that come with the classes above: `PingSweepMixin` (subnet sweeps, in
|
||||
every driver through `DeviceTypeDriver`) and `DhcpServerMixin` (static DHCP
|
||||
reservations — mixed into `FirewallDriver` and `ResidentialGatewayDriver`,
|
||||
since both commonly run the DHCP server for their networks).
|
||||
`ConfigLifecycleMixin` (config load/compare/commit/rollback) is mixed in by a
|
||||
driver itself.
|
||||
|
||||
### Fingerprints
|
||||
|
||||
`DeviceTypeDriver` carries what discovery matches a device against: `PORT_SPECS`,
|
||||
`SNMP_OBJECT_ID_PREFIX`, `SNMP_FINGERPRINT`, `SSH_FINGERPRINT`, `HTTP_FINGERPRINT` and
|
||||
`OUI_PREFIXES`. A `FingerprintRule` is a lowercase substring with a weight; `mandatory`
|
||||
disqualifies the driver when it is missing, `negative` subtracts.
|
||||
|
||||
Discovery compares drivers by score divided by the most a driver could score. Rules that
|
||||
can never match together, such as one vendor's landing page or another's, share a
|
||||
`group`: only the strongest rule of a group counts towards that maximum
|
||||
(`rules_max_weight`). Without it, a driver that recognises three vendors' pages would
|
||||
score a third as well as one that recognises a single page:
|
||||
|
||||
```python
|
||||
HTTP_FINGERPRINT = [
|
||||
FingerprintRule("aten international", 6.0, group="landing_page"), # Supermicro
|
||||
FingerprintRule("idrac", 6.0, group="landing_page"),
|
||||
FingerprintRule("integrated lights-out", 6.0, group="landing_page"), # HPE iLO
|
||||
]
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
@@ -186,8 +524,8 @@ DHCP server for their networks).
|
||||
```python
|
||||
from napalm_device_types import AccessPointDriver
|
||||
|
||||
class OpenWrtDriver(AccessPointDriver):
|
||||
|
||||
class OpenWrtDriver(AccessPointDriver):
|
||||
def get_wireless_clients(self):
|
||||
# return List[WirelessClientDict]
|
||||
...
|
||||
@@ -202,8 +540,8 @@ class OpenWrtDriver(AccessPointDriver):
|
||||
```python
|
||||
from napalm_device_types import SwitchDriver
|
||||
|
||||
class CiscoIOSDriver(SwitchDriver):
|
||||
|
||||
class CiscoIOSDriver(SwitchDriver):
|
||||
def get_spanning_tree(self):
|
||||
# return Dict[str, SpanningTreeDict]
|
||||
...
|
||||
@@ -218,8 +556,8 @@ class CiscoIOSDriver(SwitchDriver):
|
||||
```python
|
||||
from napalm_device_types import FirewallDriver
|
||||
|
||||
class PfSenseDriver(FirewallDriver):
|
||||
|
||||
class PfSenseDriver(FirewallDriver):
|
||||
def get_nat_translations(self):
|
||||
# return List[NATTranslationDict]
|
||||
...
|
||||
@@ -239,13 +577,20 @@ class PfSenseDriver(FirewallDriver):
|
||||
```python
|
||||
from napalm_device_types import HypervisorDriver
|
||||
|
||||
class ProxmoxDriver(HypervisorDriver):
|
||||
|
||||
class ProxmoxDriver(HypervisorDriver):
|
||||
def get_vms(self):
|
||||
# return List[VMDict]
|
||||
...
|
||||
|
||||
def create_vm_snapshot(self, name, snapshot, description="", include_memory=False):
|
||||
def create_vm_snapshot(self, name, snapshot, description="", include_memory=False): ...
|
||||
|
||||
def get_network_targets(self):
|
||||
# return List[NetworkTargetDict]: bridges and vnets a new VM's NIC may
|
||||
# attach to; a bridge may name its physical NICs in `uplinks`, one group
|
||||
# per way out: [["eno1", "eno2"]] for a bond, [["eno1"]] for a single NIC,
|
||||
# no internal or VLAN ports -- leave the key out when they cannot be
|
||||
# determined
|
||||
...
|
||||
|
||||
def get_vm_cpu_types(self):
|
||||
@@ -255,13 +600,26 @@ class ProxmoxDriver(HypervisorDriver):
|
||||
...
|
||||
```
|
||||
|
||||
A hypervisor that provisions VMs from cloud images declares, per guest OS,
|
||||
the agent cloud-init installs so the hypervisor can read the new VM's IP:
|
||||
`GUEST_AGENTS = {guest_os: (packages, runcmd)}`. Its keys are the guests the
|
||||
driver can provision, and `create_vm_from_cloud_init(guest_os=...)` refuses
|
||||
any other. A QEMU-based driver takes `provisioning.QEMU_GUEST_AGENTS`
|
||||
(Linux, FreeBSD, OpenBSD). `provisioning.network_config()` writes the
|
||||
network-config v2 every cloud-init flavour reads (FreeBSD's nuageinit reads
|
||||
no other), and `provisioning.split_compression()` says how a packed image is
|
||||
unpacked; both are the same for every hypervisor. A creation that fails once
|
||||
the VM exists raises `VMProvisionIncomplete`, a `RuntimeError` that names the
|
||||
half-built VM (`vmid`, `node`) for the caller to `destroy_vm`, with the failure
|
||||
as its cause.
|
||||
|
||||
### OS / Linux
|
||||
|
||||
```python
|
||||
from napalm_device_types import OSDriver
|
||||
|
||||
class LinuxDriver(OSDriver):
|
||||
|
||||
class LinuxDriver(OSDriver):
|
||||
def get_packages(self):
|
||||
# return List[PackageDict]
|
||||
...
|
||||
@@ -270,17 +628,13 @@ class LinuxDriver(OSDriver):
|
||||
# return List[ServiceDict]
|
||||
...
|
||||
|
||||
def get_users(self):
|
||||
# return List[UserDict]
|
||||
def get_os_users(self):
|
||||
# return List[UserDict] -- not NAPALM's get_users, whose shape differs
|
||||
...
|
||||
|
||||
def get_processes(self):
|
||||
# return List[ProcessDict]
|
||||
...
|
||||
|
||||
def get_cron_jobs(self):
|
||||
# return List[CronJobDict]
|
||||
...
|
||||
```
|
||||
|
||||
### Storage / NAS
|
||||
@@ -288,8 +642,8 @@ class LinuxDriver(OSDriver):
|
||||
```python
|
||||
from napalm_device_types import StorageDriver
|
||||
|
||||
class TrueNASDriver(StorageDriver):
|
||||
|
||||
class TrueNASDriver(StorageDriver):
|
||||
def get_disks(self):
|
||||
# return List[PhysicalDiskDict]
|
||||
...
|
||||
@@ -315,7 +669,7 @@ from napalm_device_types.models import (
|
||||
## Development
|
||||
|
||||
```bash
|
||||
git clone https://github.com/chrismanivong/napalm-device-types.git
|
||||
git clone https://git.netork.io/NAPALM/napalm-device-types.git
|
||||
cd napalm-device-types
|
||||
pip install -e ".[dev]"
|
||||
```
|
||||
@@ -328,10 +682,15 @@ mypy napalm_device_types
|
||||
|
||||
## Contributing
|
||||
|
||||
1. Fork the repository
|
||||
2. Create a feature branch (`git checkout -b feature/router-driver`)
|
||||
3. Implement your changes
|
||||
4. Open a Pull Request
|
||||
This package follows the
|
||||
[NAPALM driver standard](https://git.netork.io/NAPALM/driver-standard/src/branch/main/STANDARDS.md):
|
||||
test first, ruff and strict mypy, Conventional Commits, one pull request per change
|
||||
against `main`. CI checks what a tool can check:
|
||||
|
||||
```bash
|
||||
pip install "napalm-standard @ git+https://git.netork.io/NAPALM/driver-standard.git"
|
||||
napalm-standard check && napalm-standard ratchet
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
|
||||
@@ -0,0 +1,244 @@
|
||||
# Structure
|
||||
|
||||
Status: agreed plan. Nothing here is implemented yet; the migration at the end says in
|
||||
which order it will be.
|
||||
|
||||
## Principle
|
||||
|
||||
**napalm-device-types is vendor-, OS-, protocol- and product-agnostic.** It holds the
|
||||
contracts every driver implements and the generic logic that works unchanged for every
|
||||
driver of a role. It knows no shell, no paramiko, no package manager, no container CLI,
|
||||
no SNMP OID, no cloud-init and no application that uses the drivers.
|
||||
|
||||
That rules out three kinds of code that are here today:
|
||||
|
||||
- **How a kind of host is read.** Shell commands and their parsers for POSIX hosts move to
|
||||
a new package, **napalm-unix**, which napalm-linux and napalm-bsd depend on.
|
||||
- **What an application does with a device.** Polling a device over SNMP, sweeping a range
|
||||
with pings, scoring discovery evidence, choosing what a provisioned guest installs, and
|
||||
deciding what counts as a problem belong to the application, not to the drivers.
|
||||
- **Product naming.** Files, markers and messages a driver leaves on a device carry the
|
||||
driver's name or none, never the name of the application that called it.
|
||||
|
||||
```
|
||||
napalm-device-types roles, capabilities, templates, models, reconcilers <- every driver
|
||||
^
|
||||
napalm-unix POSIX host readers, SSH exec transport, privilege <- napalm-linux, napalm-bsd
|
||||
^
|
||||
napalm-linux, napalm-bsd connection, platform getters, platform steps
|
||||
|
||||
application SNMP polling, ping sweeps, discovery scoring, guest
|
||||
profiles, warnings, and the values every fix is given
|
||||
```
|
||||
|
||||
## What stays in napalm-device-types
|
||||
|
||||
| Kind | Rule | Modules |
|
||||
|---|---|---|
|
||||
| **Role** | Declares only, under `TYPE_CHECKING`. No runtime method. | `access_point`, `switch`, `firewall`, `hypervisor`, `os`, `storage`, `phone`, `media`, `residential_gateway`, `management_controller`, `backup_server` |
|
||||
| **Capability** | Declares only. | `packages`, `services`, `updates`, `nat_vpn`, `mac_acl`, `config_lifecycle`, `interface_filter`, `host_reboot` |
|
||||
| **Template** | A concrete method that runs a fixed sequence of abstract steps. A driver implements the steps, or overrides the whole method when its device does it in one go. All values come from the caller. | the fix routines below |
|
||||
| **Contract types** | Interfaces and data shapes other packages implement or fill. | `CommandChannelMixin`, `CommandResult`, `ByteStream`; the declaration of `ContainerEngineMixin`; the reader mixins' public method declarations; the SNMP profile type; `models` |
|
||||
| **Reconciler** | Concrete and vendor-neutral, without I/O of its own. | `firewall_rules` and `dhcp` diff/apply, `_matching`, `lag` |
|
||||
| **Introspection** | Concrete, without I/O. | `roles`; `base` (`FingerprintRule`, `PortSpec`, the fingerprint attributes) |
|
||||
| **Driver facts** | Class attributes that describe the device or how it is reached, not what an application does with it. | `USES_SSH`, `FULL_UPGRADE`, `platform`, `SNMP_HEALTH_PROFILE` |
|
||||
|
||||
### Fix routines are templates
|
||||
|
||||
A change the application wants made on a device, for example "run an SNMP agent that
|
||||
answers us", is one template per purpose. The template fixes the order of the steps; the
|
||||
driver knows how each step is done on its device; the caller supplies every value.
|
||||
|
||||
```python
|
||||
class SnmpAgentMixin:
|
||||
def configure_snmp_agent(self, settings: SnmpAgentSettingsDict) -> FixResultDict:
|
||||
steps = [
|
||||
self._install_snmp_agent,
|
||||
self._write_snmp_agent_config,
|
||||
self._enable_snmp_agent,
|
||||
self._verify_snmp_agent,
|
||||
]
|
||||
... # runs them in order, stops at the first failure
|
||||
|
||||
# declared only; napalm-unix implements them for net-snmp
|
||||
def _install_snmp_agent(self) -> FixStepDict: ...
|
||||
def _write_snmp_agent_config(self, settings: SnmpAgentSettingsDict) -> FixStepDict: ...
|
||||
def _enable_snmp_agent(self) -> FixStepDict: ...
|
||||
def _verify_snmp_agent(self, settings: SnmpAgentSettingsDict) -> FixStepDict: ...
|
||||
```
|
||||
|
||||
napalm-unix implements the four steps for net-snmp, for Linux and the BSDs alike. A REST
|
||||
driver such as OPNsense overrides `configure_snmp_agent` as a whole, because its device
|
||||
takes the whole configuration in one call. The community, the allowed sources, contact
|
||||
and location are the caller's.
|
||||
|
||||
The same shape replaces every action in today's `run_device_action`:
|
||||
|
||||
| Today | Template |
|
||||
|---|---|
|
||||
| `fix_snmp` | `configure_snmp_agent(settings)` |
|
||||
| `fix_apt_proxy` | `configure_package_proxy(url)` |
|
||||
| `fix_docker_permissions` | `grant_container_engine_access(user)` |
|
||||
| `install_lldpd`, `install_auc` | `ensure_package(name)` |
|
||||
|
||||
The mixin is mixed in by the driver, or by the platform package, not by a role base. A
|
||||
driver that inherits a template it cannot run on its device, as a NAS driver built on the
|
||||
Linux driver inherits the package proxy, opts out by setting the method to `None`. So a
|
||||
caller asks `callable(getattr(driver, "configure_snmp_agent", None))`, never `hasattr`.
|
||||
|
||||
The caller verifies the result from its own side where it can; the SNMP agent is asked
|
||||
by the application's SNMP client, not by a probe the driver hand-builds.
|
||||
|
||||
### SNMP: drivers declare data, the application polls
|
||||
|
||||
The drivers do not collect over SNMP. A driver whose device answers the standard UCD-MIB
|
||||
and IF-MIB declares nothing. A driver whose device publishes the same values under
|
||||
proprietary OIDs declares them as data, in a class attribute of a type defined here:
|
||||
|
||||
```python
|
||||
class ProcurveDriver(SwitchDriver):
|
||||
SNMP_HEALTH_PROFILE: ClassVar[SnmpHealthProfile] = {
|
||||
"uptime": {"oids": ["1.3.6.1.4.1.11.2.14.11.1.2.4.1.5.1"], "format": "ticks"},
|
||||
"cpu": {"oids": ["1.3.6.1.4.1.11.2.14.11.5.1.9.6.1.0", "1.3.6.1.4.1.11.2.3.7.8.6.1.0"]},
|
||||
"memory": {"total": "...", "free": "...", "unit": "bytes"},
|
||||
}
|
||||
```
|
||||
|
||||
One OID per value is not enough for the drivers there are. The profile type has to say:
|
||||
|
||||
- **Fallbacks:** an ordered list of OIDs; the first that answers wins (ProCurve's CPU).
|
||||
- **Tables:** an OID to walk and how to fold its rows: first, average or maximum (TP-Link
|
||||
reports CPU and memory per stack unit).
|
||||
- **Text values:** a pattern that takes the number out of a string (Netgear reports CPU
|
||||
as `"5 Secs ( 3.2%)"`).
|
||||
- **Memory:** total with used or free, or a percentage only, and the unit.
|
||||
- **Uptime:** which OID, standard ones included (`sysUpTime` where a device has no
|
||||
`hrSystemUptime`), and its format.
|
||||
- **Interfaces:** a pattern of names to skip, and whether the TX-error counter counts
|
||||
drops. Today the second is `True` on every firewall, because OPNsense counts drops
|
||||
there; it becomes OPNsense's profile entry.
|
||||
|
||||
Polling, the standard OIDs, the counter arithmetic (64-bit counters first, 32-bit as the
|
||||
fallback) and the defaults are the application's.
|
||||
|
||||
### Warnings are facts
|
||||
|
||||
A driver reports facts: the proxy a host is configured with, a DSL line's SNR, the
|
||||
firmware version it runs and the one it is offered. It does not compare them with a
|
||||
setting of the application or with a threshold, and it does not name a warning.
|
||||
|
||||
The same goes for lookups a device does not make itself: what firmware a vendor
|
||||
currently publishes is the application's catalogue, not a getter.
|
||||
|
||||
### Getters read, and say when they cannot
|
||||
|
||||
- A getter changes nothing on the device. It does not start a daemon, rewrite its
|
||||
configuration or install a package on the way to an answer.
|
||||
- A read that fails raises. An empty list means "there is none", never "could not ask";
|
||||
a sync that deletes and re-adds on top of an empty answer would otherwise duplicate
|
||||
everything.
|
||||
- A write reports failure from the device's answer or exit status, never `success: True`
|
||||
by default.
|
||||
|
||||
### The application uses public methods only
|
||||
|
||||
The application never calls a driver's private methods (`_get`, `_post`,
|
||||
`_send_command`, `_action_*`). What it needs from a device becomes a declared method of
|
||||
the driver.
|
||||
|
||||
## What moves to napalm-unix
|
||||
|
||||
| Module | Knows |
|
||||
|---|---|
|
||||
| `host_status` | apt, dnf, dpkg, systemd timers, `df` |
|
||||
| `kernel` | `/proc`, `/lib/modules`, gzip, base64 |
|
||||
| `listening` | `ss`, `netstat`, `/proc/<pid>/cgroup` |
|
||||
| `systemd` | `systemctl` |
|
||||
| `virtual_ips` | vip-manager, keepalived, awk |
|
||||
| `system_identity` | SMBIOS via sysfs and `dmidecode` |
|
||||
| `package_updates` | apt and dnf output |
|
||||
| `ssh_server` | OpenSSH (`sshd -T`, `sshd -t`, `sshd_config`); which algorithms to drop stays the caller's |
|
||||
| `terminal` | ANSI escape codes in shell output |
|
||||
| `channel`: `run_on_transport`, `open_stream_on_transport`, `ParamikoExecStream` | paramiko exec channels |
|
||||
| `container_engine`: `ContainerEngineConnection`, engine probing | the Docker CLI, `system dial-stdio` |
|
||||
|
||||
It also takes what napalm-linux and napalm-bsd each carry today:
|
||||
|
||||
- **Privilege:** one `sudo -S` / `sudo -n` / root layer, including the rule that a
|
||||
privileged command never gets stdin while a sudo password is set, and one way to write a
|
||||
file as root (stage with `mktemp`, then `install`). Two copies of this have already
|
||||
drifted apart.
|
||||
- **Parsers:** `/etc/passwd` with `/etc/group`, crontab lines, `ps` with `lstart`.
|
||||
- **Fix steps:** net-snmp, the package proxy, container engine group membership.
|
||||
|
||||
The FreeBSD part of `host_status` (`freebsd-version -k` against `-r`) moves to napalm-bsd.
|
||||
|
||||
In napalm-unix every reader follows one pattern:
|
||||
|
||||
- **Shape:** `FOO_COMMAND`, `parse_foo(output)`, and `FooMixin.get_foo()`.
|
||||
- **Reaching the host:** the mixin uses `run_command(command, privileged=...)` from
|
||||
`CommandChannelMixin`. The six per-reader `_run_*` hooks, with four different signatures,
|
||||
go away.
|
||||
- **Framing:** one frame helper writes and parses the `BEGIN`/`END` markers for every reader.
|
||||
- **Errors:** one exception for "cannot read" and one for "not on this host". A read that
|
||||
fails or is cut short raises; it never reads as "nothing there".
|
||||
- **Names on the host:** temporary files, markers and drop-ins carry `napalm` or the
|
||||
driver's name. A drop-in written under an older name is still recognised and replaced.
|
||||
|
||||
Layout: `napalm_unix/{posix,linux,freebsd,openssh,transport}/`.
|
||||
|
||||
## What moves to the application
|
||||
|
||||
| Today | Why it is not a driver's |
|
||||
|---|---|
|
||||
| `health_metrics`, `_ucd_metrics` | Polling over SNMP is something the application does to a device; the driver is not even connected while it runs. |
|
||||
| `ping_sweep` | Looping over addresses, capping the targets and reporting progress is a scan. A device with a faster batch keeps it as a capability of its own. |
|
||||
| `rules_max_weight` | Scoring is how one application ranks drivers. The rules themselves stay with the driver. |
|
||||
| `provisioning` (`QEMU_GUEST_AGENTS`, cloud-init network-config, decompressors) | Which agent a guest gets and how its network is configured is a guest profile. The hypervisor driver is handed the packages and commands. |
|
||||
| `TYPE_LABEL`, `REBOOT_SETTLE_SECONDS` | A display name and a polling delay. |
|
||||
| firmware catalogues (napalm-netgear's `netgear_plus`) | What a vendor publishes is not something the device reports. |
|
||||
| `run_device_action` | Replaced by the templates above. |
|
||||
|
||||
`OSDriver.ROLE` stops defaulting to `"linux"`. Each OS driver states its own role.
|
||||
|
||||
## Rules, enforced by tests in this package
|
||||
|
||||
1. Roles and capabilities own no runtime method; a template runs only declared steps.
|
||||
2. No module imports `paramiko`, `socket`, `subprocess` or `asyncio`, and none defines a
|
||||
`*_COMMAND`.
|
||||
3. No module, string or default names an application.
|
||||
4. Role methods return TypedDicts, not `Any`.
|
||||
|
||||
Each rule is added with the step that makes it pass.
|
||||
|
||||
## Migration
|
||||
|
||||
This package cannot keep re-exports pointing at napalm-unix: the contract package would
|
||||
then depend on its implementation. So a moved name is first deprecated here and then
|
||||
removed, and nothing breaks in between.
|
||||
|
||||
| Step | Change |
|
||||
|---|---|
|
||||
| 1 | The application moves to 4.9 and the drivers that need it; napalm-bsd stops passing stdin to a privileged command while a sudo password is set. |
|
||||
| 2 | This document; the application's architecture docs and this README follow it. |
|
||||
| 3 | `SnmpHealthProfile`. The application polls on its own; the four switch and phone drivers declare their OIDs as data. `HealthMetricsMixin` is deprecated. |
|
||||
| 4 | The fix templates. napalm-linux and napalm-bsd implement the steps; the application calls the templates with its values. `run_device_action` is deprecated. The other drivers follow, one issue each. |
|
||||
| 5 | Ping sweep, scoring, provisioning, `TYPE_LABEL` and `REBOOT_SETTLE_SECONDS` move to the application and are deprecated here. Drivers report facts instead of warnings. |
|
||||
| 6 | napalm-unix, with the moved code and its tests, the unified hooks and the frame helper. napalm-linux and napalm-bsd import from it; the moved names are deprecated here. |
|
||||
| 7 | 5.0 removes everything deprecated. It also splits `models.py` by area, types the remaining `Dict[str, Any]` role returns, declares what drivers already offer undeclared (`get_networks`, `search_packages`, `first_boot_pending`, the base-system update entry), and makes `install_package` declare the result drivers return. |
|
||||
| 8 | The README shrinks to the concepts; the reader details move with the readers. |
|
||||
|
||||
## Proposals this replaces
|
||||
|
||||
Several open issues proposed adding shared driver code here. Under this structure:
|
||||
|
||||
| Issue | Goes to |
|
||||
|---|---|
|
||||
| #8 netmiko session | not here: a CLI-driver helper package, or left in the drivers |
|
||||
| #9 config mode for CLI switches | not here: a CLI-driver helper package, or left in the drivers |
|
||||
| #10 LLDP rows | the projection of rows into NAPALM's two shapes stays here; the `lldpctl` parser goes to napalm-unix |
|
||||
| #11 CLI text helpers | not here; the natural port sort, which `lag` already has, may become public |
|
||||
| #12 ping | the reply builder stays here; the BusyBox and iputils parser goes to napalm-unix |
|
||||
| #13 Linux userland | napalm-unix; the snmpd part becomes the SNMP template's steps |
|
||||
| #14 `run_device_action` | replaced by the fix templates |
|
||||
| #22 hooks onto `run_command` | step 6 |
|
||||
@@ -22,6 +22,7 @@ driver do X".
|
||||
Role bases -- what a device *is*:
|
||||
|
||||
* :class:`~napalm_device_types.access_point.AccessPointDriver`
|
||||
* :class:`~napalm_device_types.backup_server.BackupServerDriver`
|
||||
* :class:`~napalm_device_types.switch.SwitchDriver`
|
||||
* :class:`~napalm_device_types.firewall.FirewallDriver`
|
||||
* :class:`~napalm_device_types.hypervisor.HypervisorDriver`
|
||||
@@ -29,79 +30,433 @@ Role bases -- what a device *is*:
|
||||
* :class:`~napalm_device_types.storage.StorageDriver`
|
||||
* :class:`~napalm_device_types.phone.PhoneDriver`
|
||||
* :class:`~napalm_device_types.media.MediaDriver`
|
||||
* :class:`~napalm_device_types.management_controller.ManagementControllerDriver`
|
||||
* :class:`~napalm_device_types.residential_gateway.ResidentialGatewayDriver`
|
||||
|
||||
Function classes -- what a device *can do*. Shared behaviour lives here once
|
||||
instead of being restated on every role that happens to need it:
|
||||
|
||||
* :class:`~napalm_device_types.channel.CommandChannelMixin`
|
||||
* :class:`~napalm_device_types.config_lifecycle.ConfigLifecycleMixin`
|
||||
* :class:`~napalm_device_types.config_sync.ConfigSyncMixin`
|
||||
* :class:`~napalm_device_types.container_engine.ContainerEngineMixin`
|
||||
* :class:`~napalm_device_types.dhcp.DhcpServerMixin`
|
||||
* :class:`~napalm_device_types.firewall_rules.FirewallRuleMixin`
|
||||
* :class:`~napalm_device_types.health_metrics.HealthMetricsMixin`
|
||||
* :class:`~napalm_device_types.host_status.HostStatusMixin`
|
||||
* :class:`~napalm_device_types.host_reboot.HostRebootMixin`
|
||||
* :class:`~napalm_device_types.host_shutdown.HostShutdownMixin`
|
||||
* :class:`~napalm_device_types.interface_filter.InterfaceFilterMixin`
|
||||
* :class:`~napalm_device_types.kernel.KernelFactsMixin`
|
||||
* :class:`~napalm_device_types.listening.ListeningSocketsMixin`
|
||||
* :class:`~napalm_device_types.local_accounts.LocalAccountsMixin`
|
||||
* :class:`~napalm_device_types.host_firewall.HostFirewallMixin`
|
||||
* :class:`~napalm_device_types.mac_acl.MacAclMixin`
|
||||
* :class:`~napalm_device_types.nat_vpn.NatVpnMixin`
|
||||
* :class:`~napalm_device_types.packages.PackageManagementMixin`
|
||||
* :class:`~napalm_device_types.ping_sweep.PingSweepMixin`
|
||||
* :class:`~napalm_device_types.service_clusters.ServiceClustersMixin`
|
||||
* :class:`~napalm_device_types.service_packages.ServicePackagesMixin`
|
||||
* :class:`~napalm_device_types.sudo_history.SudoHistoryMixin`
|
||||
* :class:`~napalm_device_types.scheduled_jobs.ScheduledJobsMixin`
|
||||
* :class:`~napalm_device_types.scheduled_jobs_posix.PosixScheduledJobsMixin`
|
||||
* :class:`~napalm_device_types.services.ServiceControlMixin`
|
||||
* :class:`~napalm_device_types.systemd.SystemdServicesMixin`
|
||||
* :class:`~napalm_device_types.updates.UpdateMixin`
|
||||
* :class:`~napalm_device_types.virtual_ips.VirtualIpsMixin`
|
||||
* :class:`~napalm_device_types.vm_stop_request.VmStopRequestMixin`
|
||||
|
||||
Introspection -- :func:`~napalm_device_types.roles.roles_of`,
|
||||
:func:`~napalm_device_types.roles.role_keys_of` and
|
||||
:func:`~napalm_device_types.roles.primary_role_of`.
|
||||
"""
|
||||
|
||||
from napalm_device_types.base import DeviceTypeDriver, FingerprintRule, PortSpec
|
||||
from napalm_device_types.access_point import AccessPointDriver
|
||||
from napalm_device_types.apt_proxy import (
|
||||
APT_PROXY_COMMAND,
|
||||
AptProxyMixin,
|
||||
https_proxy_for,
|
||||
parse_apt_fetch,
|
||||
parse_apt_proxy,
|
||||
same_proxy_url,
|
||||
)
|
||||
from napalm_device_types.arp import ARP_AGE_UNKNOWN
|
||||
from napalm_device_types.backup_server import BackupServerDriver
|
||||
from napalm_device_types.backups import (
|
||||
BACKUP_JOB_KINDS,
|
||||
BackupDatastoreDict,
|
||||
BackupGroupDict,
|
||||
BackupJobKind,
|
||||
BackupRemoteDict,
|
||||
BackupRunState,
|
||||
BackupServerJobDict,
|
||||
BackupServerRefDict,
|
||||
BackupSnapshotDict,
|
||||
BackupSnapshotRefDict,
|
||||
BackupTaskDict,
|
||||
BackupTaskStatusDict,
|
||||
BackupVerifyState,
|
||||
)
|
||||
from napalm_device_types.base import DeviceTypeDriver, FingerprintRule, PortSpec, rules_max_weight
|
||||
from napalm_device_types.channel import (
|
||||
ByteStream,
|
||||
CommandChannelMixin,
|
||||
CommandResult,
|
||||
ParamikoExecStream,
|
||||
open_stream_on_transport,
|
||||
run_on_transport,
|
||||
)
|
||||
from napalm_device_types.clusters import (
|
||||
BackupDict,
|
||||
BackupJobDict,
|
||||
ClusterDict,
|
||||
ClusterNetworkDict,
|
||||
ClusterNodeDict,
|
||||
ClusterStorageDict,
|
||||
HAResourceDict,
|
||||
)
|
||||
from napalm_device_types.config_lifecycle import ConfigLifecycleMixin
|
||||
from napalm_device_types.config_sync import ConfigSyncMixin
|
||||
from napalm_device_types.container_engine import (
|
||||
ContainerEngineConnection,
|
||||
ContainerEngineMixin,
|
||||
ContainerEngineUnavailable,
|
||||
)
|
||||
from napalm_device_types.dhcp import DhcpServerMixin, normalize_cidr, normalize_mac
|
||||
from napalm_device_types.dns_forward import DnsForwardMixin
|
||||
from napalm_device_types.firewall import FirewallDriver
|
||||
from napalm_device_types.hypervisor import HypervisorDriver
|
||||
from napalm_device_types.os import OSDriver
|
||||
from napalm_device_types.firewall_rules import FirewallRuleMixin
|
||||
from napalm_device_types.firmware import FirmwareOfferMixin
|
||||
from napalm_device_types.fixes import (
|
||||
EnsurePackageMixin,
|
||||
LldpAgentMixin,
|
||||
PackageProxyMixin,
|
||||
SnmpAgentMixin,
|
||||
run_fix_steps,
|
||||
)
|
||||
from napalm_device_types.health_metrics import HealthMetricsMixin
|
||||
from napalm_device_types.host_firewall import (
|
||||
HostFirewallMixin,
|
||||
HostFirewallUnavailable,
|
||||
parse_host_firewall,
|
||||
)
|
||||
from napalm_device_types.host_reboot import HostRebootMixin
|
||||
from napalm_device_types.host_shutdown import HostShutdownMixin
|
||||
from napalm_device_types.host_status import HOST_STATUS_COMMAND, HostStatusMixin, parse_host_status
|
||||
from napalm_device_types.hypervisor import HypervisorDriver
|
||||
from napalm_device_types.interface_filter import InterfaceFilterMixin
|
||||
from napalm_device_types.kernel import KERNEL_FACTS_COMMAND, KernelFactsMixin, parse_kernel_facts
|
||||
from napalm_device_types.lag import add_lag_interfaces
|
||||
from napalm_device_types.licenses import LicenseMixin
|
||||
from napalm_device_types.listening import (
|
||||
LISTENING_SOCKETS_COMMAND,
|
||||
ListeningSocketsMixin,
|
||||
ListeningSocketsUnavailable,
|
||||
parse_listening_sockets,
|
||||
)
|
||||
from napalm_device_types.lldp import (
|
||||
LldpNeighborDict,
|
||||
LldpNeighborRow,
|
||||
lldp_neighbors,
|
||||
lldp_neighbors_detail,
|
||||
)
|
||||
from napalm_device_types.lldpctl import LLDPCTL_KEYVALUE_COMMAND, parse_lldpctl_keyvalue
|
||||
from napalm_device_types.local_accounts import (
|
||||
LocalAccountChangeDict,
|
||||
LocalAccountDict,
|
||||
LocalAccountsMixin,
|
||||
)
|
||||
from napalm_device_types.mac_acl import MacAclMixin
|
||||
from napalm_device_types.management_controller import ManagementControllerDriver
|
||||
from napalm_device_types.media import MediaDriver
|
||||
from napalm_device_types.models import (
|
||||
POWER_ACTIONS,
|
||||
ConfigSyncDict,
|
||||
DhcpHaDict,
|
||||
DhcpHaPeerDict,
|
||||
DnsForwardDict,
|
||||
EventLogEntryDict,
|
||||
FirmwareDict,
|
||||
FirmwareOfferDict,
|
||||
FixResultDict,
|
||||
FixStepDict,
|
||||
FixStepOutcomeDict,
|
||||
HardwareInventoryDict,
|
||||
HypervisorHostDict,
|
||||
LicenseDict,
|
||||
ManagementProtocolsDict,
|
||||
OspfDict,
|
||||
OspfInterfaceDict,
|
||||
PowerAction,
|
||||
PowerStateDict,
|
||||
SnmpAgentSettingsDict,
|
||||
UpdateProgressDict,
|
||||
)
|
||||
from napalm_device_types.nat_vpn import NatVpnMixin
|
||||
from napalm_device_types.os import OSDriver
|
||||
from napalm_device_types.ospf import OspfMixin
|
||||
from napalm_device_types.package_updates import (
|
||||
APT_UPGRADABLE_COMMAND,
|
||||
DNF_SECURITY_COMMAND,
|
||||
parse_apt_upgradable,
|
||||
parse_dnf_security,
|
||||
)
|
||||
from napalm_device_types.packages import PackageManagementMixin
|
||||
from napalm_device_types.phone import PhoneDriver
|
||||
from napalm_device_types.ping_sweep import PingSweepMixin, driver_supports_ping
|
||||
from napalm_device_types.roles import primary_role_of, role_keys_of, roles_of
|
||||
from napalm_device_types.services import ServiceControlMixin
|
||||
from napalm_device_types.updates import UpdateMixin
|
||||
from napalm_device_types.provisioning import VMProvisionIncomplete
|
||||
from napalm_device_types.residential_gateway import ResidentialGatewayDriver
|
||||
from napalm_device_types.roles import primary_role_of, role_keys_of, roles_of
|
||||
from napalm_device_types.scheduled_jobs import (
|
||||
ScheduledJobChanged,
|
||||
ScheduledJobDict,
|
||||
ScheduledJobRefused,
|
||||
ScheduledJobResultDict,
|
||||
ScheduledJobsMixin,
|
||||
ScheduledJobSpecDict,
|
||||
ScheduledJobsReadingDict,
|
||||
job_digest,
|
||||
)
|
||||
from napalm_device_types.scheduled_jobs_posix import PosixScheduledJobsMixin
|
||||
from napalm_device_types.service_clusters import (
|
||||
SERVICE_CLUSTERS_COMMAND,
|
||||
ServiceClustersMixin,
|
||||
parse_service_clusters,
|
||||
service_clusters_command,
|
||||
)
|
||||
from napalm_device_types.service_packages import (
|
||||
SERVICE_PACKAGES_COMMAND,
|
||||
ServicePackagesMixin,
|
||||
parse_service_packages,
|
||||
)
|
||||
from napalm_device_types.services import ServiceControlMixin
|
||||
from napalm_device_types.snmp_health_profile import (
|
||||
SnmpHealthProfile,
|
||||
SnmpInterfacesSpec,
|
||||
SnmpMemorySpec,
|
||||
SnmpValueSpec,
|
||||
)
|
||||
from napalm_device_types.ssh_host_keys import (
|
||||
KNOWN_KEYS_ARG as SSH_HOST_KEYS_ARG,
|
||||
)
|
||||
from napalm_device_types.ssh_host_keys import (
|
||||
OBSERVER_ARG as SSH_HOST_KEY_OBSERVER_ARG,
|
||||
)
|
||||
from napalm_device_types.ssh_host_keys import (
|
||||
HostKeyCheck,
|
||||
HostKeyObservation,
|
||||
SshHostKey,
|
||||
SshHostKeyMismatch,
|
||||
)
|
||||
from napalm_device_types.ssh_server import (
|
||||
SSH_ALGORITHM_CLASSES,
|
||||
OpenSshServerMixin,
|
||||
SshHardeningPlan,
|
||||
SshHardeningResult,
|
||||
plan_ssh_hardening,
|
||||
)
|
||||
from napalm_device_types.storage import StorageDriver
|
||||
from napalm_device_types.sudo_history import (
|
||||
HistoryCleanupDict,
|
||||
HistorySudoLinesDict,
|
||||
SudoHistoryCleanupDict,
|
||||
SudoHistoryMixin,
|
||||
)
|
||||
from napalm_device_types.switch import SwitchDriver
|
||||
from napalm_device_types.system_identity import (
|
||||
SYSTEM_IDENTITY_COMMAND,
|
||||
SystemIdentityMixin,
|
||||
is_placeholder_identity,
|
||||
parse_system_identity,
|
||||
)
|
||||
from napalm_device_types.systemd import (
|
||||
SYSTEMD_SERVICES_COMMAND,
|
||||
SystemdServicesMixin,
|
||||
SystemdUnavailable,
|
||||
parse_systemd_services,
|
||||
)
|
||||
from napalm_device_types.terminal import strip_terminal_codes
|
||||
from napalm_device_types.updates import UpdateMixin
|
||||
from napalm_device_types.virtual_ips import (
|
||||
VIRTUAL_IPS_COMMAND,
|
||||
VirtualIpsMixin,
|
||||
parse_virtual_ips,
|
||||
)
|
||||
from napalm_device_types.vm_migration import MigrationPreflightDict
|
||||
from napalm_device_types.vm_stop_request import VmStopRequestMixin
|
||||
|
||||
__all__ = [
|
||||
"BACKUP_JOB_KINDS",
|
||||
"BackupDatastoreDict",
|
||||
"BackupGroupDict",
|
||||
"BackupJobKind",
|
||||
"BackupRemoteDict",
|
||||
"BackupRunState",
|
||||
"BackupServerJobDict",
|
||||
"BackupServerRefDict",
|
||||
"BackupSnapshotDict",
|
||||
"BackupSnapshotRefDict",
|
||||
"BackupTaskDict",
|
||||
"BackupTaskStatusDict",
|
||||
"BackupVerifyState",
|
||||
"BackupDict",
|
||||
"BackupJobDict",
|
||||
"ClusterDict",
|
||||
"ClusterNetworkDict",
|
||||
"ClusterNodeDict",
|
||||
"ClusterStorageDict",
|
||||
"HAResourceDict",
|
||||
"MigrationPreflightDict",
|
||||
"HypervisorHostDict",
|
||||
"DnsForwardDict",
|
||||
"DnsForwardMixin",
|
||||
"FirmwareOfferDict",
|
||||
"OspfDict",
|
||||
"OspfInterfaceDict",
|
||||
"OspfMixin",
|
||||
"UpdateProgressDict",
|
||||
"FirmwareOfferMixin",
|
||||
"LicenseDict",
|
||||
"LicenseMixin",
|
||||
"EnsurePackageMixin",
|
||||
"FixResultDict",
|
||||
"FixStepDict",
|
||||
"FixStepOutcomeDict",
|
||||
"LldpAgentMixin",
|
||||
"PackageProxyMixin",
|
||||
"APT_PROXY_COMMAND",
|
||||
"AptProxyMixin",
|
||||
"https_proxy_for",
|
||||
"parse_apt_fetch",
|
||||
"parse_apt_proxy",
|
||||
"same_proxy_url",
|
||||
"SnmpAgentMixin",
|
||||
"SnmpAgentSettingsDict",
|
||||
"run_fix_steps",
|
||||
"SnmpHealthProfile",
|
||||
"SnmpInterfacesSpec",
|
||||
"SnmpMemorySpec",
|
||||
"SnmpValueSpec",
|
||||
"AccessPointDriver",
|
||||
"BackupServerDriver",
|
||||
"ByteStream",
|
||||
"CommandChannelMixin",
|
||||
"CommandResult",
|
||||
"ContainerEngineConnection",
|
||||
"ContainerEngineMixin",
|
||||
"ContainerEngineUnavailable",
|
||||
"ParamikoExecStream",
|
||||
"open_stream_on_transport",
|
||||
"run_on_transport",
|
||||
"ConfigLifecycleMixin",
|
||||
"ConfigSyncDict",
|
||||
"ConfigSyncMixin",
|
||||
"DhcpHaDict",
|
||||
"DhcpHaPeerDict",
|
||||
"DeviceTypeDriver",
|
||||
"DhcpServerMixin",
|
||||
"FingerprintRule",
|
||||
"rules_max_weight",
|
||||
"FirewallDriver",
|
||||
"FirewallRuleMixin",
|
||||
"HealthMetricsMixin",
|
||||
"HOST_STATUS_COMMAND",
|
||||
"HostStatusMixin",
|
||||
"HostRebootMixin",
|
||||
"HostShutdownMixin",
|
||||
"HypervisorDriver",
|
||||
"InterfaceFilterMixin",
|
||||
"MacAclMixin",
|
||||
"ManagementControllerDriver",
|
||||
"ARP_AGE_UNKNOWN",
|
||||
"POWER_ACTIONS",
|
||||
"PowerAction",
|
||||
"PowerStateDict",
|
||||
"HardwareInventoryDict",
|
||||
"EventLogEntryDict",
|
||||
"FirmwareDict",
|
||||
"ManagementProtocolsDict",
|
||||
"MediaDriver",
|
||||
"NatVpnMixin",
|
||||
"OSDriver",
|
||||
"PackageManagementMixin",
|
||||
"APT_UPGRADABLE_COMMAND",
|
||||
"DNF_SECURITY_COMMAND",
|
||||
"parse_apt_upgradable",
|
||||
"parse_dnf_security",
|
||||
"parse_host_status",
|
||||
"strip_terminal_codes",
|
||||
"KernelFactsMixin",
|
||||
"KERNEL_FACTS_COMMAND",
|
||||
"parse_kernel_facts",
|
||||
"SERVICE_PACKAGES_COMMAND",
|
||||
"ServicePackagesMixin",
|
||||
"parse_service_packages",
|
||||
"LISTENING_SOCKETS_COMMAND",
|
||||
"HostFirewallMixin",
|
||||
"HostFirewallUnavailable",
|
||||
"parse_host_firewall",
|
||||
"ListeningSocketsMixin",
|
||||
"ListeningSocketsUnavailable",
|
||||
"parse_listening_sockets",
|
||||
"LLDPCTL_KEYVALUE_COMMAND",
|
||||
"LldpNeighborDict",
|
||||
"LldpNeighborRow",
|
||||
"lldp_neighbors",
|
||||
"lldp_neighbors_detail",
|
||||
"parse_lldpctl_keyvalue",
|
||||
"LocalAccountsMixin",
|
||||
"LocalAccountDict",
|
||||
"LocalAccountChangeDict",
|
||||
"PhoneDriver",
|
||||
"PingSweepMixin",
|
||||
"PortSpec",
|
||||
"ResidentialGatewayDriver",
|
||||
"ServiceControlMixin",
|
||||
"PosixScheduledJobsMixin",
|
||||
"ScheduledJobChanged",
|
||||
"ScheduledJobDict",
|
||||
"ScheduledJobRefused",
|
||||
"ScheduledJobResultDict",
|
||||
"ScheduledJobSpecDict",
|
||||
"ScheduledJobsMixin",
|
||||
"ScheduledJobsReadingDict",
|
||||
"job_digest",
|
||||
"StorageDriver",
|
||||
"SwitchDriver",
|
||||
"SYSTEMD_SERVICES_COMMAND",
|
||||
"SystemdServicesMixin",
|
||||
"SystemdUnavailable",
|
||||
"parse_systemd_services",
|
||||
"UpdateMixin",
|
||||
"VIRTUAL_IPS_COMMAND",
|
||||
"VirtualIpsMixin",
|
||||
"VmStopRequestMixin",
|
||||
"VMProvisionIncomplete",
|
||||
"SERVICE_CLUSTERS_COMMAND",
|
||||
"ServiceClustersMixin",
|
||||
"parse_service_clusters",
|
||||
"service_clusters_command",
|
||||
"SystemIdentityMixin",
|
||||
"SYSTEM_IDENTITY_COMMAND",
|
||||
"is_placeholder_identity",
|
||||
"parse_system_identity",
|
||||
"OpenSshServerMixin",
|
||||
"SSH_ALGORITHM_CLASSES",
|
||||
"SshHardeningPlan",
|
||||
"SshHardeningResult",
|
||||
"plan_ssh_hardening",
|
||||
"SudoHistoryMixin",
|
||||
"HistorySudoLinesDict",
|
||||
"HistoryCleanupDict",
|
||||
"SudoHistoryCleanupDict",
|
||||
"SSH_HOST_KEYS_ARG",
|
||||
"SSH_HOST_KEY_OBSERVER_ARG",
|
||||
"HostKeyCheck",
|
||||
"HostKeyObservation",
|
||||
"SshHostKey",
|
||||
"SshHostKeyMismatch",
|
||||
"parse_virtual_ips",
|
||||
"add_lag_interfaces",
|
||||
"driver_supports_ping",
|
||||
"normalize_cidr",
|
||||
|
||||
@@ -0,0 +1,222 @@
|
||||
"""Crontabs and anacrontabs as text: the jobs in them, and changing one line of them.
|
||||
|
||||
Pure: no I/O. :mod:`napalm_device_types.scheduled_jobs_posix` reads the files and
|
||||
writes them back; this module says what is in them and what the new text is.
|
||||
|
||||
**A line's key survives switching it off.** A job switched off is the same line
|
||||
behind the marker :data:`DISABLED_MARKER`; the key is built from the line without
|
||||
it, so the job keeps its key and the marker is all that changes. Two equal lines
|
||||
in one file get ``:2``, ``:3`` after the first.
|
||||
|
||||
**A job an application wrote** is the line after ``# napalm-scheduled-job: <name>``
|
||||
and is keyed ``job:<name>``, whatever its schedule.
|
||||
|
||||
**Everything else is left as it was**: comments, blank lines, variables, lines
|
||||
cron would not take.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import re
|
||||
from dataclasses import dataclass, field
|
||||
from shlex import quote
|
||||
|
||||
DISABLED_MARKER = "#napalm-disabled: "
|
||||
JOB_MARKER = "# napalm-scheduled-job: "
|
||||
|
||||
_VARIABLE = re.compile(r"^([A-Za-z_][A-Za-z0-9_]*)\s*=\s*(.*)$")
|
||||
_ANACRON_PERIOD = re.compile(r"^(\d+|@(daily|weekly|monthly|yearly|annually))$")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CronLine:
|
||||
"""One job in a crontab: where it is, what it says, and what was set before it."""
|
||||
|
||||
index: int # the line's number in the text, from 0
|
||||
key: str
|
||||
schedule: str
|
||||
run_as: str
|
||||
command: str
|
||||
enabled: bool
|
||||
env: dict[str, str] = field(default_factory=dict)
|
||||
name: str = "" # the name an application gave it
|
||||
description: str = ""
|
||||
|
||||
|
||||
def _unquote(value: str) -> str:
|
||||
value = value.strip()
|
||||
if len(value) >= 2 and value[0] == value[-1] and value[0] in "'\"":
|
||||
return value[1:-1]
|
||||
return value
|
||||
|
||||
|
||||
def _digest(text: str) -> str:
|
||||
return hashlib.sha256(" ".join(text.split()).encode()).hexdigest()[:8]
|
||||
|
||||
|
||||
def _split(content: str, *, system: bool, user: str) -> tuple[str, str, str] | None:
|
||||
"""(schedule, run_as, command) of a cron line; None for one cron would not take."""
|
||||
fields = 1 if content.startswith("@") else 5
|
||||
owner = 1 if system else 0
|
||||
parts = content.split(None, fields + owner)
|
||||
if len(parts) < fields + owner + 1:
|
||||
return None
|
||||
schedule = " ".join(parts[:fields])
|
||||
run_as = parts[fields] if system else user
|
||||
return schedule, run_as, parts[-1]
|
||||
|
||||
|
||||
def _anacron_split(content: str) -> tuple[str, str, str] | None:
|
||||
"""(period, job identifier, command) of an anacrontab line."""
|
||||
parts = content.split(None, 3)
|
||||
if len(parts) < 4 or not _ANACRON_PERIOD.match(parts[0]) or not parts[1].isdigit():
|
||||
return None
|
||||
return parts[0], parts[2], parts[3]
|
||||
|
||||
|
||||
@dataclass
|
||||
class _Scan:
|
||||
"""What is set while a text is read line by line."""
|
||||
|
||||
env: dict[str, str] = field(default_factory=dict)
|
||||
seen: dict[str, int] = field(default_factory=dict)
|
||||
name: str = ""
|
||||
description: str = ""
|
||||
|
||||
def key(self, base: str) -> str:
|
||||
n = self.seen.get(base, 0) + 1
|
||||
self.seen[base] = n
|
||||
return base if n == 1 else f"{base}:{n}"
|
||||
|
||||
def take_name(self) -> tuple[str, str]:
|
||||
named = self.name, self.description
|
||||
self.name, self.description = "", ""
|
||||
return named
|
||||
|
||||
|
||||
def _content(raw: str) -> tuple[str, bool] | None:
|
||||
"""The job text of *raw* and whether it is on; None for a comment or a blank."""
|
||||
line = raw.strip()
|
||||
if line.startswith(DISABLED_MARKER.strip()):
|
||||
return line[len(DISABLED_MARKER.strip()) :].strip(), False
|
||||
if not line or line.startswith("#"):
|
||||
return None
|
||||
return line, True
|
||||
|
||||
|
||||
def _remember(scan: _Scan, raw: str) -> bool:
|
||||
"""Take a variable or an application's marker from *raw*; whether it was one."""
|
||||
line = raw.strip()
|
||||
if line.startswith(JOB_MARKER.strip()):
|
||||
name, _, description = line[len(JOB_MARKER.strip()) :].strip().partition(" ")
|
||||
scan.name, scan.description = name, description.strip()
|
||||
return True
|
||||
variable = _VARIABLE.match(line)
|
||||
if variable:
|
||||
scan.env[variable[1]] = _unquote(variable[2])
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def parse_crontab(path: str, text: str, *, system: bool, user: str = "") -> list[CronLine]:
|
||||
"""The jobs of the crontab *text* at *path*.
|
||||
|
||||
*system*: a sixth field names the user (``/etc/crontab``, ``/etc/cron.d``);
|
||||
otherwise every job runs as *user*.
|
||||
"""
|
||||
scan, jobs = _Scan(), []
|
||||
for index, raw in enumerate(text.splitlines()):
|
||||
if _remember(scan, raw):
|
||||
continue
|
||||
found = _content(raw)
|
||||
split = _split(found[0], system=system, user=user) if found else None
|
||||
if found is None or split is None:
|
||||
continue
|
||||
content, enabled = found
|
||||
name, description = scan.take_name()
|
||||
key = f"job:{name}" if name else scan.key(f"cron:{path}:{_digest(content)}")
|
||||
schedule, run_as, command = split
|
||||
jobs.append(
|
||||
CronLine(
|
||||
index, key, schedule, run_as, command, enabled, dict(scan.env), name, description
|
||||
)
|
||||
)
|
||||
return jobs
|
||||
|
||||
|
||||
def parse_anacrontab(text: str) -> list[CronLine]:
|
||||
"""The jobs of an anacrontab: period, delay, identifier, command; all run as root."""
|
||||
scan, jobs = _Scan(), []
|
||||
for index, raw in enumerate(text.splitlines()):
|
||||
if _remember(scan, raw):
|
||||
continue
|
||||
found = _content(raw)
|
||||
split = _anacron_split(found[0]) if found else None
|
||||
if found is None or split is None:
|
||||
continue
|
||||
period, identifier, command = split
|
||||
key = scan.key(f"anacron:{identifier}")
|
||||
jobs.append(CronLine(index, key, period, "root", command, found[1], dict(scan.env)))
|
||||
return jobs
|
||||
|
||||
|
||||
def _replace(text: str, index: int, lines: list[str]) -> str:
|
||||
rows = text.splitlines()
|
||||
rows[index : index + 1] = lines
|
||||
return "\n".join(rows) + "\n"
|
||||
|
||||
|
||||
def switched(text: str, line: CronLine, *, enabled: bool) -> str:
|
||||
"""*text* with *line* switched on or off: the marker added or taken away, nothing else."""
|
||||
raw = text.splitlines()[line.index]
|
||||
found = _content(raw)
|
||||
if found is None: # pragma: no cover - a CronLine always points at a job
|
||||
raise ValueError(f"line {line.index} is no job")
|
||||
content = found[0]
|
||||
return _replace(text, line.index, [content if enabled else DISABLED_MARKER + content])
|
||||
|
||||
|
||||
def marked_lines(name: str, description: str, line: str, *, enabled: bool) -> list[str]:
|
||||
"""An application's job as crontab lines: its marker, then the job, maybe switched off."""
|
||||
marker = f"{JOB_MARKER}{name} {description}".rstrip()
|
||||
return [marker, line if enabled else DISABLED_MARKER + line]
|
||||
|
||||
|
||||
def with_job(text: str, jobs: list[CronLine], name: str, lines: list[str]) -> str:
|
||||
"""*text* with the application's job *name* as *lines*: replaced, or added at the end."""
|
||||
current = next((j for j in jobs if j.name == name), None)
|
||||
if current is None:
|
||||
body = text if not text or text.endswith("\n") else text + "\n"
|
||||
return body + "\n".join(lines) + "\n"
|
||||
rows = text.splitlines()
|
||||
rows[current.index - 1 : current.index + 1] = lines
|
||||
return "\n".join(rows) + "\n"
|
||||
|
||||
|
||||
def without_job(text: str, jobs: list[CronLine], name: str) -> str:
|
||||
"""*text* without the application's job *name* and its marker."""
|
||||
current = next(j for j in jobs if j.name == name)
|
||||
rows = text.splitlines()
|
||||
del rows[current.index - 1 : current.index + 1]
|
||||
return "\n".join(rows) + ("\n" if rows else "")
|
||||
|
||||
|
||||
def _unescape(part: str) -> str:
|
||||
return part.replace("\\%", "%")
|
||||
|
||||
|
||||
def run_script(command: str, env: dict[str, str]) -> str:
|
||||
"""*command* as ``sh -c`` takes it, the way cron would run it.
|
||||
|
||||
The crontab's variables are exported first. The first ``%`` that is not
|
||||
escaped ends the command; what follows is its stdin, with every further
|
||||
``%`` a newline, and ``\\%`` is a ``%``.
|
||||
"""
|
||||
parts = re.split(r"(?<!\\)%", command)
|
||||
exports = "".join(f"export {k}={quote(v)}; " for k, v in env.items() if k != "MAILTO")
|
||||
head = _unescape(parts[0])
|
||||
if len(parts) == 1:
|
||||
return exports + head
|
||||
stdin = "\n".join(_unescape(p) for p in parts[1:])
|
||||
return f"{exports}printf '%s\\n' {quote(stdin)} | {{ {head}\n}}"
|
||||
@@ -0,0 +1,46 @@
|
||||
"""Keying live and desired entries for a diff, refusing what cannot be matched.
|
||||
|
||||
A diff matches a desired entry to a live one by a key (a rule's description, a
|
||||
reservation's MAC, a subnet's CIDR). An empty key or one held twice cannot
|
||||
name a single entry, and guessing would overwrite one the caller never meant.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable, Iterable
|
||||
|
||||
|
||||
def index_live[T](entries: Iterable[T], key: Callable[[T], str]) -> tuple[dict[str, T], set[str]]:
|
||||
"""Live entries by key, and the keys more than one of them holds.
|
||||
|
||||
Entries without a key are left out: they are not the caller's to match.
|
||||
"""
|
||||
by_key: dict[str, T] = {}
|
||||
shared: set[str] = set()
|
||||
for entry in entries:
|
||||
k = key(entry)
|
||||
if not k:
|
||||
continue
|
||||
if k in by_key:
|
||||
shared.add(k)
|
||||
by_key[k] = entry
|
||||
for k in shared:
|
||||
del by_key[k]
|
||||
return by_key, shared
|
||||
|
||||
|
||||
def check_desired[T](
|
||||
entries: list[T], key: Callable[[T], str], shared: set[str], what: str
|
||||
) -> None:
|
||||
"""``ValueError`` for a desired entry with no key, a key given twice, or a
|
||||
key that names more than one live entry."""
|
||||
seen: set[str] = set()
|
||||
for entry in entries:
|
||||
k = key(entry)
|
||||
if not k:
|
||||
raise ValueError(f"a desired entry has no {what}")
|
||||
if k in seen:
|
||||
raise ValueError(f"{what} {k!r} is given twice")
|
||||
if k in shared:
|
||||
raise ValueError(f"{what} {k!r} names more than one entry on the device")
|
||||
seen.add(k)
|
||||
@@ -0,0 +1,417 @@
|
||||
"""The scheduled-jobs report of a POSIX host: the command that prints it, and its parse.
|
||||
|
||||
:mod:`napalm_device_types.scheduled_jobs_posix` sends :func:`reading_command` and
|
||||
hands what came back to :func:`parse_reading`. Pure: no I/O.
|
||||
|
||||
**One framed report, as root.** Every crontab, cron.d file, anacrontab and job
|
||||
script is printed behind a header line and with ``|`` before each of its lines,
|
||||
so no line of a file can pass for a header; run-parts scripts by their execute
|
||||
bit and checksum; systemd's timers and the units they start by ``systemctl show``;
|
||||
then the time zone and which package owns each definition file. A report whose
|
||||
end marker is missing raises: a list cut short must never read as jobs that went
|
||||
away.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import UTC, datetime
|
||||
from shlex import quote
|
||||
|
||||
from napalm_device_types._crontab import JOB_MARKER, CronLine, parse_anacrontab, parse_crontab
|
||||
from napalm_device_types.scheduled_jobs import DELETE, EDIT, job_digest
|
||||
from napalm_device_types.terminal import strip_terminal_codes
|
||||
|
||||
_BEGIN = "SJ_BEGIN"
|
||||
_END = "SJ_END"
|
||||
SCHEDULE_HEADER = "# schedule: "
|
||||
|
||||
TIMER_PROPERTIES = (
|
||||
"Id,Unit,Description,TimersCalendar,TimersMonotonic,NextElapseUSecRealtime,"
|
||||
"LastTriggerUSec,UnitFileState,ActiveState,FragmentPath"
|
||||
)
|
||||
UNIT_PROPERTIES = "Id,ExecStart,User,Result,FragmentPath"
|
||||
|
||||
#: The names run-parts runs (Debian's ``--lsbsysinit``-less default).
|
||||
_RUN_PARTS_NAME = re.compile(r"^[A-Za-z0-9_-]+$")
|
||||
_SECTION = re.compile(r"^\[([a-z-]+)(?: (.*))?\]$")
|
||||
|
||||
#: Reads definition paths on stdin, prints ``<package>: <path>`` for each a package owns.
|
||||
OWNERS_SH = (
|
||||
"if command -v dpkg >/dev/null 2>&1; then xargs dpkg -S 2>/dev/null; "
|
||||
"elif command -v rpm >/dev/null 2>&1; then while read -r p; do "
|
||||
'o=$(rpm -qf --qf \'%{NAME}\' -- "$p" 2>/dev/null) && printf \'%s: %s\\n\' "$o" "$p"; done; '
|
||||
"elif command -v apk >/dev/null 2>&1; then while read -r p; do "
|
||||
'o=$(apk info -qW -- "$p" 2>/dev/null) && [ -n "$o" ] && printf \'%s: %s\\n\' "$o" "$p"; done; '
|
||||
"else cat >/dev/null; fi"
|
||||
)
|
||||
|
||||
|
||||
#: Prints the host's time zone: an IANA name, or a path ending in ``zoneinfo/<name>``.
|
||||
TIME_ZONE_SH = "cat /etc/timezone 2>/dev/null || readlink /etc/localtime 2>/dev/null"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Paths:
|
||||
"""Where a host keeps its scheduled jobs."""
|
||||
|
||||
system_files: tuple[str, ...]
|
||||
cron_d_dirs: tuple[str, ...]
|
||||
spool_dirs: tuple[str, ...]
|
||||
run_parts: tuple[tuple[str, str], ...] # (period, directory)
|
||||
anacrontab: str
|
||||
script_dir: str
|
||||
systemd_marker: str = "/run/systemd/system"
|
||||
owners_sh: str = OWNERS_SH
|
||||
time_zone_sh: str = TIME_ZONE_SH
|
||||
|
||||
|
||||
@dataclass
|
||||
class Reading:
|
||||
"""What one report said, beyond the jobs: what writes need to know."""
|
||||
|
||||
jobs: list[dict[str, object]] = field(default_factory=list)
|
||||
time_zone: str | None = None
|
||||
systemd: bool = False
|
||||
#: per key: the unit a timer starts, the crontab line a cron job is
|
||||
units: dict[str, str] = field(default_factory=dict)
|
||||
lines: dict[str, CronLine] = field(default_factory=dict)
|
||||
|
||||
|
||||
def _q(path: str) -> str:
|
||||
return quote(path)
|
||||
|
||||
|
||||
def _files(section: str, pattern: str) -> str:
|
||||
return (
|
||||
f'for f in {pattern}; do [ -f "$f" ] || continue; printf \'[%s %s]\\n\' {section} "$f"; '
|
||||
# GNU sed keeps a missing last newline; without one of its own the next
|
||||
# header would run on into the file's last line.
|
||||
'sed \'s/^/|/\' "$f"; [ -n "$(tail -c 1 "$f")" ] && echo; P="$P\n$f"; done; '
|
||||
)
|
||||
|
||||
|
||||
def _run_parts(period: str, directory: str) -> str:
|
||||
return (
|
||||
f'for f in {_q(directory)}/*; do [ -f "$f" ] || continue; '
|
||||
'[ -x "$f" ] && x=1 || x=0; c=$(cksum < "$f" | cut -d" " -f1); '
|
||||
f'printf \'[run-parts %s %s %s %s]\\n\' {period} "$x" "$c" "$f"; P="$P\n$f"; done; '
|
||||
)
|
||||
|
||||
|
||||
def _timers(paths: Paths) -> str:
|
||||
show = f"systemctl show --no-pager -p {TIMER_PROPERTIES}"
|
||||
units = f"systemctl show --no-pager -p {UNIT_PROPERTIES}"
|
||||
return (
|
||||
f"if [ -d {_q(paths.systemd_marker)} ]; then echo '[systemd]'; "
|
||||
"t=$(systemctl list-unit-files --type=timer --no-legend --no-pager 2>/dev/null "
|
||||
"| awk '$1 !~ /@\\./ {print $1}'); "
|
||||
'if [ -n "$t" ]; then '
|
||||
f"s=$({show} --timestamp=unix $t 2>/dev/null || {show} $t 2>/dev/null); "
|
||||
"echo '[timers]'; printf '%s\\n' \"$s\" | sed 's/^/|/'; "
|
||||
"u=$(printf '%s\\n' \"$s\" | sed -n 's/^Unit=//p'); "
|
||||
f"echo '[units]'; {units} $u 2>/dev/null | sed 's/^/|/'; "
|
||||
"P=\"$P\n$(printf '%s\\n' \"$s\" | sed -n 's/^FragmentPath=//p')\"; fi; fi; "
|
||||
)
|
||||
|
||||
|
||||
def reading_command(paths: Paths) -> str:
|
||||
"""One POSIX ``sh`` script that prints every scheduled job of the host; run it as root."""
|
||||
parts = ["printf '%s%s\\n' SJ_ BEGIN; P=''; "]
|
||||
parts += [_files("system", _q(f)) for f in paths.system_files]
|
||||
parts += [_files("system", f"{_q(d)}/*") for d in paths.cron_d_dirs]
|
||||
parts += [_files("user", f"{_q(d)}/*") for d in paths.spool_dirs]
|
||||
parts.append(_files("anacron", _q(paths.anacrontab)))
|
||||
parts += [_run_parts(period, d) for period, d in paths.run_parts]
|
||||
parts.append(_files("script", f"{_q(paths.script_dir)}/*"))
|
||||
parts.append(_timers(paths))
|
||||
parts.append(
|
||||
f"echo '[tz]'; {{ {paths.time_zone_sh}; }}; "
|
||||
f"echo '[owners]'; printf '%s\\n' \"$P\" | sed '/^$/d' | {{ {paths.owners_sh}; }}; "
|
||||
"printf '%s%s\\n' SJ_ END"
|
||||
)
|
||||
return "".join(parts)
|
||||
|
||||
|
||||
# -- parse ------------------------------------------------------------------------------------
|
||||
|
||||
|
||||
@dataclass
|
||||
class _Section:
|
||||
name: str
|
||||
args: str
|
||||
lines: list[str] = field(default_factory=list)
|
||||
|
||||
|
||||
def _sections(output: str) -> list[_Section]:
|
||||
text = strip_terminal_codes(output)
|
||||
begin, end = text.rfind(_BEGIN), text.rfind(_END)
|
||||
if begin < 0 or end < begin:
|
||||
raise ValueError("the scheduled-jobs report is cut short (no end marker)")
|
||||
sections: list[_Section] = []
|
||||
for line in text[begin + len(_BEGIN) : end].splitlines():
|
||||
header = _SECTION.match(line)
|
||||
if header:
|
||||
sections.append(_Section(header[1], header[2] or ""))
|
||||
elif sections:
|
||||
sections[-1].lines.append(line[1:] if line.startswith("|") else line)
|
||||
return sections
|
||||
|
||||
|
||||
def _owned(lines: list[str]) -> set[str]:
|
||||
owned: set[str] = set()
|
||||
for line in lines:
|
||||
package, sep, path = line.partition(": ")
|
||||
if sep and path.startswith("/") and not package.startswith("diversion"):
|
||||
owned.add(path.strip())
|
||||
return owned
|
||||
|
||||
|
||||
def _time_zone(lines: list[str]) -> str | None:
|
||||
value = next((ln.strip() for ln in lines if ln.strip()), "")
|
||||
if "zoneinfo/" in value:
|
||||
value = value.split("zoneinfo/", 1)[1]
|
||||
return value or None
|
||||
|
||||
|
||||
def _blocks(lines: list[str]) -> list[dict[str, list[str]]]:
|
||||
blocks: list[dict[str, list[str]]] = [{}]
|
||||
for line in lines:
|
||||
if not line.strip():
|
||||
blocks.append({})
|
||||
continue
|
||||
key, sep, value = line.partition("=")
|
||||
if sep:
|
||||
blocks[-1].setdefault(key, []).append(value)
|
||||
return [b for b in blocks if b.get("Id")]
|
||||
|
||||
|
||||
def _one(block: dict[str, list[str]], key: str) -> str:
|
||||
return (block.get(key) or [""])[0].strip()
|
||||
|
||||
|
||||
def _timestamp(value: str) -> str | None:
|
||||
"""systemd's ``@<seconds>`` (or a UTC time) as ISO 8601; None when it never was."""
|
||||
value = value.strip()
|
||||
if value.startswith("@") and value[1:].isdigit() and int(value[1:]) > 0:
|
||||
return datetime.fromtimestamp(int(value[1:]), UTC).isoformat()
|
||||
match = re.search(r"(\d{4}-\d\d-\d\d \d\d:\d\d:\d\d) UTC$", value)
|
||||
if match:
|
||||
return datetime.strptime(match[1], "%Y-%m-%d %H:%M:%S").replace(tzinfo=UTC).isoformat()
|
||||
return None
|
||||
|
||||
|
||||
def _timer_entries(block: dict[str, list[str]], key: str) -> list[str]:
|
||||
"""``{ OnCalendar=daily ; next_elapse=… }`` → ``OnCalendar=daily``, one per entry.
|
||||
|
||||
``systemctl show`` names a monotonic setting by its property (``OnUnitActiveUSec``);
|
||||
it is given as the unit file says it (``OnUnitActiveSec``).
|
||||
"""
|
||||
entries = (v.strip(" {}").split(" ; ")[0].strip() for v in block.get(key, []) if v.strip())
|
||||
return [re.sub(r"^(On\w+)USec=", r"\1Sec=", e) for e in entries]
|
||||
|
||||
|
||||
def _argv(exec_start: list[str]) -> str:
|
||||
commands = []
|
||||
for value in exec_start:
|
||||
match = re.search(r"argv\[\]=(.*?) ; ", value)
|
||||
if match:
|
||||
commands.append(match[1].strip())
|
||||
return "; ".join(commands)
|
||||
|
||||
|
||||
@dataclass
|
||||
class _Script:
|
||||
name: str
|
||||
description: str
|
||||
schedule: str
|
||||
command: str
|
||||
|
||||
|
||||
def _script(path: str, lines: list[str]) -> _Script | None:
|
||||
"""An application's job script: its marker, schedule and command; None for any other file."""
|
||||
marker = next((ln for ln in lines if ln.startswith(JOB_MARKER)), None)
|
||||
schedule = next((ln for ln in lines if ln.startswith(SCHEDULE_HEADER)), None)
|
||||
if marker is None or schedule is None:
|
||||
return None
|
||||
name, _, description = marker[len(JOB_MARKER) :].partition(" ")
|
||||
body = [ln for ln in lines if ln not in (marker, schedule) and not ln.startswith("#!")]
|
||||
return _Script(
|
||||
name, description.strip(), schedule[len(SCHEDULE_HEADER) :], "\n".join(body).strip()
|
||||
)
|
||||
|
||||
|
||||
def _job(
|
||||
*,
|
||||
key: str,
|
||||
kind: str,
|
||||
source: str,
|
||||
run_as: str,
|
||||
schedule: str,
|
||||
schedule_kind: str,
|
||||
command: str,
|
||||
enabled: bool,
|
||||
managed_by: str,
|
||||
actions: list[str],
|
||||
content: str = "",
|
||||
**extra: object,
|
||||
) -> dict[str, object]:
|
||||
job: dict[str, object] = {
|
||||
"key": key,
|
||||
"kind": kind,
|
||||
"source": source,
|
||||
"run_as": run_as,
|
||||
"schedule": schedule,
|
||||
"schedule_kind": schedule_kind,
|
||||
"command": command,
|
||||
"enabled": enabled,
|
||||
"managed_by": managed_by,
|
||||
"actions": actions,
|
||||
}
|
||||
job["digest"] = job_digest(job, content=content)
|
||||
job.update({k: v for k, v in extra.items() if v not in (None, "")})
|
||||
return job
|
||||
|
||||
|
||||
_SWITCH = ["enable", "disable", "run"]
|
||||
_OWN = [*_SWITCH, EDIT, DELETE]
|
||||
|
||||
|
||||
def _cron_jobs(section: _Section, owned: set[str], reading: Reading) -> None:
|
||||
path = section.args
|
||||
text = "\n".join(section.lines)
|
||||
if section.name == "anacron":
|
||||
found = parse_anacrontab(text)
|
||||
else:
|
||||
user = path.rsplit("/", 1)[-1]
|
||||
found = parse_crontab(path, text, system=section.name == "system", user=user)
|
||||
kind = "anacron" if section.name == "anacron" else "cron"
|
||||
for line in found:
|
||||
own = bool(line.name)
|
||||
reading.lines[line.key] = line
|
||||
reading.jobs.append(
|
||||
_job(
|
||||
key=line.key,
|
||||
kind=kind,
|
||||
source=path,
|
||||
run_as=line.run_as,
|
||||
schedule=line.schedule,
|
||||
schedule_kind="period" if kind == "anacron" else "cron",
|
||||
command=line.command,
|
||||
enabled=line.enabled,
|
||||
managed_by="application" if own else ("package" if path in owned else "admin"),
|
||||
actions=_OWN if own else _SWITCH,
|
||||
name=line.name,
|
||||
description=line.description,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _run_parts_job(section: _Section, owned: set[str], reading: Reading) -> None:
|
||||
period, executable, checksum, path = section.args.split(" ", 3)
|
||||
if not _RUN_PARTS_NAME.match(path.rsplit("/", 1)[-1]):
|
||||
return
|
||||
reading.jobs.append(
|
||||
_job(
|
||||
key=f"run-parts:{path}",
|
||||
kind="run_parts",
|
||||
source=path,
|
||||
run_as="root",
|
||||
schedule=period,
|
||||
schedule_kind="period",
|
||||
command=path,
|
||||
enabled=executable == "1",
|
||||
managed_by="package" if path in owned else "admin",
|
||||
actions=_SWITCH,
|
||||
content=checksum,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _timer_job(
|
||||
timer: dict[str, list[str]],
|
||||
unit: dict[str, list[str]],
|
||||
scripts: dict[str, _Script],
|
||||
owned: set[str],
|
||||
) -> dict[str, object]:
|
||||
fragment = _one(timer, "FragmentPath")
|
||||
calendar = _timer_entries(timer, "TimersCalendar")
|
||||
schedule = "; ".join(calendar + _timer_entries(timer, "TimersMonotonic"))
|
||||
command = _argv(unit.get("ExecStart", []))
|
||||
script = scripts.get(command.split(" ", 1)[0])
|
||||
last_run = _timestamp(_one(timer, "LastTriggerUSec"))
|
||||
switchable = _one(timer, "UnitFileState") not in ("static", "generated", "transient")
|
||||
common = {
|
||||
"kind": "systemd_timer",
|
||||
"source": fragment,
|
||||
"run_as": _one(unit, "User") or "root",
|
||||
"enabled": _one(timer, "ActiveState") == "active",
|
||||
"last_run": last_run,
|
||||
"last_result": _one(unit, "Result") or None if last_run else None,
|
||||
"next_run": _timestamp(_one(timer, "NextElapseUSecRealtime")),
|
||||
}
|
||||
if script is not None:
|
||||
return _job(
|
||||
key=f"job:{script.name}",
|
||||
schedule=script.schedule,
|
||||
schedule_kind="cron",
|
||||
command=script.command,
|
||||
managed_by="application",
|
||||
actions=_OWN,
|
||||
name=script.name,
|
||||
description=script.description,
|
||||
**common, # type: ignore[arg-type]
|
||||
)
|
||||
packaged = fragment in owned or fragment.startswith(("/usr/lib/", "/lib/"))
|
||||
return _job(
|
||||
key=f"timer:{_one(timer, 'Id')}",
|
||||
schedule=schedule,
|
||||
schedule_kind="calendar" if calendar else "monotonic",
|
||||
command=command,
|
||||
managed_by="package" if packaged else "admin",
|
||||
actions=_SWITCH if switchable else ["run"],
|
||||
description=_one(timer, "Description"),
|
||||
**common, # type: ignore[arg-type]
|
||||
)
|
||||
|
||||
|
||||
def _timer_jobs(
|
||||
sections: dict[str, _Section], scripts: dict[str, _Script], owned: set[str], reading: Reading
|
||||
) -> None:
|
||||
timers = _blocks(sections["timers"].lines) if "timers" in sections else []
|
||||
units = (
|
||||
{_one(b, "Id"): b for b in _blocks(sections["units"].lines)} if "units" in sections else {}
|
||||
)
|
||||
for timer in timers:
|
||||
unit = units.get(_one(timer, "Unit"), {})
|
||||
job = _timer_job(timer, unit, scripts, owned)
|
||||
reading.units[str(job["key"])] = _one(timer, "Unit")
|
||||
reading.jobs.append(job)
|
||||
|
||||
|
||||
def parse_reading(output: str) -> Reading:
|
||||
"""The jobs, time zone and write context of a report from :func:`reading_command`.
|
||||
|
||||
:raises ValueError: when the report is cut short.
|
||||
"""
|
||||
sections = _sections(output)
|
||||
named = {s.name: s for s in sections if s.name in ("timers", "units", "tz", "owners")}
|
||||
owned = _owned(named["owners"].lines) if "owners" in named else set()
|
||||
reading = Reading(
|
||||
time_zone=_time_zone(named["tz"].lines) if "tz" in named else None,
|
||||
systemd=any(s.name == "systemd" for s in sections),
|
||||
)
|
||||
for section in sections:
|
||||
if section.name in ("system", "user", "anacron"):
|
||||
_cron_jobs(section, owned, reading)
|
||||
elif section.name == "run-parts":
|
||||
_run_parts_job(section, owned, reading)
|
||||
_timer_jobs(named, _scripts(sections), owned, reading)
|
||||
return reading
|
||||
|
||||
|
||||
def _scripts(sections: list[_Section]) -> dict[str, _Script]:
|
||||
"""An application's job scripts, by path."""
|
||||
scripts = ((s.args, _script(s.args, s.lines)) for s in sections if s.name == "script")
|
||||
return {path: script for path, script in scripts if script is not None}
|
||||
@@ -1,35 +1,43 @@
|
||||
"""Shared SNMP metric collection helpers for UCD-MIB and IF-MIB."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import re
|
||||
from typing import Awaitable, Callable, Dict, Optional
|
||||
from collections.abc import Awaitable, Callable
|
||||
from typing import Any
|
||||
|
||||
# ── UCD-MIB OIDs ──────────────────────────────────────────────────────────────
|
||||
OID_SYS_UPTIME = "1.3.6.1.2.1.25.1.1.0" # hrSystemUptime (centiseconds)
|
||||
OID_CPU_IDLE = "1.3.6.1.4.1.2021.11.11.0" # UCD ssCpuIdle (%)
|
||||
OID_MEM_TOTAL = "1.3.6.1.4.1.2021.4.5.0" # memTotalReal (kB)
|
||||
OID_MEM_FREE = "1.3.6.1.4.1.2021.4.6.0" # memAvailReal (kB)
|
||||
OID_MEM_BUFFER = "1.3.6.1.4.1.2021.4.14.0" # memBuffer (kB)
|
||||
OID_MEM_CACHED = "1.3.6.1.4.1.2021.4.15.0" # memCached (kB)
|
||||
OID_SWAP_TOTAL = "1.3.6.1.4.1.2021.4.3.0" # memTotalSwap (kB)
|
||||
OID_SWAP_AVAIL = "1.3.6.1.4.1.2021.4.4.0" # memAvailSwap (kB)
|
||||
OID_LOAD_1 = "1.3.6.1.4.1.2021.10.1.3.1" # laLoad 1-min
|
||||
OID_LOAD_5 = "1.3.6.1.4.1.2021.10.1.3.2" # laLoad 5-min
|
||||
OID_LOAD_15 = "1.3.6.1.4.1.2021.10.1.3.3" # laLoad 15-min
|
||||
OID_SYS_UPTIME = "1.3.6.1.2.1.25.1.1.0" # hrSystemUptime (centiseconds)
|
||||
OID_CPU_IDLE = "1.3.6.1.4.1.2021.11.11.0" # UCD ssCpuIdle (%)
|
||||
OID_MEM_TOTAL = "1.3.6.1.4.1.2021.4.5.0" # memTotalReal (kB)
|
||||
OID_MEM_FREE = "1.3.6.1.4.1.2021.4.6.0" # memAvailReal (kB)
|
||||
OID_MEM_BUFFER = "1.3.6.1.4.1.2021.4.14.0" # memBuffer (kB)
|
||||
OID_MEM_CACHED = "1.3.6.1.4.1.2021.4.15.0" # memCached (kB)
|
||||
OID_SWAP_TOTAL = "1.3.6.1.4.1.2021.4.3.0" # memTotalSwap (kB)
|
||||
OID_SWAP_AVAIL = "1.3.6.1.4.1.2021.4.4.0" # memAvailSwap (kB)
|
||||
OID_LOAD_1 = "1.3.6.1.4.1.2021.10.1.3.1" # laLoad 1-min
|
||||
OID_LOAD_5 = "1.3.6.1.4.1.2021.10.1.3.2" # laLoad 5-min
|
||||
OID_LOAD_15 = "1.3.6.1.4.1.2021.10.1.3.3" # laLoad 15-min
|
||||
|
||||
# ── IF-MIB OIDs ───────────────────────────────────────────────────────────────
|
||||
OID_IF_DESCR = "1.3.6.1.2.1.2.2.1.2"
|
||||
OID_IF_SPEED = "1.3.6.1.2.1.2.2.1.5"
|
||||
OID_IF_IN_OCT = "1.3.6.1.2.1.2.2.1.10"
|
||||
OID_IF_DESCR = "1.3.6.1.2.1.2.2.1.2"
|
||||
OID_IF_SPEED = "1.3.6.1.2.1.2.2.1.5"
|
||||
OID_IF_IN_OCT = "1.3.6.1.2.1.2.2.1.10"
|
||||
OID_IF_OUT_OCT = "1.3.6.1.2.1.2.2.1.16"
|
||||
OID_IF_IN_ERR = "1.3.6.1.2.1.2.2.1.14"
|
||||
OID_IF_IN_ERR = "1.3.6.1.2.1.2.2.1.14"
|
||||
OID_IF_OUT_ERR = "1.3.6.1.2.1.2.2.1.20"
|
||||
# ifXTable: 64-bit octets and Mbps. The 32-bit counters wrap every ~34 s at
|
||||
# 1 Gbit/s and ifSpeed stops at 4294 Mbps; an agent without ifXTable (SNMPv1)
|
||||
# falls back to them.
|
||||
OID_IF_HC_IN_OCT = "1.3.6.1.2.1.31.1.1.1.6"
|
||||
OID_IF_HC_OUT_OCT = "1.3.6.1.2.1.31.1.1.1.10"
|
||||
OID_IF_HIGH_SPEED = "1.3.6.1.2.1.31.1.1.1.15"
|
||||
|
||||
IF_SKIP_DEFAULT = re.compile(r'^(lo|sit\d|tun\d|docker|veth|br-|virbr)', re.IGNORECASE)
|
||||
IF_SKIP_DEFAULT = re.compile(r"^(lo|sit\d|tun\d|docker|veth|br-|virbr)", re.IGNORECASE)
|
||||
|
||||
SnmpGetFn = Callable[[str], Awaitable[Optional[str]]]
|
||||
SnmpWalkFn = Callable[[str], Awaitable[Dict[str, str]]]
|
||||
SnmpGetFn = Callable[[str], Awaitable[str | None]]
|
||||
SnmpWalkFn = Callable[[str], Awaitable[dict[str, str]]]
|
||||
|
||||
|
||||
def ticks_to_seconds(raw: str | None) -> int | None:
|
||||
@@ -42,36 +50,43 @@ def ticks_to_seconds(raw: str | None) -> int | None:
|
||||
|
||||
|
||||
def build_if_metrics(
|
||||
metrics: dict,
|
||||
descr: Dict[str, str],
|
||||
speed: Dict[str, str],
|
||||
in_oct: Dict[str, str],
|
||||
out_oct: Dict[str, str],
|
||||
in_err: Dict[str, str],
|
||||
out_err: Dict[str, str],
|
||||
metrics: dict[str, Any],
|
||||
descr: dict[str, str],
|
||||
speed: dict[str, str],
|
||||
in_oct: dict[str, str],
|
||||
out_oct: dict[str, str],
|
||||
in_err: dict[str, str],
|
||||
out_err: dict[str, str],
|
||||
*,
|
||||
if_skip: re.Pattern | None = None,
|
||||
if_skip: re.Pattern[str] | None = None,
|
||||
tx_err_is_drop: bool = False,
|
||||
hc_in_oct: dict[str, str] | None = None,
|
||||
hc_out_oct: dict[str, str] | None = None,
|
||||
high_speed: dict[str, str] | None = None,
|
||||
) -> None:
|
||||
if if_skip is None:
|
||||
if_skip = IF_SKIP_DEFAULT
|
||||
interfaces: dict = {}
|
||||
interfaces: dict[str, Any] = {}
|
||||
for idx, name in descr.items():
|
||||
if if_skip.match(name):
|
||||
continue
|
||||
iface: dict = {"name": name}
|
||||
iface: dict[str, Any] = {"name": name}
|
||||
try:
|
||||
iface["rx_bytes"] = int(in_oct.get(idx, 0) or 0)
|
||||
iface["tx_bytes"] = int(out_oct.get(idx, 0) or 0)
|
||||
iface["rx_errors"] = int(in_err.get(idx, 0) or 0)
|
||||
iface["rx_bytes"] = int((hc_in_oct or {}).get(idx) or in_oct.get(idx) or 0)
|
||||
iface["tx_bytes"] = int((hc_out_oct or {}).get(idx) or out_oct.get(idx) or 0)
|
||||
iface["rx_errors"] = int(in_err.get(idx, 0) or 0)
|
||||
tx_val = int(out_err.get(idx, 0) or 0)
|
||||
if tx_err_is_drop:
|
||||
iface["tx_errors"] = 0
|
||||
iface["tx_drops"] = tx_val
|
||||
iface["tx_drops"] = tx_val
|
||||
else:
|
||||
iface["tx_errors"] = tx_val
|
||||
high = (high_speed or {}).get(idx)
|
||||
spd = speed.get(idx)
|
||||
iface["speed_mbps"] = int(spd) // 1_000_000 if spd and int(spd) > 0 else 0
|
||||
if high and int(high) > 0:
|
||||
iface["speed_mbps"] = int(high)
|
||||
else:
|
||||
iface["speed_mbps"] = int(spd) // 1_000_000 if spd and int(spd) > 0 else 0
|
||||
except (ValueError, TypeError):
|
||||
pass
|
||||
interfaces[name] = iface
|
||||
@@ -84,15 +99,35 @@ async def collect_ucd_metrics(
|
||||
snmp_walk: SnmpWalkFn,
|
||||
*,
|
||||
tx_err_is_drop: bool = False,
|
||||
if_skip: re.Pattern | None = None,
|
||||
) -> dict:
|
||||
if_skip: re.Pattern[str] | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""Collect UCD-MIB system metrics + IF-MIB interface counters in parallel."""
|
||||
(
|
||||
uptime_raw, cpu_idle_raw,
|
||||
mem_total_raw, mem_free_raw, mem_buf_raw, mem_cache_raw,
|
||||
swap_total_raw, swap_avail_raw,
|
||||
load1_raw, load5_raw, load15_raw,
|
||||
), (descr, speed, in_oct, out_oct, in_err, out_err) = await asyncio.gather(
|
||||
(
|
||||
uptime_raw,
|
||||
cpu_idle_raw,
|
||||
mem_total_raw,
|
||||
mem_free_raw,
|
||||
mem_buf_raw,
|
||||
mem_cache_raw,
|
||||
swap_total_raw,
|
||||
swap_avail_raw,
|
||||
load1_raw,
|
||||
load5_raw,
|
||||
load15_raw,
|
||||
),
|
||||
(
|
||||
descr,
|
||||
speed,
|
||||
in_oct,
|
||||
out_oct,
|
||||
in_err,
|
||||
out_err,
|
||||
hc_in_oct,
|
||||
hc_out_oct,
|
||||
high_speed,
|
||||
),
|
||||
) = await asyncio.gather(
|
||||
asyncio.gather(
|
||||
snmp_get(OID_SYS_UPTIME),
|
||||
snmp_get(OID_CPU_IDLE),
|
||||
@@ -113,10 +148,13 @@ async def collect_ucd_metrics(
|
||||
snmp_walk(OID_IF_OUT_OCT),
|
||||
snmp_walk(OID_IF_IN_ERR),
|
||||
snmp_walk(OID_IF_OUT_ERR),
|
||||
snmp_walk(OID_IF_HC_IN_OCT),
|
||||
snmp_walk(OID_IF_HC_OUT_OCT),
|
||||
snmp_walk(OID_IF_HIGH_SPEED),
|
||||
),
|
||||
)
|
||||
|
||||
metrics: dict = {}
|
||||
metrics: dict[str, Any] = {}
|
||||
|
||||
secs = ticks_to_seconds(uptime_raw)
|
||||
if secs is not None:
|
||||
@@ -131,14 +169,14 @@ async def collect_ucd_metrics(
|
||||
if mem_total_raw and mem_free_raw:
|
||||
try:
|
||||
total_kb = int(mem_total_raw)
|
||||
free_kb = int(mem_free_raw)
|
||||
buf_kb = int(mem_buf_raw) if mem_buf_raw else 0
|
||||
free_kb = int(mem_free_raw)
|
||||
buf_kb = int(mem_buf_raw) if mem_buf_raw else 0
|
||||
cache_kb = int(mem_cache_raw) if mem_cache_raw else 0
|
||||
used_kb = max(0, total_kb - free_kb - buf_kb - cache_kb)
|
||||
used_kb = max(0, total_kb - free_kb - buf_kb - cache_kb)
|
||||
metrics["memory_total_bytes"] = total_kb * 1024
|
||||
metrics["memory_used_bytes"] = used_kb * 1024
|
||||
metrics["memory_percent"] = round(used_kb / total_kb * 100, 1) if total_kb else 0.0
|
||||
metrics["memory_free_bytes"] = free_kb * 1024
|
||||
metrics["memory_used_bytes"] = used_kb * 1024
|
||||
metrics["memory_percent"] = round(used_kb / total_kb * 100, 1) if total_kb else 0.0
|
||||
metrics["memory_free_bytes"] = free_kb * 1024
|
||||
except ValueError:
|
||||
pass
|
||||
|
||||
@@ -146,10 +184,10 @@ async def collect_ucd_metrics(
|
||||
try:
|
||||
stotal = int(swap_total_raw)
|
||||
savail = int(swap_avail_raw)
|
||||
sused = stotal - savail
|
||||
sused = stotal - savail
|
||||
metrics["swap_total_bytes"] = stotal * 1024
|
||||
metrics["swap_used_bytes"] = sused * 1024
|
||||
metrics["swap_percent"] = round(sused / stotal * 100, 1) if stotal else 0.0
|
||||
metrics["swap_used_bytes"] = sused * 1024
|
||||
metrics["swap_percent"] = round(sused / stotal * 100, 1) if stotal else 0.0
|
||||
except ValueError:
|
||||
pass
|
||||
|
||||
@@ -160,6 +198,18 @@ async def collect_ucd_metrics(
|
||||
except ValueError:
|
||||
pass
|
||||
|
||||
build_if_metrics(metrics, descr, speed, in_oct, out_oct, in_err, out_err,
|
||||
if_skip=if_skip, tx_err_is_drop=tx_err_is_drop)
|
||||
build_if_metrics(
|
||||
metrics,
|
||||
descr,
|
||||
speed,
|
||||
in_oct,
|
||||
out_oct,
|
||||
in_err,
|
||||
out_err,
|
||||
if_skip=if_skip,
|
||||
tx_err_is_drop=tx_err_is_drop,
|
||||
hc_in_oct=hc_in_oct,
|
||||
hc_out_oct=hc_out_oct,
|
||||
high_speed=high_speed,
|
||||
)
|
||||
return metrics
|
||||
@@ -10,14 +10,12 @@ Usage::
|
||||
...
|
||||
"""
|
||||
|
||||
from typing import Any, Dict, List, TYPE_CHECKING
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from napalm_device_types.base import DeviceTypeDriver
|
||||
from napalm_device_types.mac_acl import MacAclMixin
|
||||
from napalm_device_types.updates import UpdateMixin
|
||||
from napalm_device_types.services import ServiceControlMixin
|
||||
from napalm_device_types.packages import PackageManagementMixin
|
||||
from napalm_device_types.health_metrics import HealthMetricsMixin
|
||||
from napalm_device_types.interface_filter import InterfaceFilterMixin
|
||||
from napalm_device_types.mac_acl import MacAclMixin
|
||||
from napalm_device_types.models import (
|
||||
Dot1XConfigDict,
|
||||
FastTransitionConfigDict,
|
||||
@@ -26,9 +24,20 @@ from napalm_device_types.models import (
|
||||
SSIDBridgeDict,
|
||||
WirelessConfigDict,
|
||||
)
|
||||
from napalm_device_types.packages import PackageManagementMixin
|
||||
from napalm_device_types.services import ServiceControlMixin
|
||||
from napalm_device_types.updates import UpdateMixin
|
||||
|
||||
|
||||
class AccessPointDriver(MacAclMixin, UpdateMixin, ServiceControlMixin, PackageManagementMixin, HealthMetricsMixin, InterfaceFilterMixin, DeviceTypeDriver):
|
||||
class AccessPointDriver(
|
||||
MacAclMixin,
|
||||
UpdateMixin,
|
||||
ServiceControlMixin,
|
||||
PackageManagementMixin,
|
||||
HealthMetricsMixin,
|
||||
InterfaceFilterMixin,
|
||||
DeviceTypeDriver,
|
||||
):
|
||||
"""
|
||||
Abstract intermediate driver for wireless access points.
|
||||
|
||||
@@ -42,11 +51,6 @@ class AccessPointDriver(MacAclMixin, UpdateMixin, ServiceControlMixin, PackageMa
|
||||
ROLE: str = "access_point"
|
||||
TYPE_LABEL: str = "Access Point"
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
# AP-specific methods (get_wireless_clients, get_ssids, get_radio_status,
|
||||
# get_interfaces, get_vlans, …) are intentionally NOT defined here.
|
||||
# AccessPointDriver sits AFTER the mixin classes in the MRO of concrete
|
||||
@@ -86,7 +90,7 @@ class AccessPointDriver(MacAclMixin, UpdateMixin, ServiceControlMixin, PackageMa
|
||||
"""
|
||||
...
|
||||
|
||||
def get_fast_transition_config(self) -> Dict[str, FastTransitionConfigDict]:
|
||||
def get_fast_transition_config(self) -> dict[str, FastTransitionConfigDict]:
|
||||
"""
|
||||
Returns the 802.11r Fast BSS Transition (FT) configuration per SSID.
|
||||
|
||||
@@ -118,7 +122,7 @@ class AccessPointDriver(MacAclMixin, UpdateMixin, ServiceControlMixin, PackageMa
|
||||
"""
|
||||
...
|
||||
|
||||
def get_mesh_config(self) -> Dict[str, MeshConfigDict]:
|
||||
def get_mesh_config(self) -> dict[str, MeshConfigDict]:
|
||||
"""
|
||||
Returns the 802.11s mesh configuration per mesh interface.
|
||||
|
||||
@@ -148,7 +152,7 @@ class AccessPointDriver(MacAclMixin, UpdateMixin, ServiceControlMixin, PackageMa
|
||||
"""
|
||||
...
|
||||
|
||||
def get_mesh_peers(self) -> List[MeshPeerDict]:
|
||||
def get_mesh_peers(self) -> list[MeshPeerDict]:
|
||||
"""
|
||||
Returns a list of currently active 802.11s mesh peers.
|
||||
|
||||
@@ -178,7 +182,7 @@ class AccessPointDriver(MacAclMixin, UpdateMixin, ServiceControlMixin, PackageMa
|
||||
"""
|
||||
...
|
||||
|
||||
def get_ssid_bridge_config(self) -> Dict[str, SSIDBridgeDict]:
|
||||
def get_ssid_bridge_config(self) -> dict[str, SSIDBridgeDict]:
|
||||
"""
|
||||
Returns the Layer-2 bridging configuration for each SSID, i.e. which
|
||||
bridge interface and VLAN each SSID is mapped to.
|
||||
@@ -186,7 +190,8 @@ class AccessPointDriver(MacAclMixin, UpdateMixin, ServiceControlMixin, PackageMa
|
||||
Keys are SSID names. Each value contains:
|
||||
|
||||
* ssid (string) - SSID name (repeated for convenience)
|
||||
* bridge (string) - bridge interface the VAP is attached to (e.g. ``"br-lan"``, ``"br-guest"``)
|
||||
* bridge (string) - bridge interface the VAP is attached to
|
||||
(e.g. ``"br-lan"``, ``"br-guest"``)
|
||||
* vlan_id (int) - 802.1Q VLAN ID (0 = untagged / no VLAN separation)
|
||||
* tagged (bool) - whether traffic is 802.1Q-tagged on the uplink port
|
||||
* client_isolation (bool) - whether clients on this SSID are isolated from each other
|
||||
@@ -219,9 +224,7 @@ class AccessPointDriver(MacAclMixin, UpdateMixin, ServiceControlMixin, PackageMa
|
||||
"""
|
||||
...
|
||||
|
||||
|
||||
|
||||
def get_dot1x_config(self) -> Dict[str, Dot1XConfigDict]:
|
||||
def get_dot1x_config(self) -> dict[str, Dot1XConfigDict]:
|
||||
"""
|
||||
Returns the 802.1X / WPA-Enterprise (RADIUS) configuration per SSID.
|
||||
|
||||
@@ -268,12 +271,3 @@ class AccessPointDriver(MacAclMixin, UpdateMixin, ServiceControlMixin, PackageMa
|
||||
}
|
||||
"""
|
||||
...
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,665 @@
|
||||
"""apt's proxy: what apt fetches through, whether a proxy answers, and pointing apt at one.
|
||||
|
||||
Read the same way on every Debian-family host, so the commands and their parse
|
||||
live here once, like :mod:`~napalm_device_types.host_status`; a driver only
|
||||
carries a command across and runs one as root.
|
||||
|
||||
**What apt fetches through** is what ``apt-config dump`` says, not one file. apt
|
||||
reads every file in ``Dir::Etc::parts`` (``apt.conf.d``) in byte order, then
|
||||
``Dir::Etc::main`` (``apt.conf``); for a single value the last one read wins. The
|
||||
Debian installer writes its proxy into ``apt.conf``, Ubuntu's into
|
||||
``apt.conf.d/90curtin-aptproxy``: both win over an ``00proxy``
|
||||
(NetOrk/netork#1397). ``DIRECT`` for the proxy, or for one host
|
||||
(``Acquire::http::Proxy::<host>``), means no proxy there.
|
||||
|
||||
**https** goes through the http proxy too. apt's https method asks
|
||||
``Acquire::https::Proxy`` first and ``Acquire::http::Proxy`` after it, for one
|
||||
host and then for all (NetOrk/netork#1811). An http proxy alone therefore takes
|
||||
every https source as well, and a cache that does not tunnel them answers 403.
|
||||
|
||||
**Where it is set** is every line of apt's files that sets the http proxy, in the
|
||||
order apt reads them; the https proxy's lines are named apart. A line of the form
|
||||
``Acquire::http::Proxy "…";`` can be commented out; a setting inside a block
|
||||
(``Acquire::http { Proxy "…"; };``) is named but left alone. apt skips a file in
|
||||
``apt.conf.d`` whose name has other characters than letters, digits, ``_``, ``-``
|
||||
and ``.``, or an extension other than ``.conf`` (``00proxy.bak``, ``*.dpkg-old``);
|
||||
so is it here.
|
||||
|
||||
**Whether a proxy answers** is a fetch with apt's own downloader,
|
||||
``apt-helper download-file``, which is on every apt host and needs no root:
|
||||
straight to a page the caller names (past any proxy), or through the proxy apt is
|
||||
set to. An HTTP error is an answer too: the other end was reached. Whether the
|
||||
page is the right one, the caller says (``expect``).
|
||||
|
||||
**Pointing apt at a proxy** writes :attr:`AptProxyMixin.APT_PROXY_FILE`, comments
|
||||
out every other line that sets a different one, reads the result back from
|
||||
``apt-config``, and fetches the ``InRelease`` of one http source through it. The
|
||||
last step is what proves it: apt reached a mirror through the proxy it is now set
|
||||
to. The source chosen is one apt does not fetch ``DIRECT``. Then it fetches one
|
||||
https source the same way; where the proxy refuses it, https goes ``DIRECT``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import shlex
|
||||
from collections.abc import Callable
|
||||
from typing import TYPE_CHECKING
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
from typing_extensions import TypedDict
|
||||
|
||||
from napalm_device_types.fixes import PackageProxyMixin, run_fix_steps
|
||||
from napalm_device_types.models import FixResultDict, FixStepOutcomeDict
|
||||
from napalm_device_types.terminal import strip_terminal_codes
|
||||
|
||||
|
||||
class AptProxySettingDict(TypedDict):
|
||||
"""One line of apt's configuration that sets a proxy (``AptProxyMixin``)."""
|
||||
|
||||
file: str
|
||||
line: int # 1-based
|
||||
value: str
|
||||
#: True for ``Acquire::http::Proxy "…";`` on a line of its own, which can be
|
||||
#: commented out; False for a setting inside a block.
|
||||
editable: bool
|
||||
|
||||
|
||||
class AptProxyReportDict(TypedDict):
|
||||
"""What apt fetches through, and where that is set (``get_package_proxy_report``)."""
|
||||
|
||||
#: The http proxy apt fetches through; None for none or ``DIRECT``.
|
||||
proxy: str | None
|
||||
#: ``Acquire::http::Proxy::<host>``: a proxy, or ``DIRECT``, for one host.
|
||||
per_host: dict[str, str]
|
||||
#: ``Acquire::http::Proxy-Auto-Detect``: a program that names the proxy.
|
||||
auto_detect: str | None
|
||||
#: Every line that sets it, in the order apt reads them; the last one wins.
|
||||
settings: list[AptProxySettingDict]
|
||||
#: The proxy apt takes for https: ``Acquire::https::Proxy``, else the http one;
|
||||
#: None for none or ``DIRECT``.
|
||||
https_proxy: str | None
|
||||
#: The per-host proxies https takes: ``Acquire::https::Proxy::<host>``, else
|
||||
#: ``Acquire::http::Proxy::<host>``.
|
||||
https_per_host: dict[str, str]
|
||||
#: Every line that sets ``Acquire::https::Proxy``, in the order apt reads them.
|
||||
https_settings: list[AptProxySettingDict]
|
||||
#: The base URIs of the host's sources over http and over https.
|
||||
http_sources: list[str]
|
||||
https_sources: list[str]
|
||||
|
||||
|
||||
class ProxyFetchDict(TypedDict):
|
||||
"""A fetch from the host with apt's downloader (``check_package_proxy``)."""
|
||||
|
||||
#: Whether the other end answered at all; an HTTP error is an answer.
|
||||
reachable: bool
|
||||
#: Whether the page contained what the caller expected; None when it named nothing
|
||||
#: or nothing came back.
|
||||
expected: bool | None
|
||||
#: apt's reason when the fetch failed, else "".
|
||||
error: str
|
||||
|
||||
|
||||
class HttpsSourceFetchDict(ProxyFetchDict):
|
||||
"""One https source fetched the way apt fetches it (``check_https_source``)."""
|
||||
|
||||
#: What was fetched: the source's ``InRelease``.
|
||||
url: str
|
||||
#: The proxy apt took for it.
|
||||
proxy: str
|
||||
|
||||
|
||||
_BEGIN = "APROXY_BEGIN"
|
||||
_END = "APROXY_END"
|
||||
_APT_HELPER = "/usr/lib/apt/apt-helper"
|
||||
|
||||
#: A proxy URL apt's file and a shell word can both carry.
|
||||
PROXY_URL = re.compile(r"^https?://[A-Za-z0-9.\-\[\]:@%_~/]+$")
|
||||
|
||||
#: One line, POSIX ``sh``, read-only, no privileges. In a subshell, so neither the
|
||||
#: locale nor its variables stay behind in an interactive session; the frame
|
||||
#: markers are printed in two halves so that an echoing transport does not show
|
||||
#: them early.
|
||||
APT_PROXY_COMMAND = (
|
||||
"( export LC_ALL=C; printf '%s%s\\n' APROXY_ BEGIN; "
|
||||
"echo '[dump]'; apt-config dump 2>/dev/null | grep -i -E '^Acquire::https?::Proxy'; "
|
||||
'eval "$(apt-config shell ap_parts Dir::Etc::parts/d ap_main Dir::Etc::main/f 2>/dev/null)"; '
|
||||
"echo '[parts]'; for f in \"${ap_parts:-/etc/apt/apt.conf.d/}\"*; do "
|
||||
'[ -f "$f" ] && grep -Hin proxy "$f"; done; '
|
||||
'echo \'[main]\'; f="${ap_main:-/etc/apt/apt.conf}"; [ -f "$f" ] && grep -Hin proxy "$f"; '
|
||||
"echo '[sources]'; apt-get indextargets --no-release-info --format '$(BASE_URI)' "
|
||||
"2>/dev/null | sort -u; "
|
||||
"printf '%s%s\\n' APROXY_ END ) 2>/dev/null | cat"
|
||||
)
|
||||
|
||||
_DUMP = re.compile(r'^Acquire::(https?)::Proxy(::\S+|-Auto-Detect)?\s+"([^"]*)"', re.I)
|
||||
_GREP = re.compile(r"^(/[^:]+):(\d+):(.*)$")
|
||||
_FLAT = re.compile(r'^\s*Acquire::http::Proxy\s+"([^"]*)"\s*;', re.I)
|
||||
_NOT_HTTP_PROXY = re.compile(
|
||||
r"^\s*(//|#)|Acquire::(https|ftp|socks)\b|::Proxy::|Proxy-Auto-Detect", re.I
|
||||
)
|
||||
_FLAT_HTTPS = re.compile(r'^\s*Acquire::https::Proxy\s+"([^"]*)"\s*;', re.I)
|
||||
#: A line that sets the https proxy inside a block, all on the line itself.
|
||||
_HTTPS_BLOCK = re.compile(r'^\s*Acquire::https\s*\{[^}]*\bProxy\s+"([^"]*)"', re.I)
|
||||
_IN_BLOCK = re.compile(r'\bProxy\s+"([^"]*)"', re.I)
|
||||
#: The names apt reads in ``apt.conf.d``: no extension, or ``.conf``.
|
||||
_PART_NAME = re.compile(r"^(?:[A-Za-z0-9_\-]+|[A-Za-z0-9_.\-]+\.conf)$")
|
||||
_NO_PROXY = frozenset({"", "direct"})
|
||||
|
||||
|
||||
def _sections(output: str) -> dict[str, list[str]]:
|
||||
lines = [line.rstrip() for line in strip_terminal_codes(output).splitlines()]
|
||||
stripped = [line.strip() for line in lines]
|
||||
try:
|
||||
start = stripped.index(_BEGIN)
|
||||
end = stripped.index(_END, start)
|
||||
except ValueError:
|
||||
raise ValueError("no intact apt proxy report in the output") from None
|
||||
sections: dict[str, list[str]] = {}
|
||||
current: list[str] = []
|
||||
for line, bare in zip(lines[start + 1 : end], stripped[start + 1 : end], strict=True):
|
||||
if bare.startswith("[") and bare.endswith("]") and " " not in bare:
|
||||
current = sections.setdefault(bare[1:-1], [])
|
||||
elif bare:
|
||||
current.append(line)
|
||||
return sections
|
||||
|
||||
|
||||
def _grep_line(line: str, main: bool) -> tuple[str, int, str] | None:
|
||||
"""File, line number and text of one line grep printed; None for a file apt skips."""
|
||||
found = _GREP.match(line)
|
||||
if not found:
|
||||
return None
|
||||
path, number, text = found.groups()
|
||||
if not main and not _PART_NAME.match(path.rsplit("/", 1)[-1]):
|
||||
return None
|
||||
return path, int(number), text
|
||||
|
||||
|
||||
def _setting(line: str, main: bool) -> AptProxySettingDict | None:
|
||||
found = _grep_line(line, main)
|
||||
if not found:
|
||||
return None
|
||||
path, number, text = found
|
||||
flat = _FLAT.match(text)
|
||||
if flat:
|
||||
return {"file": path, "line": int(number), "value": flat.group(1), "editable": True}
|
||||
if _NOT_HTTP_PROXY.search(text):
|
||||
return None
|
||||
nested = _IN_BLOCK.search(text)
|
||||
if not nested:
|
||||
return None
|
||||
return {"file": path, "line": int(number), "value": nested.group(1), "editable": False}
|
||||
|
||||
|
||||
def _https_setting(line: str, main: bool) -> AptProxySettingDict | None:
|
||||
found = _grep_line(line, main)
|
||||
if not found:
|
||||
return None
|
||||
path, number, text = found
|
||||
flat = _FLAT_HTTPS.match(text)
|
||||
if flat:
|
||||
return {"file": path, "line": number, "value": flat.group(1), "editable": True}
|
||||
block = _HTTPS_BLOCK.match(text)
|
||||
if block:
|
||||
return {"file": path, "line": number, "value": block.group(1), "editable": False}
|
||||
return None
|
||||
|
||||
|
||||
class _Dump(TypedDict):
|
||||
proxy: str | None
|
||||
per_host: dict[str, str]
|
||||
auto_detect: str | None
|
||||
#: ``Acquire::https::Proxy`` as written; None when unset or empty.
|
||||
https: str | None
|
||||
https_per_host: dict[str, str]
|
||||
|
||||
|
||||
def _dump(lines: list[str]) -> _Dump:
|
||||
"""The proxies, the per-host proxies and the auto-detect program in ``apt-config dump``."""
|
||||
dump: _Dump = {
|
||||
"proxy": None,
|
||||
"per_host": {},
|
||||
"auto_detect": None,
|
||||
"https": None,
|
||||
"https_per_host": {},
|
||||
}
|
||||
for found in (_DUMP.match(line.strip()) for line in lines):
|
||||
if found:
|
||||
_take(dump, *found.groups())
|
||||
return dump
|
||||
|
||||
|
||||
def _take(dump: _Dump, scheme: str, suffix: str | None, value: str) -> None:
|
||||
"""One line of the dump into *dump*; a later one of the same key wins, as in apt."""
|
||||
https = scheme.lower() == "https"
|
||||
if not suffix:
|
||||
_take_for_all(dump, https, value)
|
||||
elif suffix.lower() == "-auto-detect":
|
||||
if not https:
|
||||
dump["auto_detect"] = value or None
|
||||
elif value:
|
||||
dump["https_per_host" if https else "per_host"][suffix[2:]] = value
|
||||
|
||||
|
||||
def _take_for_all(dump: _Dump, https: bool, value: str) -> None:
|
||||
"""The proxy for every host: https keeps an empty value as unset, apt's fallback."""
|
||||
if https:
|
||||
dump["https"] = value or None
|
||||
else:
|
||||
dump["proxy"] = None if value.lower() in _NO_PROXY else value
|
||||
|
||||
|
||||
def _https_proxy(dump: _Dump) -> str | None:
|
||||
"""apt's https method takes its own setting and falls back to the http one."""
|
||||
if dump["https"] is None:
|
||||
return dump["proxy"]
|
||||
return None if dump["https"].lower() in _NO_PROXY else dump["https"]
|
||||
|
||||
|
||||
def https_proxy_for(report: AptProxyReportDict, host: str) -> str | None:
|
||||
"""The proxy apt takes for an https source on *host*; None for ``DIRECT`` or none.
|
||||
|
||||
The host's own setting first, https before http, then the one for all hosts.
|
||||
"""
|
||||
for per_host in (report["https_per_host"], report["per_host"]):
|
||||
value = next((v for h, v in per_host.items() if h.lower() == host.lower()), None)
|
||||
if value:
|
||||
return None if value.lower() in _NO_PROXY else value
|
||||
return report["https_proxy"]
|
||||
|
||||
|
||||
def parse_apt_proxy(output: str) -> AptProxyReportDict:
|
||||
"""The report :data:`APT_PROXY_COMMAND` prints, as :class:`AptProxyReportDict`.
|
||||
|
||||
:raises ValueError: if the output carried no intact report.
|
||||
"""
|
||||
sections = _sections(output)
|
||||
dump = _dump(sections.get("dump", []))
|
||||
lines = [
|
||||
(main, line)
|
||||
for main, key in ((False, "parts"), (True, "main"))
|
||||
for line in sections.get(key, [])
|
||||
]
|
||||
sources = [line.strip() for line in sections.get("sources", [])]
|
||||
return {
|
||||
"proxy": dump["proxy"],
|
||||
"per_host": dump["per_host"],
|
||||
"auto_detect": dump["auto_detect"],
|
||||
"settings": _settings(lines, _setting),
|
||||
"https_proxy": _https_proxy(dump),
|
||||
"https_per_host": {**dump["per_host"], **dump["https_per_host"]},
|
||||
"https_settings": _settings(lines, _https_setting),
|
||||
"http_sources": _of_scheme(sources, "http"),
|
||||
"https_sources": _of_scheme(sources, "https"),
|
||||
}
|
||||
|
||||
|
||||
def _settings(
|
||||
lines: list[tuple[bool, str]], parse: Callable[[str, bool], AptProxySettingDict | None]
|
||||
) -> list[AptProxySettingDict]:
|
||||
return [s for s in (parse(line, main) for main, line in lines) if s]
|
||||
|
||||
|
||||
def _of_scheme(sources: list[str], scheme: str) -> list[str]:
|
||||
return [s for s in sources if s.startswith(f"{scheme}://")]
|
||||
|
||||
|
||||
_DEFAULT_PORT = {"http": 80, "https": 443}
|
||||
|
||||
|
||||
def _endpoint(url: str | None) -> tuple[str, str, int | None, str] | None:
|
||||
"""Scheme, host, port (the scheme's own when none is given) and path; None for no URL."""
|
||||
if not url:
|
||||
return None
|
||||
try:
|
||||
parts = urlsplit(url)
|
||||
port = parts.port or _DEFAULT_PORT.get(parts.scheme)
|
||||
except ValueError:
|
||||
return None
|
||||
return parts.scheme, parts.hostname or "", port, parts.path.rstrip("/")
|
||||
|
||||
|
||||
def same_proxy_url(a: str | None, b: str | None) -> bool:
|
||||
"""Whether *a* and *b* name the same proxy: scheme, host and port, a trailing ``/`` aside."""
|
||||
first = _endpoint(a)
|
||||
return first is not None and first == _endpoint(b)
|
||||
|
||||
|
||||
def apt_fetch_command(url: str, *, direct: bool = False, expect: str = "") -> str:
|
||||
"""The command that fetches *url* with apt's downloader into a temporary file.
|
||||
|
||||
*direct* goes past any proxy apt is set to, to *url*'s host itself; without it
|
||||
the fetch takes apt's own way. *expect* is text the page should contain.
|
||||
|
||||
:raises ValueError: if *url* is not an http(s) URL a shell word can carry.
|
||||
"""
|
||||
if not PROXY_URL.match(url):
|
||||
raise ValueError(f"not an http URL: {url!r}")
|
||||
options = ["-o Acquire::Retries=0", "-o Acquire::http::Timeout=10"]
|
||||
host = urlsplit(url).hostname or ""
|
||||
if direct and host and ":" not in host:
|
||||
# *host* comes from a URL PROXY_URL let through: no quote in it. For https
|
||||
# the host's https setting comes first, so both are set.
|
||||
if url.startswith("https://"):
|
||||
options.append(f"-o 'Acquire::https::Proxy::{host}=DIRECT'")
|
||||
options.append(f"-o 'Acquire::http::Proxy::{host}=DIRECT'")
|
||||
match = f"echo '[match]'; grep -c -F -- {shlex.quote(expect)} \"$t\"; " if expect else ""
|
||||
return (
|
||||
"( export LC_ALL=C; t=$(mktemp) || exit 1; printf '%s%s\\n' APROXY_ BEGIN; "
|
||||
f"echo '[fetch]'; {_APT_HELPER} {' '.join(options)} download-file "
|
||||
f'{shlex.quote(url)} "$t" 2>&1; rc=$?; echo \'[rc]\'; echo "$rc"; '
|
||||
f"{match}rm -f \"$t\"; printf '%s%s\\n' APROXY_ END ) | cat"
|
||||
)
|
||||
|
||||
|
||||
_FAILED = re.compile(r"^E: Failed to fetch \S+\s+(.*)$")
|
||||
_HTTP_STATUS = re.compile(r"^\d{3}\s")
|
||||
|
||||
|
||||
def parse_apt_fetch(output: str) -> ProxyFetchDict:
|
||||
"""What :func:`apt_fetch_command` printed, as :class:`ProxyFetchDict`.
|
||||
|
||||
:raises ValueError: if the output carried no intact report, or apt-helper is
|
||||
missing -- then nothing could be tried.
|
||||
"""
|
||||
sections = _sections(output)
|
||||
try:
|
||||
rc = int(sections["rc"][0])
|
||||
except (KeyError, IndexError, ValueError):
|
||||
raise ValueError("no exit status in the apt fetch report") from None
|
||||
if rc in (126, 127):
|
||||
raise ValueError(f"{_APT_HELPER} cannot run on this host")
|
||||
match = sections.get("match")
|
||||
if rc == 0:
|
||||
expected = (match[0].strip() not in ("", "0")) if match else None
|
||||
return {"reachable": True, "expected": expected, "error": ""}
|
||||
error = next(
|
||||
(m.group(1).strip() for m in map(_FAILED.match, sections.get("fetch", [])) if m),
|
||||
"the download failed",
|
||||
)
|
||||
answered = bool(_HTTP_STATUS.match(error))
|
||||
return {"reachable": answered, "expected": False if answered else None, "error": error}
|
||||
|
||||
|
||||
#: sed's edit of one line: comment it out, if it still sets the proxy of *scheme*.
|
||||
_COMMENT_OUT = "-e '{line}{{/^[[:space:]]*Acquire::{scheme}::Proxy[[:space:]]/Is|^|// |}}'"
|
||||
_SCHEMES = ("http", "https")
|
||||
|
||||
|
||||
def comment_out_command(path: str, lines: list[int], scheme: str = "http") -> str:
|
||||
"""The root command that comments out *lines* of *path* where they set *scheme*'s proxy."""
|
||||
if scheme not in _SCHEMES:
|
||||
raise ValueError(f"not a scheme apt proxies: {scheme!r}")
|
||||
edits = " ".join(_COMMENT_OUT.format(line=int(n), scheme=scheme) for n in lines)
|
||||
return f"sed -i {edits} {shlex.quote(path)}"
|
||||
|
||||
|
||||
def _editable_lines(settings: list[AptProxySettingDict]) -> dict[str, list[int]]:
|
||||
by_file: dict[str, list[int]] = {}
|
||||
for s in settings:
|
||||
if s["editable"]:
|
||||
by_file.setdefault(s["file"], []).append(s["line"])
|
||||
return by_file
|
||||
|
||||
|
||||
def _described(s: AptProxySettingDict) -> str:
|
||||
what = "commented out" if s["editable"] else "left alone (inside a block)"
|
||||
return f"{s['file']}:{s['line']} ({s['value']}) {what}"
|
||||
|
||||
|
||||
class AptProxyMixin(PackageProxyMixin):
|
||||
""":class:`~napalm_device_types.fixes.PackageProxyMixin` for a Debian-family host.
|
||||
|
||||
The template form, like :class:`~napalm_device_types.host_status.HostStatusMixin`:
|
||||
the reading, the parse and the order of the fix are the same everywhere; a
|
||||
driver supplies the transports below. Mixed in by the drivers that can, so
|
||||
``callable(getattr(driver, "configure_package_proxy", None))`` stays a truthful
|
||||
answer.
|
||||
"""
|
||||
|
||||
#: The file the proxy is written to. Its name carries no application's.
|
||||
APT_PROXY_FILE = "/etc/apt/apt.conf.d/00proxy"
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def _run_apt_command(self, command: str) -> str:
|
||||
"""Run *command* on the host with ``sh``, without privileges; what it printed."""
|
||||
...
|
||||
|
||||
def _run_apt_root_command(self, command: str, done: str) -> FixStepOutcomeDict:
|
||||
"""Run *command* as root; ``ok`` with *done* when it exited 0."""
|
||||
...
|
||||
|
||||
def _write_apt_file(self, path: str, content: str) -> FixStepOutcomeDict:
|
||||
"""Replace *path* with *content*, as a new file owned by root, mode 644."""
|
||||
...
|
||||
|
||||
def _is_apt_host(self) -> bool:
|
||||
"""Whether apt is the host's package manager; a driver that can tell overrides it."""
|
||||
return True
|
||||
|
||||
def get_package_proxy_report(self) -> AptProxyReportDict | None:
|
||||
"""What apt fetches through and where that is set; None on a host without apt.
|
||||
|
||||
* proxy (string or None): the http proxy apt fetches through
|
||||
* per_host ({host: proxy or "DIRECT"})
|
||||
* auto_detect (string or None): ``Acquire::http::Proxy-Auto-Detect``
|
||||
* settings (list of {file, line, value, editable}), in apt's reading order
|
||||
* https_proxy (string or None): the proxy https takes, the http one if unset
|
||||
* https_per_host ({host: proxy or "DIRECT"}), https_settings (as settings)
|
||||
* http_sources, https_sources (lists of the sources' base URIs)
|
||||
|
||||
:raises ValueError: if the host's output carried no intact report.
|
||||
"""
|
||||
if not self._is_apt_host():
|
||||
return None
|
||||
return parse_apt_proxy(self._run_apt_command(APT_PROXY_COMMAND))
|
||||
|
||||
def get_package_proxy(self) -> str | None:
|
||||
"""The proxy apt fetches through now; None for none, or a host without apt."""
|
||||
report = self.get_package_proxy_report()
|
||||
return report["proxy"] if report else None
|
||||
|
||||
def check_package_proxy(self, url: str, expect: str = "") -> ProxyFetchDict:
|
||||
"""Fetch *url* from the host, past any proxy apt is set to.
|
||||
|
||||
Whether the host reaches it, and whether the page contains *expect*.
|
||||
|
||||
:raises ValueError: if *url* is no http URL, or the host cannot fetch at all.
|
||||
:raises NotImplementedError: on a host without apt.
|
||||
"""
|
||||
if not self._is_apt_host():
|
||||
raise NotImplementedError("not an apt host")
|
||||
return parse_apt_fetch(
|
||||
self._run_apt_command(apt_fetch_command(url, direct=True, expect=expect))
|
||||
)
|
||||
|
||||
def check_https_source(self) -> HttpsSourceFetchDict | None:
|
||||
"""Fetch one https source's ``InRelease`` the way apt does, through its proxy.
|
||||
|
||||
None when no https source goes through a proxy: there are none, or https
|
||||
goes ``DIRECT``.
|
||||
|
||||
:raises ValueError: if the host's output carried no intact report.
|
||||
:raises NotImplementedError: on a host without apt.
|
||||
"""
|
||||
if not self._is_apt_host():
|
||||
raise NotImplementedError("not an apt host")
|
||||
report = self._report()
|
||||
found = _https_source(report, lambda proxy: proxy is not None)
|
||||
if found is None:
|
||||
return None
|
||||
target, proxy = found
|
||||
fetched = parse_apt_fetch(self._run_apt_command(apt_fetch_command(target)))
|
||||
return {**fetched, "url": target, "proxy": proxy}
|
||||
|
||||
def configure_package_proxy(self, url: str) -> FixResultDict:
|
||||
"""apt fetches through *url*, and is shown to; ``""`` removes :attr:`APT_PROXY_FILE`.
|
||||
|
||||
https goes through *url* too, unless *url* refuses it: then ``DIRECT``.
|
||||
"""
|
||||
if not url:
|
||||
return run_fix_steps([("configure", self._remove_package_proxy)])
|
||||
return run_fix_steps(
|
||||
[
|
||||
("configure", lambda: self._write_package_proxy(url)),
|
||||
("conflicts", lambda: self._comment_out_other_proxies(url)),
|
||||
("verify", lambda: self._verify_package_proxy(url)),
|
||||
("fetch", lambda: self._fetch_through_package_proxy(url)),
|
||||
("https", lambda: self._https_through_package_proxy(url)),
|
||||
]
|
||||
)
|
||||
|
||||
def _not_apt(self) -> FixStepOutcomeDict:
|
||||
return {"ok": False, "output": "not an apt host"}
|
||||
|
||||
def _remove_package_proxy(self) -> FixStepOutcomeDict:
|
||||
if not self._is_apt_host():
|
||||
return self._not_apt()
|
||||
return self._run_apt_root_command(
|
||||
f"rm -f {shlex.quote(self.APT_PROXY_FILE)}", "proxy removed"
|
||||
)
|
||||
|
||||
def _write_package_proxy(self, url: str, https_direct: bool = False) -> FixStepOutcomeDict:
|
||||
if not self._is_apt_host():
|
||||
return self._not_apt()
|
||||
if not PROXY_URL.match(url):
|
||||
raise ValueError(f"not a proxy URL: {url!r}")
|
||||
content = f'Acquire::http::Proxy "{url}";\n'
|
||||
if https_direct:
|
||||
content += 'Acquire::https::Proxy "DIRECT";\n'
|
||||
return self._write_apt_file(self.APT_PROXY_FILE, content)
|
||||
|
||||
def _report(self) -> AptProxyReportDict:
|
||||
# The steps after "configure" run only once it passed, on an apt host.
|
||||
return parse_apt_proxy(self._run_apt_command(APT_PROXY_COMMAND))
|
||||
|
||||
def _comment_out_other_proxies(self, url: str) -> FixStepOutcomeDict:
|
||||
others = [
|
||||
s
|
||||
for s in self._report()["settings"]
|
||||
if s["file"] != self.APT_PROXY_FILE and not same_proxy_url(s["value"], url)
|
||||
]
|
||||
for path, numbers in _editable_lines(others).items():
|
||||
outcome = self._run_apt_root_command(
|
||||
comment_out_command(path, numbers), f"commented out in {path}"
|
||||
)
|
||||
if not outcome["ok"]:
|
||||
return outcome
|
||||
return {"ok": True, "output": "; ".join(map(_described, others)) or "no other proxy set"}
|
||||
|
||||
def _verify_package_proxy(self, url: str) -> FixStepOutcomeDict:
|
||||
report = self._report()
|
||||
if same_proxy_url(report["proxy"], url):
|
||||
return {"ok": True, "output": f"apt fetches through {url}"}
|
||||
where = [
|
||||
f"{s['file']}:{s['line']}"
|
||||
for s in report["settings"]
|
||||
if s["value"] == report["proxy"] and s["file"] != self.APT_PROXY_FILE
|
||||
]
|
||||
return {
|
||||
"ok": False,
|
||||
"output": f"apt still fetches through {report['proxy'] or 'no proxy'}"
|
||||
+ (f", set in {', '.join(where)}" if where else ""),
|
||||
}
|
||||
|
||||
def _fetch_through_package_proxy(self, url: str) -> FixStepOutcomeDict:
|
||||
report = self._report()
|
||||
direct = {h.lower() for h, v in report["per_host"].items() if v.lower() == "direct"}
|
||||
source = next(
|
||||
(
|
||||
s
|
||||
for s in report["http_sources"]
|
||||
if "/dists/" in s and (urlsplit(s).hostname or "").lower() not in direct
|
||||
),
|
||||
None,
|
||||
)
|
||||
if source is None:
|
||||
return {"ok": True, "output": "no http source to fetch through the proxy; not tried"}
|
||||
target = source.rstrip("/") + "/InRelease"
|
||||
fetched = parse_apt_fetch(self._run_apt_command(apt_fetch_command(target)))
|
||||
if fetched["reachable"] and not fetched["error"]:
|
||||
return {"ok": True, "output": f"fetched {target} through {url}"}
|
||||
return {"ok": False, "output": f"{target}: {fetched['error']}"}
|
||||
|
||||
def _https_through_package_proxy(self, url: str) -> FixStepOutcomeDict:
|
||||
"""An https source through *url*; where *url* refuses it, https goes ``DIRECT``."""
|
||||
found = _https_source(self._report(), lambda proxy: same_proxy_url(proxy, url))
|
||||
if found is None:
|
||||
return {"ok": True, "output": f"no https source goes through {url}; not tried"}
|
||||
target = found[0]
|
||||
fetched = parse_apt_fetch(self._run_apt_command(apt_fetch_command(target)))
|
||||
# An HTTP error came from the mirror, through the tunnel: the cache let it by.
|
||||
if fetched["reachable"]:
|
||||
return {"ok": True, "output": f"fetched {target} through {url}"}
|
||||
refused = f"{url} refused https ({fetched['error']})"
|
||||
outcome = self._https_direct(url)
|
||||
if not outcome["ok"]:
|
||||
return {"ok": False, "output": f"{refused}; {outcome['output']}"}
|
||||
direct = parse_apt_fetch(self._run_apt_command(apt_fetch_command(target)))
|
||||
if not direct["reachable"]:
|
||||
return {"ok": False, "output": f"{refused}; direct too: {target}: {direct['error']}"}
|
||||
named = f"; {outcome['output']}" if outcome["output"] else ""
|
||||
return {
|
||||
"ok": True,
|
||||
"output": f"{refused}{named}; https sources go direct now: fetched {target}",
|
||||
}
|
||||
|
||||
def _https_direct(self, url: str) -> FixStepOutcomeDict:
|
||||
"""Write https ``DIRECT`` beside *url*, and comment out the later lines to *url*."""
|
||||
written = self._write_package_proxy(url, https_direct=True)
|
||||
if not written["ok"]:
|
||||
return written
|
||||
later = self._later_https_lines(self._report(), url)
|
||||
for path, numbers in _editable_lines(later).items():
|
||||
outcome = self._run_apt_root_command(
|
||||
comment_out_command(path, numbers, scheme="https"), f"commented out in {path}"
|
||||
)
|
||||
if not outcome["ok"]:
|
||||
return outcome
|
||||
report = self._report()
|
||||
if same_proxy_url(report["https_proxy"], url):
|
||||
still = [f"{s['file']}:{s['line']}" for s in self._later_https_lines(report, url)]
|
||||
where = f", set in {', '.join(still)}" if still else ""
|
||||
return {"ok": False, "output": f"https still goes through {url}{where}"}
|
||||
return {"ok": True, "output": "; ".join(map(_described, later))}
|
||||
|
||||
def _later_https_lines(self, report: AptProxyReportDict, url: str) -> list[AptProxySettingDict]:
|
||||
"""The lines outside :attr:`APT_PROXY_FILE` that send https to *url*."""
|
||||
return [
|
||||
s
|
||||
for s in report["https_settings"]
|
||||
if s["file"] != self.APT_PROXY_FILE and same_proxy_url(s["value"], url)
|
||||
]
|
||||
|
||||
|
||||
def _https_source(
|
||||
report: AptProxyReportDict, through: Callable[[str | None], bool]
|
||||
) -> tuple[str, str] | None:
|
||||
"""The ``InRelease`` of the first https source whose proxy *through* accepts, and that proxy."""
|
||||
for source in report["https_sources"]:
|
||||
if "/dists/" not in source:
|
||||
continue
|
||||
proxy = https_proxy_for(report, urlsplit(source).hostname or "")
|
||||
if through(proxy):
|
||||
return source.rstrip("/") + "/InRelease", proxy or ""
|
||||
return None
|
||||
|
||||
|
||||
__all__ = [
|
||||
"APT_PROXY_COMMAND",
|
||||
"AptProxyMixin",
|
||||
"AptProxyReportDict",
|
||||
"AptProxySettingDict",
|
||||
"HttpsSourceFetchDict",
|
||||
"ProxyFetchDict",
|
||||
"PROXY_URL",
|
||||
"apt_fetch_command",
|
||||
"comment_out_command",
|
||||
"https_proxy_for",
|
||||
"parse_apt_fetch",
|
||||
"parse_apt_proxy",
|
||||
"same_proxy_url",
|
||||
]
|
||||
@@ -0,0 +1,13 @@
|
||||
"""The ``age`` of an entry in NAPALM's neighbour tables when the device does not say.
|
||||
|
||||
NAPALM requires an ``age`` on every entry of ``get_arp_table`` and
|
||||
``get_ipv6_neighbors_table``: seconds since the device last confirmed that
|
||||
neighbour. Many devices do not report one. ``0.0`` means "confirmed just now"
|
||||
and must never stand in for "not reported": a caller ranking sightings by
|
||||
freshness takes it at its word (NetOrk/netork#1327). An expiry countdown is not
|
||||
an age either. The value is NAPALM's own; its IOS driver reports ``-1.0``.
|
||||
"""
|
||||
|
||||
from typing import Final
|
||||
|
||||
ARP_AGE_UNKNOWN: Final[float] = -1.0
|
||||
@@ -0,0 +1,238 @@
|
||||
"""
|
||||
Abstract base class for backup server drivers.
|
||||
|
||||
Usage::
|
||||
|
||||
from napalm_device_types import BackupServerDriver, StorageDriver
|
||||
|
||||
class ProxmoxBackupDriver(BackupServerDriver, StorageDriver):
|
||||
def get_backup_datastores(self):
|
||||
...
|
||||
"""
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from napalm_device_types.base import DeviceTypeDriver
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - for type checkers only
|
||||
from napalm_device_types.backups import (
|
||||
BackupDatastoreDict,
|
||||
BackupGroupDict,
|
||||
BackupJobKind,
|
||||
BackupRemoteDict,
|
||||
BackupServerJobDict,
|
||||
BackupSnapshotDict,
|
||||
BackupSnapshotRefDict,
|
||||
BackupTaskDict,
|
||||
BackupTaskStatusDict,
|
||||
)
|
||||
|
||||
|
||||
class BackupServerDriver(DeviceTypeDriver):
|
||||
"""
|
||||
Abstract intermediate driver for backup servers (e.g. Proxmox Backup
|
||||
Server): the device hypervisors and hosts write their backups to.
|
||||
|
||||
A backup server keeps its backups in datastores, one group per source (a
|
||||
VM, a container, a host) and one snapshot per backup run, and keeps them
|
||||
healthy with scheduled jobs: garbage collection, verify, prune, sync with
|
||||
other servers and tape. None of that is a storage pool, a volume or a
|
||||
replication job, so it is not modelled through ``StorageDriver``. A server
|
||||
that also manages its own disks and pools lists ``StorageDriver`` as a
|
||||
second role, after this one.
|
||||
|
||||
The actions start a task on the server and return at once with its id;
|
||||
:meth:`get_backup_task` follows it.
|
||||
"""
|
||||
|
||||
#: Stable key netOrk exposes as ``device_class``. The order in which a
|
||||
#: driver lists its role bases is the ranking; see
|
||||
#: :func:`napalm_device_types.roles.primary_role_of`.
|
||||
ROLE: str = "backup_server"
|
||||
TYPE_LABEL: str = "Backup Server"
|
||||
|
||||
if TYPE_CHECKING:
|
||||
# ------------------------------------------------------------------
|
||||
# Read
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def get_backup_datastores(self) -> list[BackupDatastoreDict]:
|
||||
"""
|
||||
Returns the server's datastores with their usage.
|
||||
|
||||
Example::
|
||||
|
||||
[
|
||||
{
|
||||
"name": "store1",
|
||||
"path": "/mnt/datastore/store1",
|
||||
"backend": "local",
|
||||
"total": 4000787030016,
|
||||
"used": 1529008357376,
|
||||
"available": 2471778672640,
|
||||
"estimated_full": 1798761600,
|
||||
"maintenance": None,
|
||||
"comment": None,
|
||||
},
|
||||
]
|
||||
|
||||
:raises Exception: when the server could not be read; never an
|
||||
empty list for a failed read.
|
||||
"""
|
||||
...
|
||||
|
||||
def get_backup_groups(self, datastore: str = "") -> list[BackupGroupDict]:
|
||||
"""
|
||||
Returns the backup groups of every namespace of *datastore*, or of
|
||||
every datastore when it is empty, each with its newest snapshot
|
||||
only. :meth:`get_backup_snapshots` lists all of a group's.
|
||||
|
||||
Example::
|
||||
|
||||
[
|
||||
{
|
||||
"datastore": "store1",
|
||||
"namespace": "",
|
||||
"type": "vm",
|
||||
"id": "101",
|
||||
"owner": "backup@pbs",
|
||||
"count": 14,
|
||||
"newest": {
|
||||
"time": 1793660400,
|
||||
"size": 34359738368,
|
||||
"verify": "ok",
|
||||
"protected": False,
|
||||
"comment": "web-01",
|
||||
},
|
||||
"oldest_time": 1792537200,
|
||||
"comment": None,
|
||||
},
|
||||
]
|
||||
|
||||
:raises Exception: when the server could not be read.
|
||||
"""
|
||||
...
|
||||
|
||||
def get_backup_snapshots(
|
||||
self, datastore: str, namespace: str, group_type: str, group_id: str
|
||||
) -> list[BackupSnapshotDict]:
|
||||
"""
|
||||
Returns every snapshot of one group, newest first.
|
||||
|
||||
:param namespace: ``""`` for the datastore's root.
|
||||
:raises ValueError: when the datastore or the group does not exist.
|
||||
"""
|
||||
...
|
||||
|
||||
def get_backup_jobs(self) -> list[BackupServerJobDict]:
|
||||
"""
|
||||
Returns the server's scheduled jobs of every kind, garbage
|
||||
collection included (one per datastore, ``id`` = the datastore).
|
||||
|
||||
Example::
|
||||
|
||||
[
|
||||
{
|
||||
"id": "store1",
|
||||
"kind": "gc",
|
||||
"datastore": "store1",
|
||||
"namespace": None,
|
||||
"schedule": "daily",
|
||||
"enabled": True,
|
||||
"last_run": 1793664000,
|
||||
"last_state": "ok",
|
||||
"last_error": None,
|
||||
"next_run": 1793750400,
|
||||
"remote": None,
|
||||
"direction": None,
|
||||
"comment": None,
|
||||
},
|
||||
{
|
||||
"id": "offsite",
|
||||
"kind": "sync",
|
||||
"datastore": "store1",
|
||||
"namespace": "",
|
||||
"schedule": "sat 03:00",
|
||||
"enabled": True,
|
||||
"last_run": 1793502000,
|
||||
"last_state": "error",
|
||||
"last_error": "connection refused",
|
||||
"next_run": 1794106800,
|
||||
"remote": "pbs-offsite",
|
||||
"direction": "push",
|
||||
"comment": None,
|
||||
},
|
||||
]
|
||||
|
||||
:raises Exception: when the server could not be read.
|
||||
"""
|
||||
...
|
||||
|
||||
def get_backup_remotes(self) -> list[BackupRemoteDict]:
|
||||
"""
|
||||
Returns the other backup servers this one syncs with, without their
|
||||
secrets.
|
||||
|
||||
:raises Exception: when the server could not be read.
|
||||
"""
|
||||
...
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Actions
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def start_garbage_collection(self, datastore: str) -> BackupTaskDict:
|
||||
"""
|
||||
Starts garbage collection on *datastore*.
|
||||
|
||||
:raises ValueError: when the datastore does not exist.
|
||||
:raises RuntimeError: when the server refuses (e.g. one is running).
|
||||
"""
|
||||
...
|
||||
|
||||
def run_backup_job(self, kind: BackupJobKind, job_id: str) -> BackupTaskDict:
|
||||
"""
|
||||
Runs a scheduled job now, outside its schedule.
|
||||
|
||||
:param kind: a :data:`~napalm_device_types.backups.BackupJobKind`;
|
||||
``"gc"`` runs the datastore named *job_id*'s garbage collection.
|
||||
:raises ValueError: when there is no such job.
|
||||
:raises RuntimeError: when the server refuses.
|
||||
"""
|
||||
...
|
||||
|
||||
def verify_backup_snapshot(self, snapshot: BackupSnapshotRefDict) -> BackupTaskDict:
|
||||
"""
|
||||
Verifies one snapshot now.
|
||||
|
||||
:raises ValueError: when the snapshot does not exist.
|
||||
"""
|
||||
...
|
||||
|
||||
def set_backup_snapshot_protected(
|
||||
self, snapshot: BackupSnapshotRefDict, protected: bool
|
||||
) -> None:
|
||||
"""
|
||||
Protects a snapshot from pruning and deletion, or lifts that.
|
||||
|
||||
:raises ValueError: when the snapshot does not exist.
|
||||
"""
|
||||
...
|
||||
|
||||
def delete_backup_snapshot(self, snapshot: BackupSnapshotRefDict) -> None:
|
||||
"""
|
||||
Deletes one snapshot. The data is gone once garbage collection has
|
||||
run; there is no undo.
|
||||
|
||||
:raises ValueError: when the snapshot does not exist.
|
||||
:raises RuntimeError: when it is protected.
|
||||
"""
|
||||
...
|
||||
|
||||
def get_backup_task(self, task: str) -> BackupTaskStatusDict:
|
||||
"""
|
||||
Returns the state and the log so far of a task an action started.
|
||||
|
||||
:raises ValueError: when the server does not know the task.
|
||||
"""
|
||||
...
|
||||
@@ -0,0 +1,119 @@
|
||||
"""The shapes of a backup server (``BackupServerDriver``).
|
||||
|
||||
Apart from ``models.py``, which is over the file size limit. A value the
|
||||
platform does not report is None; it is never guessed.
|
||||
"""
|
||||
|
||||
from typing import Literal, TypedDict, get_args
|
||||
|
||||
#: What a backup server's jobs do. Garbage collection is one of them: it has a
|
||||
#: schedule and a last run like the others.
|
||||
BackupJobKind = Literal["gc", "verify", "prune", "sync", "tape"]
|
||||
BACKUP_JOB_KINDS: tuple[str, ...] = get_args(BackupJobKind)
|
||||
|
||||
#: How a job's or a task's run ended; "running" while it has not.
|
||||
BackupRunState = Literal["ok", "warning", "error", "running"]
|
||||
|
||||
#: The outcome of the last verification of a snapshot.
|
||||
BackupVerifyState = Literal["ok", "failed"]
|
||||
|
||||
|
||||
class BackupDatastoreDict(TypedDict):
|
||||
"""A datastore: where a backup server keeps its backups."""
|
||||
|
||||
name: str
|
||||
path: str | None # local path; for an object-store backend the local cache, if any
|
||||
backend: str # "local", "s3", "removable", ... as the platform names it
|
||||
total: int | None # bytes
|
||||
used: int | None # bytes
|
||||
available: int | None # bytes
|
||||
estimated_full: int | None # unix seconds the platform expects it to be full
|
||||
maintenance: str | None # "read-only", "offline", ...; None when in service
|
||||
comment: str | None
|
||||
|
||||
|
||||
class BackupSnapshotDict(TypedDict):
|
||||
"""One backup of a group: the state of its source at one time."""
|
||||
|
||||
time: int # unix seconds the backup was taken
|
||||
size: int | None # bytes
|
||||
verify: BackupVerifyState | None # None: never verified
|
||||
protected: bool # exempt from pruning and deletion
|
||||
comment: str | None
|
||||
|
||||
|
||||
class BackupGroupDict(TypedDict):
|
||||
"""The backups of one source (a VM, a container, a host) in one namespace."""
|
||||
|
||||
datastore: str
|
||||
namespace: str # "" for the datastore's root
|
||||
type: str # "vm", "ct", "host"
|
||||
id: str # the guest's id ("101") or the host's name
|
||||
owner: str | None # the account that writes the group's backups
|
||||
count: int # snapshots in the group
|
||||
newest: BackupSnapshotDict | None # None for a group without a snapshot
|
||||
oldest_time: int | None # unix seconds of the oldest snapshot
|
||||
comment: str | None
|
||||
|
||||
|
||||
class BackupSnapshotRefDict(TypedDict):
|
||||
"""Names one snapshot, for an action on it."""
|
||||
|
||||
datastore: str
|
||||
namespace: str # "" for the datastore's root
|
||||
group_type: str # BackupGroupDict.type
|
||||
group_id: str # BackupGroupDict.id
|
||||
time: int # BackupSnapshotDict.time
|
||||
|
||||
|
||||
class BackupServerJobDict(TypedDict):
|
||||
"""A scheduled job of a backup server."""
|
||||
|
||||
id: str # for "gc", the datastore's name
|
||||
kind: BackupJobKind
|
||||
datastore: str | None
|
||||
namespace: str | None
|
||||
schedule: str | None # as the platform writes it: "daily", "sat 02:00", "*:0/30"
|
||||
enabled: bool
|
||||
last_run: int | None # unix seconds the last run ended (or started, while running)
|
||||
last_state: BackupRunState | None # None: never ran
|
||||
last_error: str | None
|
||||
next_run: int | None # unix seconds
|
||||
remote: str | None # a sync job's BackupRemoteDict.name
|
||||
direction: Literal["pull", "push"] | None # a sync job's
|
||||
comment: str | None
|
||||
|
||||
|
||||
class BackupRemoteDict(TypedDict):
|
||||
"""Another backup server this one syncs with. Never its secret."""
|
||||
|
||||
name: str
|
||||
host: str
|
||||
port: int | None
|
||||
fingerprint: str | None # of the remote's TLS certificate
|
||||
auth_id: str | None # the account the sync logs in as
|
||||
comment: str | None
|
||||
|
||||
|
||||
class BackupTaskDict(TypedDict):
|
||||
"""A task an action started on the server, to follow with ``get_backup_task``."""
|
||||
|
||||
task: str # the platform's task id
|
||||
|
||||
|
||||
class BackupTaskStatusDict(TypedDict):
|
||||
task: str
|
||||
running: bool
|
||||
state: BackupRunState | None # "running" while it runs
|
||||
error: str | None
|
||||
log: list[str] # the task's log lines so far
|
||||
|
||||
|
||||
class BackupServerRefDict(TypedDict):
|
||||
"""Which backup server and datastore a hypervisor's backup storage writes to."""
|
||||
|
||||
server: str # host name or address, as the hypervisor has it configured
|
||||
port: int | None
|
||||
datastore: str
|
||||
namespace: str | None
|
||||
fingerprint: str | None # of the server's TLS certificate, as the hypervisor pins it
|
||||
@@ -8,11 +8,14 @@ live connection.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Iterable
|
||||
from typing import NamedTuple
|
||||
|
||||
from napalm.base import NetworkDriver
|
||||
|
||||
from napalm_device_types.channel import CommandChannelMixin
|
||||
from napalm_device_types.host_reboot import HostRebootMixin
|
||||
from napalm_device_types.host_shutdown import HostShutdownMixin
|
||||
from napalm_device_types.ping_sweep import PingSweepMixin
|
||||
|
||||
|
||||
@@ -23,12 +26,36 @@ class FingerprintRule(NamedTuple):
|
||||
weight — evidence added to the driver's score on match
|
||||
mandatory — pattern absent → driver immediately disqualified
|
||||
negative — pattern present → weight subtracted instead of added
|
||||
group — rules sharing a group are alternatives that never match
|
||||
together (one vendor's landing page *or* another's); only the
|
||||
strongest of them counts towards the most a driver can score
|
||||
"""
|
||||
|
||||
pattern: str
|
||||
weight: float = 1.0
|
||||
mandatory: bool = False
|
||||
negative: bool = False
|
||||
group: str | None = None
|
||||
|
||||
|
||||
def rules_max_weight(rules: Iterable[FingerprintRule]) -> float:
|
||||
"""The most evidence *rules* can add together.
|
||||
|
||||
Every positive rule counts, except that a group of alternatives counts once,
|
||||
with its strongest rule. Discovery divides a driver's score by its maximum to
|
||||
compare drivers; summing alternatives would make a driver that recognises
|
||||
three vendors' pages lose to one that recognises none.
|
||||
"""
|
||||
total = 0.0
|
||||
group_best: dict[str, float] = {}
|
||||
for rule in rules:
|
||||
if rule.negative or rule.weight <= 0:
|
||||
continue
|
||||
if rule.group is None:
|
||||
total += rule.weight
|
||||
else:
|
||||
group_best[rule.group] = max(group_best.get(rule.group, 0.0), rule.weight)
|
||||
return total + sum(group_best.values())
|
||||
|
||||
|
||||
class PortSpec(NamedTuple):
|
||||
@@ -48,7 +75,13 @@ class PortSpec(NamedTuple):
|
||||
mandatory: bool = False
|
||||
|
||||
|
||||
class DeviceTypeDriver(PingSweepMixin, HostRebootMixin, NetworkDriver):
|
||||
class DeviceTypeDriver(
|
||||
PingSweepMixin,
|
||||
HostRebootMixin,
|
||||
HostShutdownMixin,
|
||||
CommandChannelMixin,
|
||||
NetworkDriver, # type: ignore[misc]
|
||||
):
|
||||
"""Common base for all netOrk device-type drivers.
|
||||
|
||||
Sits between napalm.base.NetworkDriver and the type-specific abstract
|
||||
|
||||
@@ -0,0 +1,196 @@
|
||||
"""The command channel: a public way to run a command on a host, or open a stream.
|
||||
|
||||
A driver already reaches its host, over SSH, a REST API or WinRM. What it did
|
||||
not offer was a *public* way for the layer above to use that reach: netOrk sent
|
||||
shell strings through napalm-linux's private ``_send``, an interactive PTY that
|
||||
merges stdout and stderr and has no exit code.
|
||||
|
||||
:class:`CommandChannelMixin` declares two methods, and a driver that can
|
||||
implements them:
|
||||
|
||||
* :meth:`run_command` runs a command and returns stdout, stderr and the exit code.
|
||||
* :meth:`open_stream` starts a command and returns a :class:`ByteStream` to its
|
||||
stdin and stdout. That is how netOrk speaks the Docker Engine API, over
|
||||
``docker system dial-stdio`` (see :mod:`napalm_device_types.container_engine`).
|
||||
|
||||
Like the role bases, the mixin only declares (under ``TYPE_CHECKING``), so
|
||||
``hasattr(driver, "run_command")`` stays a truthful answer.
|
||||
|
||||
:func:`run_on_transport` and :func:`open_stream_on_transport` are the SSH
|
||||
implementation over a paramiko ``Transport``, for any SSH driver: an exec
|
||||
channel, so no PTY, separate stderr and a real exit status. How a command gains
|
||||
root (``sudo -S`` with the password on stdin, ``sudo -n``, or nothing as root)
|
||||
stays with the driver.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import time
|
||||
from typing import TYPE_CHECKING, Any, NamedTuple, Protocol, cast
|
||||
|
||||
#: How much of a stream's stderr is kept. Enough to tell "permission denied"
|
||||
#: from "no such command"; a stream that writes megabytes there keeps its tail.
|
||||
STDERR_TAIL = 64 * 1024
|
||||
|
||||
_READ = 32 * 1024
|
||||
_POLL = 0.01
|
||||
|
||||
|
||||
class CommandResult(NamedTuple):
|
||||
"""What a command printed, and how it ended."""
|
||||
|
||||
stdout: str
|
||||
stderr: str
|
||||
exit_code: int
|
||||
|
||||
|
||||
class ByteStream(Protocol):
|
||||
"""A running command's stdin and stdout, as bytes.
|
||||
|
||||
``read`` returns ``b""`` at end of stream and raises :class:`TimeoutError`
|
||||
when nothing arrives in time. ``write`` never closes anything:
|
||||
``dial-stdio`` hands the daemon a half-close as "client gone", and the
|
||||
daemon then answers an unfinished request with HTTP 499 (netork#771). Only
|
||||
``close_write`` closes the writing side.
|
||||
"""
|
||||
|
||||
def read(self, max_bytes: int, timeout: float | None = None) -> bytes: ...
|
||||
|
||||
def write(self, data: bytes) -> None: ...
|
||||
|
||||
def close_write(self) -> None: ...
|
||||
|
||||
def close(self) -> None: ...
|
||||
|
||||
@property
|
||||
def exit_status(self) -> int | None: ...
|
||||
|
||||
@property
|
||||
def stderr(self) -> str: ...
|
||||
|
||||
|
||||
class CommandChannelMixin:
|
||||
"""Declares the channel; a driver that can reach a shell on its host implements it."""
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def run_command(
|
||||
self,
|
||||
command: str,
|
||||
*,
|
||||
privileged: bool = False,
|
||||
timeout: float = 60,
|
||||
stdin: bytes | None = None,
|
||||
) -> CommandResult:
|
||||
"""Run *command* with the host's shell; *stdin*, if given, is written first."""
|
||||
...
|
||||
|
||||
def open_stream(self, command: str, *, privileged: bool = False) -> ByteStream:
|
||||
"""Start *command* and return a stream to its stdin and stdout."""
|
||||
...
|
||||
|
||||
|
||||
class ParamikoExecStream:
|
||||
"""A :class:`ByteStream` over a paramiko exec channel.
|
||||
|
||||
stderr is drained on every read: stdout and stderr share one window, and an
|
||||
unread stderr would stall the stream.
|
||||
"""
|
||||
|
||||
def __init__(self, channel: Any, *, stderr_limit: int = STDERR_TAIL) -> None:
|
||||
self._channel = channel
|
||||
self._stderr = b""
|
||||
self._limit = stderr_limit
|
||||
|
||||
def _drain_stderr(self) -> None:
|
||||
while self._channel.recv_stderr_ready():
|
||||
self._stderr = (self._stderr + self._channel.recv_stderr(_READ))[-self._limit :]
|
||||
|
||||
def read(self, max_bytes: int, timeout: float | None = None) -> bytes:
|
||||
self._channel.settimeout(timeout)
|
||||
try:
|
||||
return cast(bytes, self._channel.recv(max_bytes))
|
||||
except TimeoutError:
|
||||
raise TimeoutError(f"no data within {timeout}s") from None
|
||||
finally:
|
||||
self._drain_stderr()
|
||||
|
||||
def write(self, data: bytes) -> None:
|
||||
self._channel.sendall(data)
|
||||
|
||||
def close_write(self) -> None:
|
||||
self._channel.shutdown_write()
|
||||
|
||||
def close(self) -> None:
|
||||
self._channel.close()
|
||||
|
||||
@property
|
||||
def exit_status(self) -> int | None:
|
||||
if not self._channel.exit_status_ready():
|
||||
return None
|
||||
return cast(int, self._channel.recv_exit_status())
|
||||
|
||||
@property
|
||||
def stderr(self) -> str:
|
||||
self._drain_stderr()
|
||||
return self._stderr.decode("utf-8", "replace")
|
||||
|
||||
|
||||
def _collect(channel: Any, command: str, timeout: float) -> tuple[bytes, bytes]:
|
||||
out, err = b"", b""
|
||||
deadline = time.monotonic() + timeout
|
||||
while True:
|
||||
if channel.recv_ready():
|
||||
out += channel.recv(_READ)
|
||||
elif channel.recv_stderr_ready():
|
||||
err += channel.recv_stderr(_READ)
|
||||
elif channel.exit_status_ready():
|
||||
# sshd may send the exit status before the last output: read both to EOF.
|
||||
try:
|
||||
while chunk := channel.recv(_READ):
|
||||
out += chunk
|
||||
while chunk := channel.recv_stderr(_READ):
|
||||
err += chunk
|
||||
except TimeoutError:
|
||||
raise TimeoutError(f"{command!r} did not finish within {timeout}s") from None
|
||||
return out, err
|
||||
elif time.monotonic() > deadline:
|
||||
raise TimeoutError(f"{command!r} did not finish within {timeout}s")
|
||||
else:
|
||||
time.sleep(_POLL)
|
||||
|
||||
|
||||
def run_on_transport(
|
||||
transport: Any, command: str, *, stdin: bytes | None = None, timeout: float = 60
|
||||
) -> CommandResult:
|
||||
"""Run *command* on an exec channel of the paramiko *transport*."""
|
||||
channel = transport.open_session()
|
||||
try:
|
||||
channel.settimeout(timeout)
|
||||
channel.exec_command(command)
|
||||
if stdin is not None:
|
||||
channel.sendall(stdin)
|
||||
channel.shutdown_write()
|
||||
out, err = _collect(channel, command, timeout)
|
||||
return CommandResult(
|
||||
out.decode("utf-8", "replace"),
|
||||
err.decode("utf-8", "replace"),
|
||||
channel.recv_exit_status(),
|
||||
)
|
||||
finally:
|
||||
channel.close()
|
||||
|
||||
|
||||
def open_stream_on_transport(
|
||||
transport: Any, command: str, *, stdin_prefix: bytes | None = None
|
||||
) -> ParamikoExecStream:
|
||||
"""Start *command* on an exec channel and return its stream.
|
||||
|
||||
*stdin_prefix* is written before anything else, which is how a sudo password
|
||||
reaches ``sudo -S`` ahead of the stream's own bytes.
|
||||
"""
|
||||
channel = transport.open_session()
|
||||
channel.exec_command(command)
|
||||
if stdin_prefix:
|
||||
channel.sendall(stdin_prefix)
|
||||
return ParamikoExecStream(channel)
|
||||
@@ -0,0 +1,104 @@
|
||||
"""The shapes of a hypervisor's clusters (``HypervisorDriver.get_clusters()``).
|
||||
|
||||
Apart from ``models.py``, which is over the file size limit.
|
||||
"""
|
||||
|
||||
from typing import NotRequired, TypedDict
|
||||
|
||||
from napalm_device_types.backups import BackupServerRefDict
|
||||
|
||||
|
||||
class ClusterNodeDict(TypedDict):
|
||||
"""One host of a cluster, as its cluster sees it. A value the platform does not
|
||||
report is None; it is never guessed."""
|
||||
|
||||
name: str # the node's name in its cluster, as get_vms() reports it in ``node``
|
||||
status: str # "online", "offline" or "unknown"
|
||||
local: bool # the node this connection talks to; False on a manager (vCenter)
|
||||
maintenance: bool | None # in maintenance mode
|
||||
cpu_threads: int | None # logical CPUs
|
||||
cpu_cores: int | None # physical cores
|
||||
cpu_model: str | None
|
||||
cpu_flags: list[str] | None # cpuinfo flags; None where the platform does not say
|
||||
cpu_usage: float | None # 0.0–1.0
|
||||
memory_total: int | None # bytes
|
||||
memory_used: int | None # bytes
|
||||
votes: NotRequired[int | None] # quorum votes of this node; None standalone/unreadable
|
||||
|
||||
|
||||
class ClusterStorageDict(TypedDict):
|
||||
"""A storage of a cluster and the nodes that see it."""
|
||||
|
||||
name: str # usable as create_vm_from_cloud_init(storage=...) on each of ``nodes``
|
||||
type: str # backend type: "dir", "lvmthin", "zfspool", "nfs", "rbd", "vmfs", ...
|
||||
shared: bool # the same storage on every node in ``nodes``, so a VM on it can move
|
||||
content: list[str] # what it holds: "images", "iso", "snippets", ...
|
||||
nodes: list[str] # the nodes it is available on
|
||||
total: int | None # bytes; None where the platform does not report it
|
||||
used: int | None # bytes
|
||||
# A backup storage that writes to a backup server: which one. Left out for every
|
||||
# other storage; None where the platform does not say.
|
||||
backup_server: NotRequired[BackupServerRefDict | None]
|
||||
|
||||
|
||||
class ClusterNetworkDict(TypedDict):
|
||||
"""A network a VM's NIC may attach to, and the nodes that carry it."""
|
||||
|
||||
name: str # usable as NICConfigDict.bridge on each of ``nodes``
|
||||
kind: str # "bridge", "vnet" or "portgroup", as in NetworkTargetDict
|
||||
vlan_aware: bool # True if a NICConfigDict.vlan_tag may be set on top of it
|
||||
fixed_vlan_tag: NotRequired[int | None] # vnet/portgroup: the VLAN baked into it
|
||||
nodes: list[str]
|
||||
|
||||
|
||||
class HAResourceDict(TypedDict):
|
||||
"""A guest the cluster's HA manager looks after."""
|
||||
|
||||
vmid: int
|
||||
kind: str # "qemu" or "lxc"
|
||||
requested: str # state HA is asked to keep: "started", "stopped", "disabled", "ignored"
|
||||
status: str | None # what the HA manager reports now: "started", "fence", "migrate", ...
|
||||
node: str | None # the node the HA manager places it on now
|
||||
group: NotRequired[str | None] # HA group (Proxmox VE 8)
|
||||
rules: list[str] # ids of the HA rules that name it (Proxmox VE 9); [] where there are none
|
||||
|
||||
|
||||
class BackupJobDict(TypedDict):
|
||||
"""A scheduled backup job of the cluster."""
|
||||
|
||||
id: str
|
||||
enabled: bool
|
||||
schedule: str | None # as the platform writes it: "sat 02:00", "daily", "mon..fri 21:00"
|
||||
storage: str | None
|
||||
mode: str | None # "snapshot", "suspend", "stop"
|
||||
all: bool # every guest, minus ``exclude``
|
||||
vmids: list[int] # the guests it names; a pool job: the pool's members now; [] with all
|
||||
exclude: list[int]
|
||||
pool: str | None
|
||||
node: str | None # the one node it runs on; None for every node
|
||||
|
||||
|
||||
class BackupDict(TypedDict):
|
||||
"""The backups of one guest on one storage."""
|
||||
|
||||
vmid: int
|
||||
storage: str
|
||||
time: int # unix seconds of the newest backup of this guest on this storage
|
||||
size: int | None # bytes of that newest backup
|
||||
count: int # backups of this guest on this storage
|
||||
|
||||
|
||||
class ClusterDict(TypedDict):
|
||||
"""A cluster the connection can see. A standalone host is a cluster of one,
|
||||
with no name."""
|
||||
|
||||
id: str | None # stable across connections where the platform has one; None otherwise
|
||||
name: str | None # the cluster's name; None for a standalone host
|
||||
quorate: bool | None # None where the platform has no quorum (standalone, vSphere)
|
||||
nodes: list[ClusterNodeDict]
|
||||
storages: list[ClusterStorageDict]
|
||||
networks: list[ClusterNetworkDict]
|
||||
# None: could not be read; []: read, nothing there.
|
||||
ha_resources: NotRequired[list[HAResourceDict] | None]
|
||||
backup_jobs: NotRequired[list[BackupJobDict] | None]
|
||||
backups: NotRequired[list[BackupDict] | None]
|
||||
@@ -1,8 +1,7 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
from __future__ import annotations
|
||||
|
||||
import difflib
|
||||
from typing import ClassVar, TYPE_CHECKING
|
||||
from typing import TYPE_CHECKING, ClassVar
|
||||
|
||||
from napalm.base.exceptions import (
|
||||
CommandErrorException,
|
||||
@@ -41,15 +40,11 @@ class ConfigLifecycleMixin:
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
||||
def _get_running_config(self) -> str:
|
||||
...
|
||||
def _get_running_config(self) -> str: ...
|
||||
|
||||
def commit_config(self, message: str = "", revert_in: int | None = None) -> None:
|
||||
...
|
||||
def commit_config(self, message: str = "", revert_in: int | None = None) -> None: ...
|
||||
|
||||
def load_merge_candidate(
|
||||
self, filename: str | None = None, config: str | None = None
|
||||
) -> None:
|
||||
def load_merge_candidate(self, filename: str | None = None, config: str | None = None) -> None:
|
||||
if filename is not None:
|
||||
try:
|
||||
with open(filename) as fh:
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
"""Configuration sync between the two hosts of an HA pair.
|
||||
|
||||
A host that pushes its configuration to a peer (OPNsense's XMLRPC sync, say)
|
||||
makes the peer's configuration of the synced services a copy of its own: a
|
||||
change written on the receiving side lasts only until the next push. Who pushes
|
||||
what is therefore a fact a caller needs before it writes to either host.
|
||||
|
||||
Declared under ``if TYPE_CHECKING``: a contract, not a placeholder. A driver
|
||||
mixes it in; no role promises it.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from napalm_device_types.models import ConfigSyncDict
|
||||
|
||||
|
||||
class ConfigSyncMixin:
|
||||
"""Where a host pushes its configuration, and for which of its services."""
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
||||
def get_config_sync(self) -> ConfigSyncDict:
|
||||
"""
|
||||
Returns the peer this host pushes its configuration to (None when it
|
||||
pushes to no one) and the services whose configuration it pushes, by
|
||||
the name ``get_services`` reports. Sync items the driver has no
|
||||
service name for are listed under ``unmapped`` as the host names them.
|
||||
|
||||
:raises Exception: When the setting could not be read -- never
|
||||
``peer=None`` for "could not ask".
|
||||
"""
|
||||
...
|
||||
@@ -0,0 +1,153 @@
|
||||
"""Access to a host's container engine, and nothing more.
|
||||
|
||||
This package abstracts the connection, the driver builds it, and netOrk does
|
||||
the talking (NetOrk/netork#765). So there is no container model here and no
|
||||
parsing:
|
||||
|
||||
* :meth:`ContainerEngineMixin.container_engines` says which engines the host
|
||||
offers, and which API each speaks.
|
||||
* :meth:`ContainerEngineMixin.open_container_engine` returns a
|
||||
:class:`ContainerEngineConnection`. It opens a stream to the engine's API
|
||||
(``docker system dial-stdio``), and it runs the engine's CLI with arguments
|
||||
the caller chooses. The CLI is there for what the API cannot do: compose,
|
||||
pulls that need the host user's registry login, and private registry digests.
|
||||
|
||||
The driver decides *how* the engine is reached. The hook is
|
||||
:meth:`ContainerEngineMixin._container_engine_binary`. QNAP returns its
|
||||
Container Station path, so callers never see where the binary lives. The
|
||||
command runs through the driver's own :class:`~napalm_device_types.channel.CommandChannelMixin`,
|
||||
whose privilege handling applies unchanged; Docker normally needs none, because
|
||||
the login user is in the ``docker`` group.
|
||||
|
||||
Mixed in by a driver, not by a role base: a Windows host is an
|
||||
:class:`~napalm_device_types.os.OSDriver` too, and without a shell channel it
|
||||
has no ``dial-stdio`` to offer. ``hasattr(driver, "open_container_engine")``
|
||||
stays truthful.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shlex
|
||||
from collections.abc import Sequence
|
||||
from typing import TYPE_CHECKING, Any, cast
|
||||
|
||||
from napalm_device_types.channel import ByteStream, CommandResult
|
||||
from napalm_device_types.models import ContainerEngineDict, FixResultDict
|
||||
|
||||
#: Engines this package knows how to reach, and the API each speaks.
|
||||
ENGINES: dict[str, str] = {"docker": "docker-engine"}
|
||||
|
||||
_PROBE_TIMEOUT = 15
|
||||
_CLI_TIMEOUT = 120
|
||||
|
||||
|
||||
class ContainerEngineUnavailable(NotImplementedError):
|
||||
"""The host offers no such engine, or this package does not know how to reach it."""
|
||||
|
||||
|
||||
class ContainerEngineConnection:
|
||||
"""The way to one container engine on one host.
|
||||
|
||||
Built by :meth:`ContainerEngineMixin.open_container_engine`. Callers pass
|
||||
arguments; binary, quoting and the channel are the driver's.
|
||||
"""
|
||||
|
||||
def __init__(self, driver: Any, engine: str, binary: str) -> None:
|
||||
self._driver = driver
|
||||
self._binary = binary
|
||||
self.engine = engine
|
||||
|
||||
def _command(self, args: Sequence[str]) -> str:
|
||||
return shlex.join([self._binary, *args])
|
||||
|
||||
def open_api(self) -> ByteStream:
|
||||
"""A stream to the engine's API: HTTP/1.1 over ``system dial-stdio``."""
|
||||
return cast(ByteStream, self._driver.open_stream(self._command(["system", "dial-stdio"])))
|
||||
|
||||
def run_cli(
|
||||
self,
|
||||
args: Sequence[str],
|
||||
*,
|
||||
stdin: bytes | None = None,
|
||||
timeout: float = _CLI_TIMEOUT,
|
||||
privileged: bool = False,
|
||||
) -> CommandResult:
|
||||
"""Run the engine's CLI with *args* and wait for it.
|
||||
|
||||
*privileged* runs it as root, the way the driver gains root for any
|
||||
command: for a call the engine refused to the login user.
|
||||
"""
|
||||
return cast(
|
||||
CommandResult,
|
||||
self._driver.run_command(
|
||||
self._command(args), privileged=privileged, timeout=timeout, stdin=stdin
|
||||
),
|
||||
)
|
||||
|
||||
def stream_cli(
|
||||
self, args: Sequence[str], *, merge_stderr: bool = False, privileged: bool = False
|
||||
) -> ByteStream:
|
||||
"""Start the engine's CLI with *args* and stream its output.
|
||||
|
||||
*merge_stderr* folds stderr into the stream, for tools that report
|
||||
progress there (``compose up``); *privileged* as for :meth:`run_cli`.
|
||||
"""
|
||||
command = self._command(args)
|
||||
return cast(
|
||||
ByteStream,
|
||||
self._driver.open_stream(
|
||||
f"{command} 2>&1" if merge_stderr else command, privileged=privileged
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
class ContainerEngineMixin:
|
||||
"""Adds container engine access to a driver that implements the channel."""
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def run_command(
|
||||
self,
|
||||
command: str,
|
||||
*,
|
||||
privileged: bool = False,
|
||||
timeout: float = 60,
|
||||
stdin: bytes | None = None,
|
||||
) -> CommandResult: ...
|
||||
|
||||
def open_stream(self, command: str, *, privileged: bool = False) -> ByteStream: ...
|
||||
|
||||
def grant_container_engine_access(self, user: str) -> FixResultDict:
|
||||
"""Let *user* reach the container engine without privilege (the
|
||||
``docker`` group on most hosts). A fix template: see
|
||||
:mod:`napalm_device_types.fixes`. Declared only; a driver implements it."""
|
||||
...
|
||||
|
||||
def _container_engine_binary(self, engine: str) -> str:
|
||||
"""The host command that is *engine*'s CLI. Override where it is not on PATH."""
|
||||
return engine
|
||||
|
||||
def container_engines(self) -> list[ContainerEngineDict]:
|
||||
"""
|
||||
Returns the container engines the host offers:
|
||||
|
||||
* engine (string) - e.g. "docker"
|
||||
* api (string) - what its API is, e.g. "docker-engine"
|
||||
|
||||
An engine is listed when its CLI exists on the host. Whether the login
|
||||
user may use it is decided by the first API call, which says
|
||||
"permission denied" in the stream's stderr.
|
||||
"""
|
||||
found: list[ContainerEngineDict] = []
|
||||
for engine, api in ENGINES.items():
|
||||
probe = shlex.join(["command", "-v", self._container_engine_binary(engine)])
|
||||
result = self.run_command(probe, timeout=_PROBE_TIMEOUT)
|
||||
if result.exit_code == 0 and result.stdout.strip():
|
||||
found.append({"engine": engine, "api": api})
|
||||
return found
|
||||
|
||||
def open_container_engine(self, engine: str) -> ContainerEngineConnection:
|
||||
"""The connection to *engine*; :class:`ContainerEngineUnavailable` if unknown."""
|
||||
if engine not in ENGINES:
|
||||
raise ContainerEngineUnavailable(f"no way to reach container engine {engine!r}")
|
||||
return ContainerEngineConnection(self, engine, self._container_engine_binary(engine))
|
||||
@@ -0,0 +1,203 @@
|
||||
"""Cron expressions: what they mean, when they fire next, and as systemd ``OnCalendar``.
|
||||
|
||||
A scheduled job is given as a cron expression on every host, and a host with
|
||||
systemd runs it as a timer. So the expression has to mean the same in three
|
||||
places -- on the host's cron, in a preview of the next runs, and in the
|
||||
``OnCalendar`` lines a timer is written with -- and the meaning lives here once.
|
||||
|
||||
**What cron means is Vixie cron's.** Five fields (minute, hour, day of month,
|
||||
month, day of week), each ``*``, a value, a range ``a-b``, a step ``*/n`` or
|
||||
``a-b/n``, or a list of those; month and weekday names in any case; ``7`` is
|
||||
Sunday as well as ``0``; the ``@hourly`` … ``@yearly`` aliases. When both day
|
||||
fields are restricted -- neither starts with ``*`` -- a day matches if *either*
|
||||
does; otherwise both must (``*/2`` counts as starting with ``*``).
|
||||
|
||||
**``OnCalendar`` by expansion.** Every field is written as its values, so a range
|
||||
with a step needs no systemd syntax of its own, and a timer fires on exactly the
|
||||
minutes cron would. The *either* rule is two ``OnCalendar`` lines, which systemd
|
||||
ORs. ``@reboot`` has no calendar; a timer takes it as ``OnBootSec``.
|
||||
|
||||
**Next runs are in the host's time zone.** A local time the clock skips when
|
||||
summer time starts is left out; cron implementations disagree on it, and a
|
||||
preview that promised it would promise too much.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import date, datetime, timedelta
|
||||
from typing import NamedTuple
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
REBOOT = "@reboot"
|
||||
|
||||
ALIASES = {
|
||||
"@yearly": "0 0 1 1 *",
|
||||
"@annually": "0 0 1 1 *",
|
||||
"@monthly": "0 0 1 * *",
|
||||
"@weekly": "0 0 * * 0",
|
||||
"@daily": "0 0 * * *",
|
||||
"@midnight": "0 0 * * *",
|
||||
"@hourly": "0 * * * *",
|
||||
}
|
||||
|
||||
_MONTHS = ("jan", "feb", "mar", "apr", "may", "jun", "jul", "aug", "sep", "oct", "nov", "dec")
|
||||
_WEEKDAYS = ("sun", "mon", "tue", "wed", "thu", "fri", "sat")
|
||||
_SYSTEMD_WEEKDAYS = ("Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat")
|
||||
|
||||
#: (lowest, highest, names) per field; a name stands for its index plus the lowest.
|
||||
_FIELDS = (
|
||||
(0, 59, ()),
|
||||
(0, 23, ()),
|
||||
(1, 31, ()),
|
||||
(1, 12, _MONTHS),
|
||||
(0, 7, _WEEKDAYS),
|
||||
)
|
||||
|
||||
#: How far ahead a next run is looked for: a 29 February comes within eight years.
|
||||
_HORIZON_DAYS = 8 * 366
|
||||
|
||||
|
||||
class CronSchedule(NamedTuple):
|
||||
"""An expression's values per field, and whether a day matches on either day field."""
|
||||
|
||||
minutes: frozenset[int]
|
||||
hours: frozenset[int]
|
||||
days: frozenset[int]
|
||||
months: frozenset[int]
|
||||
weekdays: frozenset[int] # 0 = Sunday
|
||||
either_day: bool
|
||||
|
||||
|
||||
def is_reboot(expr: str) -> bool:
|
||||
"""Whether *expr* is ``@reboot``: once at boot, no calendar."""
|
||||
return expr.strip().lower() == REBOOT
|
||||
|
||||
|
||||
def _value(token: str, low: int, names: tuple[str, ...]) -> int:
|
||||
if token.isdigit():
|
||||
return int(token)
|
||||
if token.lower() in names:
|
||||
return names.index(token.lower()) + low
|
||||
raise ValueError(f"not a value: {token!r}")
|
||||
|
||||
|
||||
def _part(part: str, low: int, high: int, names: tuple[str, ...]) -> set[int]:
|
||||
span, _, step_text = part.partition("/")
|
||||
step = 1
|
||||
if step_text:
|
||||
if not step_text.isdigit() or int(step_text) == 0:
|
||||
raise ValueError(f"not a step: {part!r}")
|
||||
step = int(step_text)
|
||||
if span == "*":
|
||||
first, last = low, high
|
||||
elif "-" in span:
|
||||
a, _, b = span.partition("-")
|
||||
first, last = _value(a, low, names), _value(b, low, names)
|
||||
else:
|
||||
first = _value(span, low, names)
|
||||
last = high if step_text else first
|
||||
if not low <= first <= last <= high:
|
||||
raise ValueError(f"out of range {low}-{high}: {part!r}")
|
||||
return set(range(first, last + 1, step))
|
||||
|
||||
|
||||
def _field(text: str, low: int, high: int, names: tuple[str, ...]) -> frozenset[int]:
|
||||
values: set[int] = set()
|
||||
for part in text.split(","):
|
||||
if not part:
|
||||
raise ValueError(f"empty list item in {text!r}")
|
||||
values |= _part(part, low, high, names)
|
||||
return frozenset(values)
|
||||
|
||||
|
||||
def parse_cron(expr: str) -> CronSchedule:
|
||||
"""The values *expr* stands for.
|
||||
|
||||
:raises ValueError: for anything Vixie cron would not take, and for
|
||||
``@reboot``, which names no time.
|
||||
"""
|
||||
text = expr.strip()
|
||||
if is_reboot(text):
|
||||
raise ValueError("@reboot names no time; it runs once at boot")
|
||||
if text.startswith("@"):
|
||||
if text.lower() not in ALIASES:
|
||||
raise ValueError(f"unknown alias {text!r}")
|
||||
text = ALIASES[text.lower()]
|
||||
fields = text.split()
|
||||
if len(fields) != len(_FIELDS):
|
||||
raise ValueError(f"a cron expression has five fields, not {len(fields)}: {expr!r}")
|
||||
minutes, hours, days, months, weekdays = (
|
||||
_field(f, low, high, names) for f, (low, high, names) in zip(fields, _FIELDS, strict=True)
|
||||
)
|
||||
if 7 in weekdays:
|
||||
weekdays = (weekdays - {7}) | {0}
|
||||
either = not fields[2].startswith("*") and not fields[4].startswith("*")
|
||||
return CronSchedule(minutes, hours, days, months, frozenset(weekdays), either)
|
||||
|
||||
|
||||
def _day_matches(s: CronSchedule, day: date) -> bool:
|
||||
if day.month not in s.months:
|
||||
return False
|
||||
on_day = day.day in s.days
|
||||
on_weekday = (day.isoweekday() % 7) in s.weekdays
|
||||
return (on_day or on_weekday) if s.either_day else (on_day and on_weekday)
|
||||
|
||||
|
||||
def _local(day: date, hour: int, minute: int, tz: ZoneInfo) -> datetime | None:
|
||||
"""*day* at *hour*:*minute* in *tz*; None when the clock skips that time."""
|
||||
naive = datetime(day.year, day.month, day.day, hour, minute)
|
||||
aware = naive.replace(tzinfo=tz)
|
||||
if aware.astimezone(ZoneInfo("UTC")).astimezone(tz).replace(tzinfo=None) != naive:
|
||||
return None
|
||||
return aware
|
||||
|
||||
|
||||
def next_runs(expr: str, after: datetime, tz: ZoneInfo, count: int = 5) -> list[datetime]:
|
||||
"""The next *count* times *expr* fires after *after*, in *tz*.
|
||||
|
||||
``@reboot`` and a day that never comes (31 February) give none.
|
||||
|
||||
:raises ValueError: for an expression :func:`parse_cron` refuses.
|
||||
"""
|
||||
if is_reboot(expr):
|
||||
return []
|
||||
s = parse_cron(expr)
|
||||
start = after.astimezone(tz)
|
||||
runs: list[datetime] = []
|
||||
for offset in range(_HORIZON_DAYS):
|
||||
day = start.date() + timedelta(days=offset)
|
||||
if not _day_matches(s, day):
|
||||
continue
|
||||
for hour in sorted(s.hours):
|
||||
for minute in sorted(s.minutes):
|
||||
at = _local(day, hour, minute, tz)
|
||||
if at is None or at <= after:
|
||||
continue
|
||||
runs.append(at)
|
||||
if len(runs) == count:
|
||||
return runs
|
||||
return runs
|
||||
|
||||
|
||||
def _values(values: frozenset[int], low: int, high: int) -> str:
|
||||
if values == frozenset(range(low, high + 1)):
|
||||
return "*"
|
||||
return ",".join(f"{v:02d}" for v in sorted(values))
|
||||
|
||||
|
||||
def to_on_calendar(expr: str) -> list[str]:
|
||||
"""``OnCalendar`` lines that fire exactly when *expr* does.
|
||||
|
||||
:raises ValueError: for an expression :func:`parse_cron` refuses, ``@reboot``
|
||||
among them -- a timer takes that as ``OnBootSec``.
|
||||
"""
|
||||
s = parse_cron(expr)
|
||||
time = f"{_values(s.hours, 0, 23)}:{_values(s.minutes, 0, 59)}:00"
|
||||
months = _values(s.months, 1, 12)
|
||||
days = _values(s.days, 1, 31)
|
||||
weekdays = ",".join(_SYSTEMD_WEEKDAYS[d] for d in (1, 2, 3, 4, 5, 6, 0) if d in s.weekdays)
|
||||
every_weekday = s.weekdays == frozenset(range(7))
|
||||
if s.either_day:
|
||||
return [f"*-{months}-{days} {time}", f"{weekdays} *-{months}-* {time}"]
|
||||
prefix = "" if every_weekday else f"{weekdays} "
|
||||
return [f"{prefix}*-{months}-{days} {time}"]
|
||||
+65
-51
@@ -1,4 +1,3 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""DHCP desired-state management, shared by firewalls and gateways.
|
||||
|
||||
Covers two independent desired-state sets:
|
||||
@@ -24,14 +23,17 @@ takes DHCP down for an entire VLAN.
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from typing import Any, Dict, Iterator, List, Optional, TYPE_CHECKING
|
||||
from collections.abc import Iterator
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from napalm_device_types._matching import check_desired, index_live
|
||||
from napalm_device_types.models import (
|
||||
DhcpReservationDiffDict,
|
||||
DhcpHaDict,
|
||||
DhcpReservationDict,
|
||||
DhcpReservationDiffDict,
|
||||
DhcpReservationUpdateDict,
|
||||
DhcpSubnetDiffDict,
|
||||
DhcpSubnetDict,
|
||||
DhcpSubnetDiffDict,
|
||||
DhcpSubnetUpdateDict,
|
||||
)
|
||||
|
||||
@@ -56,7 +58,7 @@ _DHCP_SUBNET_COMPARE_FIELDS = (
|
||||
_HEX_ONLY = re.compile(r"[^0-9a-f]")
|
||||
|
||||
|
||||
def normalize_mac(mac: Optional[str]) -> str:
|
||||
def normalize_mac(mac: str | None) -> str:
|
||||
"""Reduces a MAC address to lowercase colon-separated form.
|
||||
|
||||
Devices report MACs in whatever form their config store happens to use --
|
||||
@@ -78,7 +80,7 @@ def normalize_mac(mac: Optional[str]) -> str:
|
||||
return ":".join(hex_digits[i : i + 2] for i in range(0, 12, 2))
|
||||
|
||||
|
||||
def normalize_cidr(cidr: Optional[str]) -> str:
|
||||
def normalize_cidr(cidr: str | None) -> str:
|
||||
"""Reduces a subnet CIDR to a canonical string for matching.
|
||||
|
||||
Only whitespace and case are normalised -- deliberately not the network
|
||||
@@ -89,9 +91,7 @@ def normalize_cidr(cidr: Optional[str]) -> str:
|
||||
return (cidr or "").strip().lower()
|
||||
|
||||
|
||||
def _subnet_field_differs(
|
||||
field: str, live: DhcpSubnetDict, desired: DhcpSubnetDict
|
||||
) -> bool:
|
||||
def _subnet_field_differs(field: str, live: DhcpSubnetDict, desired: DhcpSubnetDict) -> bool:
|
||||
"""Compares one subnet field, with `option_data` handled specially.
|
||||
|
||||
For `option_data` only the options `desired` actually names are compared;
|
||||
@@ -103,9 +103,7 @@ def _subnet_field_differs(
|
||||
|
||||
desired_options = desired.get("option_data") or {}
|
||||
live_options = live.get("option_data") or {}
|
||||
return any(
|
||||
live_options.get(option) != value for option, value in desired_options.items()
|
||||
)
|
||||
return any(live_options.get(option) != value for option, value in desired_options.items())
|
||||
|
||||
|
||||
class DhcpServerMixin:
|
||||
@@ -123,7 +121,7 @@ class DhcpServerMixin:
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
||||
def get_dhcp_reservations(self) -> List[DhcpReservationDict]:
|
||||
def get_dhcp_reservations(self) -> list[DhcpReservationDict]:
|
||||
"""
|
||||
Returns all static DHCP reservations currently configured on the
|
||||
device, across all subnets.
|
||||
@@ -137,8 +135,8 @@ class DhcpServerMixin:
|
||||
...
|
||||
|
||||
def apply_dhcp_reservation(
|
||||
self, reservation: DhcpReservationDict, *, uuid: Optional[str] = None
|
||||
) -> Dict[str, Any]:
|
||||
self, reservation: DhcpReservationDict, *, uuid: str | None = None
|
||||
) -> dict[str, Any]:
|
||||
"""
|
||||
Creates or updates a single static DHCP reservation on the device.
|
||||
|
||||
@@ -155,7 +153,7 @@ class DhcpServerMixin:
|
||||
"""
|
||||
...
|
||||
|
||||
def get_dhcp_subnets(self) -> List[DhcpSubnetDict]:
|
||||
def get_dhcp_subnets(self) -> list[DhcpSubnetDict]:
|
||||
"""
|
||||
Returns every DHCPv4 subnet the device serves, with its pools and
|
||||
per-subnet options.
|
||||
@@ -166,8 +164,8 @@ class DhcpServerMixin:
|
||||
...
|
||||
|
||||
def apply_dhcp_subnet(
|
||||
self, subnet: DhcpSubnetDict, *, uuid: Optional[str] = None
|
||||
) -> Dict[str, Any]:
|
||||
self, subnet: DhcpSubnetDict, *, uuid: str | None = None
|
||||
) -> dict[str, Any]:
|
||||
"""
|
||||
Creates or updates a single DHCPv4 subnet on the device.
|
||||
|
||||
@@ -187,7 +185,7 @@ class DhcpServerMixin:
|
||||
"""
|
||||
...
|
||||
|
||||
def commit_dhcp_subnets(self) -> Dict[str, Any]:
|
||||
def commit_dhcp_subnets(self) -> dict[str, Any]:
|
||||
"""
|
||||
Applies pending subnet changes (e.g. Kea's ``service/reconfigure``).
|
||||
|
||||
@@ -202,7 +200,7 @@ class DhcpServerMixin:
|
||||
"""
|
||||
...
|
||||
|
||||
def commit_dhcp_reservations(self) -> Dict[str, Any]:
|
||||
def commit_dhcp_reservations(self) -> dict[str, Any]:
|
||||
"""
|
||||
Applies pending reservation changes (e.g. Kea's ``service/reconfigure``,
|
||||
or a dnsmasq reload).
|
||||
@@ -218,13 +216,25 @@ class DhcpServerMixin:
|
||||
"""
|
||||
...
|
||||
|
||||
def get_dhcp_ha(self) -> DhcpHaDict:
|
||||
"""
|
||||
Returns what the server's own HA configuration declares: whether HA
|
||||
is switched on, which peer this server is, and each peer with its
|
||||
role as the server words it.
|
||||
|
||||
Optional: a driver without it has no ``get_dhcp_ha`` at all, and a
|
||||
caller asks ``callable(getattr(driver, "get_dhcp_ha", None))``.
|
||||
|
||||
:raises Exception: When the configuration could not be read -- never
|
||||
``enabled=False`` for "could not ask".
|
||||
"""
|
||||
...
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Generic, vendor-neutral algorithms.
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def diff_dhcp_reservations(
|
||||
self, desired: List[DhcpReservationDict]
|
||||
) -> DhcpReservationDiffDict:
|
||||
def diff_dhcp_reservations(self, desired: list[DhcpReservationDict]) -> DhcpReservationDiffDict:
|
||||
"""
|
||||
Compares `desired` against the device's current reservations and
|
||||
returns what would need to change to reach that state.
|
||||
@@ -238,20 +248,27 @@ class DhcpServerMixin:
|
||||
those apart from ones simply no longer wanted. Callers wanting
|
||||
delete/cleanup semantics must implement that themselves, deliberately.
|
||||
|
||||
Live reservations without a MAC are never matched. A desired one
|
||||
without a MAC, a MAC given twice, or one more than one live
|
||||
reservation holds raise ``ValueError``.
|
||||
|
||||
:param desired: The complete desired reservation set.
|
||||
:returns: ``{"add": [...], "update": [{"uuid", "reservation",
|
||||
"changed_fields"}, ...]}``.
|
||||
:raises ValueError: When a desired reservation cannot name a single live one.
|
||||
"""
|
||||
live_by_mac: Dict[str, DhcpReservationDict] = {
|
||||
normalize_mac(reservation.get("mac")): reservation
|
||||
for reservation in self.get_dhcp_reservations()
|
||||
}
|
||||
|
||||
add: List[DhcpReservationDict] = []
|
||||
update: List[DhcpReservationUpdateDict] = []
|
||||
def key(reservation: DhcpReservationDict) -> str:
|
||||
return normalize_mac(reservation.get("mac"))
|
||||
|
||||
live_by_mac, shared = index_live(self.get_dhcp_reservations(), key)
|
||||
check_desired(desired, key, shared, "MAC")
|
||||
|
||||
add: list[DhcpReservationDict] = []
|
||||
update: list[DhcpReservationUpdateDict] = []
|
||||
|
||||
for desired_reservation in desired:
|
||||
live = live_by_mac.get(normalize_mac(desired_reservation.get("mac")))
|
||||
live = live_by_mac.get(key(desired_reservation))
|
||||
if live is None:
|
||||
add.append(desired_reservation)
|
||||
continue
|
||||
@@ -272,9 +289,7 @@ class DhcpServerMixin:
|
||||
|
||||
return {"add": add, "update": update}
|
||||
|
||||
def apply_dhcp_reservationset(
|
||||
self, desired: List[DhcpReservationDict]
|
||||
) -> Iterator[str]:
|
||||
def apply_dhcp_reservationset(self, desired: list[DhcpReservationDict]) -> Iterator[str]:
|
||||
"""
|
||||
Computes the diff against `desired` and applies it, yielding one
|
||||
human-readable progress line per change, then commits.
|
||||
@@ -297,8 +312,7 @@ class DhcpServerMixin:
|
||||
self.apply_dhcp_reservation(entry["reservation"], uuid=entry["uuid"])
|
||||
fields = ", ".join(entry["changed_fields"])
|
||||
yield (
|
||||
f"[update] {entry['reservation']['mac']} -> "
|
||||
f"{entry['reservation']['ip']} ({fields})"
|
||||
f"[update] {entry['reservation']['mac']} -> {entry['reservation']['ip']} ({fields})"
|
||||
)
|
||||
|
||||
if not diff["add"] and not diff["update"]:
|
||||
@@ -306,12 +320,9 @@ class DhcpServerMixin:
|
||||
return
|
||||
|
||||
self.commit_dhcp_reservations()
|
||||
yield (
|
||||
f"[commit] applied {len(diff['add'])} add(s), "
|
||||
f"{len(diff['update'])} update(s)"
|
||||
)
|
||||
yield (f"[commit] applied {len(diff['add'])} add(s), {len(diff['update'])} update(s)")
|
||||
|
||||
def diff_dhcp_subnets(self, desired: List[DhcpSubnetDict]) -> DhcpSubnetDiffDict:
|
||||
def diff_dhcp_subnets(self, desired: list[DhcpSubnetDict]) -> DhcpSubnetDiffDict:
|
||||
"""
|
||||
Compares `desired` against the device's current subnets and returns
|
||||
what would need to change to reach that state.
|
||||
@@ -333,19 +344,25 @@ class DhcpServerMixin:
|
||||
subnet takes DHCP down for a whole VLAN, and this method cannot tell
|
||||
"no longer wanted" from "was never netOrk's to describe".
|
||||
|
||||
An empty or repeated subnet raises ``ValueError``, as for reservations.
|
||||
|
||||
:param desired: The complete desired subnet set.
|
||||
:returns: ``{"add": [...], "update": [{"uuid", "subnet",
|
||||
"changed_fields"}, ...]}``.
|
||||
:raises ValueError: When a desired subnet cannot name a single live one.
|
||||
"""
|
||||
live_by_cidr: Dict[str, DhcpSubnetDict] = {
|
||||
normalize_cidr(live.get("subnet")): live for live in self.get_dhcp_subnets()
|
||||
}
|
||||
|
||||
add: List[DhcpSubnetDict] = []
|
||||
update: List[DhcpSubnetUpdateDict] = []
|
||||
def key(subnet: DhcpSubnetDict) -> str:
|
||||
return normalize_cidr(subnet.get("subnet"))
|
||||
|
||||
live_by_cidr, shared = index_live(self.get_dhcp_subnets(), key)
|
||||
check_desired(desired, key, shared, "subnet")
|
||||
|
||||
add: list[DhcpSubnetDict] = []
|
||||
update: list[DhcpSubnetUpdateDict] = []
|
||||
|
||||
for desired_subnet in desired:
|
||||
live = live_by_cidr.get(normalize_cidr(desired_subnet.get("subnet")))
|
||||
live = live_by_cidr.get(key(desired_subnet))
|
||||
if live is None:
|
||||
add.append(desired_subnet)
|
||||
continue
|
||||
@@ -366,7 +383,7 @@ class DhcpServerMixin:
|
||||
|
||||
return {"add": add, "update": update}
|
||||
|
||||
def apply_dhcp_subnetset(self, desired: List[DhcpSubnetDict]) -> Iterator[str]:
|
||||
def apply_dhcp_subnetset(self, desired: list[DhcpSubnetDict]) -> Iterator[str]:
|
||||
"""
|
||||
Computes the diff against `desired` and applies it, yielding one
|
||||
human-readable progress line per change, then commits.
|
||||
@@ -394,7 +411,4 @@ class DhcpServerMixin:
|
||||
return
|
||||
|
||||
self.commit_dhcp_subnets()
|
||||
yield (
|
||||
f"[commit] applied {len(diff['add'])} add(s), "
|
||||
f"{len(diff['update'])} update(s)"
|
||||
)
|
||||
yield (f"[commit] applied {len(diff['add'])} add(s), {len(diff['update'])} update(s)")
|
||||
@@ -0,0 +1,34 @@
|
||||
"""Domains a device's DNS resolver forwards to another server.
|
||||
|
||||
An application that makes the device's resolver ask another server for a zone
|
||||
-- the authority for an internal domain -- edits these. The driver writes them
|
||||
in its device's terms and applies them; the forwards are the caller's.
|
||||
Declared only; a driver whose device resolves DNS implements it.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from napalm_device_types.models import DnsForwardDict
|
||||
|
||||
|
||||
class DnsForwardMixin:
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def get_dns_forwards(self) -> list[DnsForwardDict]:
|
||||
"""Every domain the resolver forwards, enabled or not."""
|
||||
...
|
||||
|
||||
def set_dns_forward(self, domain: str, server: str, port: int = 53) -> None:
|
||||
"""Forward *domain* to *server*: added, or the existing forward of
|
||||
*domain* changed. Applied before it returns.
|
||||
|
||||
:raises RuntimeError: when the device refuses it.
|
||||
"""
|
||||
...
|
||||
|
||||
def remove_dns_forward(self, domain: str) -> None:
|
||||
"""Stop forwarding *domain*; nothing to do when it is not forwarded.
|
||||
Applied before it returns."""
|
||||
...
|
||||
@@ -10,21 +10,28 @@ Usage::
|
||||
...
|
||||
"""
|
||||
|
||||
from typing import Any, ClassVar, Dict, Iterator, List, Optional, TYPE_CHECKING
|
||||
from typing import TYPE_CHECKING, Any, ClassVar
|
||||
|
||||
from napalm_device_types.base import DeviceTypeDriver
|
||||
from napalm_device_types.nat_vpn import NatVpnMixin
|
||||
from napalm_device_types.packages import PackageManagementMixin
|
||||
from napalm_device_types.health_metrics import HealthMetricsMixin
|
||||
from napalm_device_types.dhcp import DhcpServerMixin
|
||||
from napalm_device_types.firewall_rules import FirewallRuleMixin
|
||||
from napalm_device_types.health_metrics import HealthMetricsMixin
|
||||
from napalm_device_types.models import (
|
||||
SecurityZoneDict,
|
||||
SessionDict,
|
||||
)
|
||||
from napalm_device_types.nat_vpn import NatVpnMixin
|
||||
from napalm_device_types.packages import PackageManagementMixin
|
||||
|
||||
|
||||
|
||||
class FirewallDriver(NatVpnMixin, PackageManagementMixin, HealthMetricsMixin, FirewallRuleMixin, DhcpServerMixin, DeviceTypeDriver):
|
||||
class FirewallDriver(
|
||||
NatVpnMixin,
|
||||
PackageManagementMixin,
|
||||
HealthMetricsMixin,
|
||||
FirewallRuleMixin,
|
||||
DhcpServerMixin,
|
||||
DeviceTypeDriver,
|
||||
):
|
||||
"""
|
||||
Abstract intermediate driver for firewall/security devices.
|
||||
|
||||
@@ -42,11 +49,9 @@ class FirewallDriver(NatVpnMixin, PackageManagementMixin, HealthMetricsMixin, Fi
|
||||
# OPNsense reports drops in the out-error counter.
|
||||
_SNMP_TX_ERR_IS_DROP: ClassVar[bool] = True
|
||||
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
||||
|
||||
def get_security_zones(self) -> Dict[str, SecurityZoneDict]:
|
||||
def get_security_zones(self) -> dict[str, SecurityZoneDict]:
|
||||
"""
|
||||
Returns the security zone configuration.
|
||||
|
||||
@@ -73,7 +78,7 @@ class FirewallDriver(NatVpnMixin, PackageManagementMixin, HealthMetricsMixin, Fi
|
||||
"""
|
||||
...
|
||||
|
||||
def get_sessions(self) -> List[SessionDict]:
|
||||
def get_sessions(self) -> list[SessionDict]:
|
||||
"""
|
||||
Returns a list of active connection sessions (stateful flows).
|
||||
|
||||
@@ -103,8 +108,7 @@ class FirewallDriver(NatVpnMixin, PackageManagementMixin, HealthMetricsMixin, Fi
|
||||
"""
|
||||
...
|
||||
|
||||
|
||||
def send_wake_on_lan(self, mac_address: str, interface: str = "") -> Dict[str, Any]:
|
||||
def send_wake_on_lan(self, mac_address: str, interface: str = "") -> dict[str, Any]:
|
||||
"""
|
||||
Sends a Wake-on-LAN "magic packet" to wake a host on the network.
|
||||
|
||||
@@ -142,19 +146,9 @@ class FirewallDriver(NatVpnMixin, PackageManagementMixin, HealthMetricsMixin, Fi
|
||||
"""
|
||||
...
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Firewall rule diff/apply. get_firewall_rules/apply_firewall_rule/
|
||||
# commit_firewall_rules are abstract (device communication); everything
|
||||
# else here is a concrete, vendor-neutral algorithm -- see README.md
|
||||
# "Design principle: generic vs. device-specific logic".
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Firewall rule reconciliation: generic here, device communication in the driver.
|
||||
|
||||
The split follows the rule in this package's README: matching, diffing and the
|
||||
@@ -14,8 +13,10 @@ working implementation inherited from elsewhere.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any, Dict, Iterator, List, Optional, TYPE_CHECKING
|
||||
from collections.abc import Iterator
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from napalm_device_types._matching import check_desired, index_live
|
||||
from napalm_device_types.models import (
|
||||
FirewallRuleDict,
|
||||
FirewallRuleDiffDict,
|
||||
@@ -42,7 +43,7 @@ class FirewallRuleMixin:
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
||||
def get_firewall_rules(self) -> List[FirewallRuleDict]:
|
||||
def get_firewall_rules(self) -> list[FirewallRuleDict]:
|
||||
"""
|
||||
Returns all firewall filter rules currently configured on the device.
|
||||
|
||||
@@ -54,9 +55,10 @@ class FirewallRuleMixin:
|
||||
firewall rules.
|
||||
"""
|
||||
...
|
||||
|
||||
def apply_firewall_rule(
|
||||
self, rule: FirewallRuleDict, *, uuid: Optional[str] = None
|
||||
) -> Dict[str, Any]:
|
||||
self, rule: FirewallRuleDict, *, uuid: str | None = None
|
||||
) -> dict[str, Any]:
|
||||
"""
|
||||
Creates or updates a single firewall filter rule on the device.
|
||||
|
||||
@@ -72,7 +74,8 @@ class FirewallRuleMixin:
|
||||
:returns: A dict with at least ``{"success": bool}``.
|
||||
"""
|
||||
...
|
||||
def commit_firewall_rules(self) -> Dict[str, Any]:
|
||||
|
||||
def commit_firewall_rules(self) -> dict[str, Any]:
|
||||
"""
|
||||
Applies pending firewall filter rule changes (e.g. reloads pf/pfctl,
|
||||
or whatever the device's equivalent of "Apply Changes" is).
|
||||
@@ -87,7 +90,7 @@ class FirewallRuleMixin:
|
||||
"""
|
||||
...
|
||||
|
||||
def diff_firewall_rules(self, desired: List[FirewallRuleDict]) -> FirewallRuleDiffDict:
|
||||
def diff_firewall_rules(self, desired: list[FirewallRuleDict]) -> FirewallRuleDiffDict:
|
||||
"""
|
||||
Compares `desired` against the device's current rules and returns
|
||||
what would need to change to reach that state.
|
||||
@@ -102,19 +105,28 @@ class FirewallRuleMixin:
|
||||
from ones simply no longer wanted. Callers wanting delete/cleanup
|
||||
semantics must implement that themselves, deliberately.
|
||||
|
||||
Live rules without a description are never matched. A desired rule
|
||||
without one, two desired rules with the same one, or one whose
|
||||
description more than one live rule holds raise ``ValueError``: any
|
||||
guess would overwrite a rule the caller did not mean.
|
||||
|
||||
:param desired: The complete desired rule set.
|
||||
:returns: ``{"add": [...], "update": [{"uuid", "rule",
|
||||
"changed_fields"}, ...]}``.
|
||||
:raises ValueError: When a desired rule cannot name a single live one.
|
||||
"""
|
||||
live_by_description: Dict[str, FirewallRuleDict] = {
|
||||
rule["description"]: rule for rule in self.get_firewall_rules()
|
||||
}
|
||||
|
||||
add: List[FirewallRuleDict] = []
|
||||
update: List[FirewallRuleUpdateDict] = []
|
||||
def key(rule: FirewallRuleDict) -> str:
|
||||
return (rule.get("description") or "").strip()
|
||||
|
||||
live_by_description, shared = index_live(self.get_firewall_rules(), key)
|
||||
check_desired(desired, key, shared, "description")
|
||||
|
||||
add: list[FirewallRuleDict] = []
|
||||
update: list[FirewallRuleUpdateDict] = []
|
||||
|
||||
for desired_rule in desired:
|
||||
live_rule = live_by_description.get(desired_rule["description"])
|
||||
live_rule = live_by_description.get(key(desired_rule))
|
||||
if live_rule is None:
|
||||
add.append(desired_rule)
|
||||
continue
|
||||
@@ -134,7 +146,8 @@ class FirewallRuleMixin:
|
||||
)
|
||||
|
||||
return {"add": add, "update": update}
|
||||
def apply_firewall_ruleset(self, desired: List[FirewallRuleDict]) -> Iterator[str]:
|
||||
|
||||
def apply_firewall_ruleset(self, desired: list[FirewallRuleDict]) -> Iterator[str]:
|
||||
"""
|
||||
Computes the diff against `desired` and applies it, yielding one
|
||||
human-readable progress line per change, then commits.
|
||||
|
||||
@@ -0,0 +1,31 @@
|
||||
"""The firmware a vendor offers for a device.
|
||||
|
||||
A fact for the caller (docs/STRUCTURE.md): a driver may know the newer firmware
|
||||
from the vendor's download catalogue or the device's own update check; whether
|
||||
running an older one is a problem is decided above the driver. Comparing the
|
||||
vendor's version strings is vendor knowledge, so the driver says itself whether
|
||||
the offer is newer. Declared only; a driver that can tell implements it.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from napalm_device_types.models import FirmwareOfferDict
|
||||
|
||||
|
||||
class FirmwareOfferMixin:
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def get_firmware_offer(self) -> FirmwareOfferDict | None:
|
||||
"""The firmware running and the newest the vendor offers.
|
||||
|
||||
None when the driver cannot tell for this device (an unknown model,
|
||||
a catalogue that did not answer).
|
||||
|
||||
Example::
|
||||
|
||||
{"current_version": "1.0.2.10", "latest_version": "1.0.2.12",
|
||||
"newer": True, "download_url": "https://.../GS308E_1.0.2.12.zip"}
|
||||
"""
|
||||
...
|
||||
@@ -0,0 +1,219 @@
|
||||
"""Fix routines: changes an application wants made on a device, as templates.
|
||||
|
||||
A template fixes the order of the steps; the driver knows how each step is done on
|
||||
its device; the caller supplies every value -- the community, the allowed sources,
|
||||
the proxy URL, the user. A driver whose device takes the whole change in one call
|
||||
(a REST API) overrides the template method instead of its steps.
|
||||
|
||||
None of these is mixed into a role base. A driver mixes in the fixes its device can
|
||||
take, so ``callable(getattr(driver, "configure_snmp_agent", None))`` stays a truthful
|
||||
answer; a driver that inherits one it cannot run sets it to ``None``.
|
||||
|
||||
They replace ``run_device_action`` (docs/STRUCTURE.md):
|
||||
|
||||
========================== ===================================================
|
||||
``fix_snmp`` :meth:`SnmpAgentMixin.configure_snmp_agent`
|
||||
``fix_apt_proxy`` :meth:`PackageProxyMixin.configure_package_proxy`
|
||||
``fix_docker_permissions`` ``ContainerEngineMixin.grant_container_engine_access``
|
||||
``install_lldpd`` and co. :meth:`EnsurePackageMixin.ensure_package`
|
||||
========================== ===================================================
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable, Sequence
|
||||
from typing import TYPE_CHECKING, Any, ClassVar
|
||||
|
||||
from napalm_device_types.models import (
|
||||
FixResultDict,
|
||||
FixStepDict,
|
||||
FixStepOutcomeDict,
|
||||
PackageDict,
|
||||
SnmpAgentSettingsDict,
|
||||
)
|
||||
|
||||
#: One step of a fix: its name and the call that does it.
|
||||
FixStep = tuple[str, Callable[[], FixStepOutcomeDict]]
|
||||
|
||||
|
||||
def run_fix_steps(steps: Sequence[FixStep]) -> FixResultDict:
|
||||
"""Run *steps* in order and stop at the first that fails.
|
||||
|
||||
A step that raises has failed, with the exception's message as its output.
|
||||
"""
|
||||
done: list[FixStepDict] = []
|
||||
for name, step in steps:
|
||||
try:
|
||||
outcome = step()
|
||||
ok, output = bool(outcome["ok"]), str(outcome.get("output", ""))
|
||||
except Exception as exc: # noqa: BLE001 - any failure of a step ends the fix
|
||||
ok, output = False, str(exc)
|
||||
done.append({"step": name, "ok": ok, "output": output})
|
||||
if not ok:
|
||||
break
|
||||
return {
|
||||
"success": bool(done) and all(s["ok"] for s in done) and len(done) == len(steps),
|
||||
"steps": done,
|
||||
"output": "\n".join(f"[{s['step']}] {s['output']}".rstrip() for s in done),
|
||||
}
|
||||
|
||||
|
||||
class SnmpAgentMixin:
|
||||
"""Run an SNMP agent on the device that answers the caller.
|
||||
|
||||
The steps: install the agent, write its configuration, let the allowed sources
|
||||
through the host's firewall, start it. Whether it answers is the caller's to ask,
|
||||
with its own SNMP client.
|
||||
"""
|
||||
|
||||
def configure_snmp_agent(self, settings: SnmpAgentSettingsDict) -> FixResultDict:
|
||||
"""Install, configure, open and start the agent with *settings*."""
|
||||
return run_fix_steps(
|
||||
[
|
||||
("install", self._install_snmp_agent),
|
||||
("configure", lambda: self._write_snmp_agent_config(settings)),
|
||||
("firewall", lambda: self._allow_snmp_sources(settings)),
|
||||
("start", self._start_snmp_agent),
|
||||
]
|
||||
)
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def get_management_peer(self) -> str | None:
|
||||
"""The address the device sees this management connection come from
|
||||
(for SSH, the client side of ``SSH_CONNECTION``), or None when it cannot
|
||||
tell. A fact for the caller choosing ``allowed_sources``; optional --
|
||||
a driver that cannot tell does not implement it."""
|
||||
...
|
||||
|
||||
# The steps a driver implements:
|
||||
|
||||
def _install_snmp_agent(self) -> FixStepOutcomeDict: ...
|
||||
|
||||
def _write_snmp_agent_config(
|
||||
self, settings: SnmpAgentSettingsDict
|
||||
) -> FixStepOutcomeDict: ...
|
||||
|
||||
def _allow_snmp_sources(self, settings: SnmpAgentSettingsDict) -> FixStepOutcomeDict:
|
||||
"""Let ``settings["allowed_sources"]`` reach the agent; ``ok`` with nothing
|
||||
to do where the host has no firewall of its own."""
|
||||
...
|
||||
|
||||
def _start_snmp_agent(self) -> FixStepOutcomeDict:
|
||||
"""(Re)start the agent so it reads the new configuration, and enable it at boot."""
|
||||
...
|
||||
|
||||
|
||||
class EnsurePackageMixin:
|
||||
"""Make sure a package is installed, installing it only where it is missing.
|
||||
|
||||
Works over :class:`~napalm_device_types.packages.PackageManagementMixin`'s
|
||||
``get_packages`` and ``install_package``, which the driver implements.
|
||||
"""
|
||||
|
||||
def ensure_package(self, name: str) -> FixResultDict:
|
||||
"""Install *name* unless it is installed already."""
|
||||
|
||||
def install() -> FixStepOutcomeDict:
|
||||
if any(p.get("name") == name and p.get("installed") for p in self.get_packages()):
|
||||
return {"ok": True, "output": f"{name} is already installed"}
|
||||
result: Any = self.install_package(name) # type: ignore[func-returns-value]
|
||||
# The contract says None (and raise on failure); some drivers still
|
||||
# return {"success", "output"}.
|
||||
if isinstance(result, dict):
|
||||
return {"ok": bool(result.get("success")), "output": str(result.get("output", ""))}
|
||||
return {"ok": True, "output": f"installed {name}"}
|
||||
|
||||
return run_fix_steps([("install", install)])
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover
|
||||
|
||||
def get_packages(self) -> list[PackageDict]: ...
|
||||
|
||||
def install_package(self, name: str, version: str = "") -> None: ...
|
||||
|
||||
|
||||
class LldpAgentMixin:
|
||||
"""Run an LLDP agent on the device, on the interfaces the caller names.
|
||||
|
||||
The steps: install the agent, write the interfaces it runs on, enable it at boot
|
||||
and restart it so it reads them. Installing and starting are generic, over
|
||||
``ensure_package`` and ``manage_service`` with ``LLDP_AGENT_PACKAGE`` and
|
||||
``LLDP_AGENT_SERVICE``; the driver writes the configuration. Which interfaces is
|
||||
the caller's choice -- an agent on every interface also talks to the uplink of a
|
||||
WAN port. Whether neighbours appear is the caller's to read with
|
||||
``get_lldp_neighbors``, after the agent's first advertisement interval.
|
||||
"""
|
||||
|
||||
#: The package that brings the agent, as ``ensure_package`` takes it.
|
||||
LLDP_AGENT_PACKAGE: ClassVar[str] = "lldpd"
|
||||
#: The agent's service, as ``manage_service`` takes it.
|
||||
LLDP_AGENT_SERVICE: ClassVar[str] = "lldpd"
|
||||
|
||||
def configure_lldp_agent(self, interfaces: Sequence[str]) -> FixResultDict:
|
||||
"""Install, configure for *interfaces*, enable and restart the agent.
|
||||
|
||||
:raises ValueError: for no interface, before anything changes.
|
||||
"""
|
||||
names = [str(name) for name in interfaces]
|
||||
if not names:
|
||||
raise ValueError("name at least one interface for the LLDP agent")
|
||||
return run_fix_steps(
|
||||
[
|
||||
("install", self._install_lldp_agent),
|
||||
("configure", lambda: self._write_lldp_agent_config(names)),
|
||||
("start", self._start_lldp_agent),
|
||||
]
|
||||
)
|
||||
|
||||
def _install_lldp_agent(self) -> FixStepOutcomeDict:
|
||||
result = self.ensure_package(self.LLDP_AGENT_PACKAGE)
|
||||
return {"ok": bool(result["success"]), "output": str(result.get("output", ""))}
|
||||
|
||||
def _start_lldp_agent(self) -> FixStepOutcomeDict:
|
||||
service = self.LLDP_AGENT_SERVICE
|
||||
for action in ("enable", "restart"):
|
||||
result = self.manage_service(service, action)
|
||||
if not result.get("success"):
|
||||
output = str(result.get("output", "")).strip()
|
||||
return {"ok": False, "output": f"{action} {service}: {output}".rstrip(": ")}
|
||||
return {"ok": True, "output": f"{service} enabled and restarted"}
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def ensure_package(self, name: str) -> FixResultDict: ...
|
||||
|
||||
def manage_service(self, name: str, action: str) -> dict[str, Any]: ...
|
||||
|
||||
# The step a driver implements:
|
||||
|
||||
def _write_lldp_agent_config(self, interfaces: list[str]) -> FixStepOutcomeDict:
|
||||
"""Have the agent send and listen on exactly *interfaces* (names as the
|
||||
device's interface getters report them), replacing the list it had."""
|
||||
...
|
||||
|
||||
|
||||
class PackageProxyMixin:
|
||||
"""Point the device's package manager at a caching proxy."""
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover
|
||||
|
||||
def configure_package_proxy(self, url: str) -> FixResultDict:
|
||||
"""Have the package manager fetch through *url*; an empty *url* removes it."""
|
||||
...
|
||||
|
||||
def get_package_proxy(self) -> str | None:
|
||||
"""The proxy the package manager fetches through now; None for none.
|
||||
|
||||
A fact: whether it is the right one is the caller's to judge."""
|
||||
...
|
||||
|
||||
|
||||
__all__ = [
|
||||
"EnsurePackageMixin",
|
||||
"FixStep",
|
||||
"LldpAgentMixin",
|
||||
"PackageProxyMixin",
|
||||
"SnmpAgentMixin",
|
||||
"run_fix_steps",
|
||||
]
|
||||
@@ -1,6 +1,10 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""SNMP health collection, shared by every device type that speaks UCD-MIB.
|
||||
|
||||
**Deprecated** (docs/STRUCTURE.md): drivers do not collect over SNMP. The
|
||||
application polls; a driver whose device publishes values under its own OIDs
|
||||
declares them as ``SNMP_HEALTH_PROFILE``
|
||||
(:class:`~napalm_device_types.snmp_health_profile.SnmpHealthProfile`). This mixin goes in 5.0.
|
||||
|
||||
The collection itself is generic: CPU, memory, uptime, load and per-interface
|
||||
counters come from the same standard OIDs on any UCD-MIB/IF-MIB capable device.
|
||||
Only two details vary by device type, and both are class attributes rather than
|
||||
@@ -15,7 +19,9 @@ implement ``get_health_metrics`` themselves.
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from typing import Any, Awaitable, Callable, ClassVar, Dict, cast
|
||||
import warnings
|
||||
from collections.abc import Awaitable, Callable
|
||||
from typing import Any, ClassVar, cast
|
||||
|
||||
from napalm_device_types._ucd_metrics import IF_SKIP_DEFAULT, collect_ucd_metrics
|
||||
from napalm_device_types.models import HealthMetricsDict
|
||||
@@ -37,9 +43,19 @@ class HealthMetricsMixin:
|
||||
async def get_health_metrics(
|
||||
cls,
|
||||
snmp_get: Callable[..., Awaitable[Any]],
|
||||
snmp_walk: Callable[..., Awaitable[Dict[str, Any]]],
|
||||
snmp_walk: Callable[..., Awaitable[dict[str, Any]]],
|
||||
) -> HealthMetricsDict:
|
||||
"""Collect SNMP health metrics (CPU, memory, uptime, load, interfaces)."""
|
||||
"""Collect SNMP health metrics (CPU, memory, uptime, load, interfaces).
|
||||
|
||||
Deprecated: the application polls; see the module docstring.
|
||||
"""
|
||||
warnings.warn(
|
||||
"HealthMetricsMixin.get_health_metrics is deprecated and goes in 5.0: "
|
||||
"the application polls over SNMP; a driver declares only its OIDs, "
|
||||
"as SNMP_HEALTH_PROFILE",
|
||||
DeprecationWarning,
|
||||
stacklevel=2,
|
||||
)
|
||||
# collect_ucd_metrics is untyped shared plumbing; the shape it returns is
|
||||
# the contract HealthMetricsDict describes.
|
||||
return cast(
|
||||
|
||||
@@ -0,0 +1,770 @@
|
||||
"""A Linux host's own firewall: its netfilter rules, read once, in one shape.
|
||||
|
||||
Whether a service is reachable from outside its host is decided twice: by the
|
||||
address it listens on (:mod:`napalm_device_types.listening`), and by the host's
|
||||
own firewall, which may drop the packets first. Reading the firewall is the
|
||||
same on every Linux host, so the command and its parse live here, and a driver
|
||||
only carries the command across.
|
||||
|
||||
**What is read.** ``nft -j list ruleset`` is the primary reading: every table,
|
||||
the host's own ``nftables.service`` ones included. Rules that iptables-nft wrote
|
||||
are in it too, but their extensions show only as an opaque ``xt`` -- a
|
||||
``-m conntrack --ctstate`` or ``-m set`` says nothing there. ``iptables-save``
|
||||
spells them out, so the tables iptables manages are taken from it, and the
|
||||
others from nft. A *legacy* iptables is a second ruleset beside nft, and both
|
||||
apply to every packet, so then both are kept whole.
|
||||
|
||||
**One shape.** Each chain says its family, table, hook, priority and policy;
|
||||
each rule a list of matches and a verdict. A match is ``field``, ``op``
|
||||
(``==``/``!=``) and ``values``. The fields are the few that decide reachability
|
||||
from outside: ``iifname``, ``oifname``, ``saddr``, ``daddr``, ``l4proto``,
|
||||
``dport``, ``sport``, ``ct_state``, ``ct_orig_dport`` and ``nfproto``.
|
||||
Everything else -- an ipset, ``addrtype``, a rate limit, a ``fib`` lookup -- is
|
||||
``unknown`` with its text, never dropped: a rule that matches on something this
|
||||
does not model must not read as one that matches everything.
|
||||
|
||||
**Root, and only root.** Both tools need ``CAP_NET_ADMIN``; without it they
|
||||
print an error, not an empty ruleset. So there is no reading without root:
|
||||
:class:`HostFirewallUnavailable` is raised rather than a host without rules.
|
||||
|
||||
Whether a rule set makes a service unreachable is the consumer's call.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import ipaddress
|
||||
import json
|
||||
import re
|
||||
import shlex
|
||||
from collections.abc import Iterator
|
||||
from shlex import quote
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from napalm_device_types.models import (
|
||||
HostFirewallDict,
|
||||
NetfilterChainDict,
|
||||
NetfilterMatchDict,
|
||||
NetfilterRuleDict,
|
||||
)
|
||||
from napalm_device_types.terminal import strip_terminal_codes
|
||||
|
||||
_BEGIN = "FW_BEGIN"
|
||||
_END = "FW_END"
|
||||
_RC_RE = re.compile(r"^__FW_RC=(\d+)$")
|
||||
|
||||
#: One line, POSIX ``sh``, read-only. The frame markers are printed in two
|
||||
#: halves so that a transport which echoes the command does not show them early.
|
||||
#: Both tools live in ``/usr/sbin`` on most systems, outside a login user's ``PATH``.
|
||||
HOST_FIREWALL_COMMAND = (
|
||||
"PATH=$PATH:/usr/sbin:/sbin; "
|
||||
"printf '%s%s\\n' FW_ BEGIN; "
|
||||
"if command -v nft >/dev/null 2>&1; then "
|
||||
"echo '[nft]'; nft -j list ruleset 2>&1; echo \"__FW_RC=$?\"; fi; "
|
||||
"for t in iptables-save ip6tables-save; do "
|
||||
"if command -v $t >/dev/null 2>&1; then "
|
||||
'echo "[$t]"; $t 2>&1; echo "__FW_RC=$?"; fi; done; '
|
||||
"printf '%s%s\\n' FW_ END"
|
||||
)
|
||||
|
||||
#: ``iptables-save`` reads the IPv4 tables, ``ip6tables-save`` the IPv6 ones.
|
||||
_IPTABLES_FAMILY = {"iptables-save": "ip", "ip6tables-save": "ip6"}
|
||||
|
||||
#: The built-in chains of iptables' tables: ``(type, hook, priority)``, as
|
||||
#: iptables-nft registers them (and as the legacy tables run in the kernel).
|
||||
_BUILT_IN: dict[str, dict[str, tuple[str, str, int]]] = {
|
||||
"raw": {"PREROUTING": ("filter", "prerouting", -300), "OUTPUT": ("filter", "output", -300)},
|
||||
"mangle": {
|
||||
"PREROUTING": ("filter", "prerouting", -150),
|
||||
"INPUT": ("filter", "input", -150),
|
||||
"FORWARD": ("filter", "forward", -150),
|
||||
"OUTPUT": ("route", "output", -150),
|
||||
"POSTROUTING": ("filter", "postrouting", -150),
|
||||
},
|
||||
"nat": {
|
||||
"PREROUTING": ("nat", "prerouting", -100),
|
||||
"INPUT": ("nat", "input", 100),
|
||||
"OUTPUT": ("nat", "output", -100),
|
||||
"POSTROUTING": ("nat", "postrouting", 100),
|
||||
},
|
||||
"filter": {
|
||||
"INPUT": ("filter", "input", 0),
|
||||
"FORWARD": ("filter", "forward", 0),
|
||||
"OUTPUT": ("filter", "output", 0),
|
||||
},
|
||||
"security": {
|
||||
"INPUT": ("filter", "input", 50),
|
||||
"FORWARD": ("filter", "forward", 50),
|
||||
"OUTPUT": ("filter", "output", 50),
|
||||
},
|
||||
}
|
||||
|
||||
_TERMINAL = {"ACCEPT": "accept", "DROP": "drop", "REJECT": "reject", "RETURN": "return"}
|
||||
#: Targets that look at a packet, mark it or log it, and let it go on.
|
||||
_NON_TERMINAL = frozenset(
|
||||
{
|
||||
"LOG",
|
||||
"NFLOG",
|
||||
"ULOG",
|
||||
"TRACE",
|
||||
"MARK",
|
||||
"CONNMARK",
|
||||
"CT",
|
||||
"NOTRACK",
|
||||
"TCPMSS",
|
||||
"CHECKSUM",
|
||||
"AUDIT",
|
||||
"SET",
|
||||
"DSCP",
|
||||
"TOS",
|
||||
"TTL",
|
||||
"HL",
|
||||
"CLASSIFY",
|
||||
"CONNSECMARK",
|
||||
"SECMARK",
|
||||
"IDLETIMER",
|
||||
"LED",
|
||||
"RATEEST",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
class HostFirewallUnavailable(NotImplementedError):
|
||||
"""Nothing on the host could be read: no ``nft``, no ``iptables-save``, or no root."""
|
||||
|
||||
|
||||
def _frame(output: str) -> list[str]:
|
||||
lines = [line.rstrip() for line in strip_terminal_codes(output).splitlines()]
|
||||
stripped = [line.strip() for line in lines]
|
||||
try:
|
||||
start = stripped.index(_BEGIN)
|
||||
end = stripped.index(_END, start)
|
||||
except ValueError:
|
||||
raise ValueError("no intact host firewall report in the output") from None
|
||||
return lines[start + 1 : end]
|
||||
|
||||
|
||||
def _sections(lines: list[str]) -> dict[str, tuple[list[str], int | None]]:
|
||||
"""Each tool's output and exit status."""
|
||||
sections: dict[str, tuple[list[str], int | None]] = {}
|
||||
name: str | None = None
|
||||
body: list[str] = []
|
||||
for line in lines:
|
||||
text = line.strip()
|
||||
if text in ("[nft]", "[iptables-save]", "[ip6tables-save]"):
|
||||
name, body = text[1:-1], []
|
||||
sections[name] = (body, None)
|
||||
continue
|
||||
rc = _RC_RE.match(text)
|
||||
if rc and name is not None:
|
||||
sections[name] = (body, int(rc.group(1)))
|
||||
name = None
|
||||
continue
|
||||
if name is not None:
|
||||
body.append(line)
|
||||
return sections
|
||||
|
||||
|
||||
# ── iptables-save ────────────────────────────────────────────────────────────
|
||||
|
||||
#: The token that negates the option after it (``! -s 10.0.0.0/8``).
|
||||
_NEGATION = "!"
|
||||
|
||||
#: Options that start a new part of a rule, outside any ``-m`` module.
|
||||
_TOP_LEVEL = frozenset(
|
||||
{
|
||||
"-m",
|
||||
"--match",
|
||||
"-j",
|
||||
"--jump",
|
||||
"-g",
|
||||
"--goto",
|
||||
"-p",
|
||||
"--protocol",
|
||||
"-s",
|
||||
"--source",
|
||||
"-d",
|
||||
"--destination",
|
||||
"-i",
|
||||
"--in-interface",
|
||||
"-o",
|
||||
"--out-interface",
|
||||
"-f",
|
||||
"--fragment",
|
||||
"-c",
|
||||
"--set-counters",
|
||||
}
|
||||
)
|
||||
#: Modules whose options this reads; any other module is an ``unknown`` match.
|
||||
_KNOWN_MODULES = frozenset(
|
||||
{"tcp", "udp", "sctp", "udplite", "multiport", "state", "conntrack", "comment"}
|
||||
)
|
||||
|
||||
|
||||
def _match(field: str, negated: bool, values: list[str]) -> NetfilterMatchDict:
|
||||
return {"field": field, "op": "!=" if negated else "==", "values": values}
|
||||
|
||||
|
||||
def _unknown(text: str) -> NetfilterMatchDict:
|
||||
return {"field": "unknown", "op": "==", "values": [text]}
|
||||
|
||||
|
||||
def _iface(name: str) -> str:
|
||||
"""iptables writes a wildcard interface as ``eth+``; nft as ``eth*``."""
|
||||
return name[:-1] + "*" if name.endswith("+") else name
|
||||
|
||||
|
||||
def _ports(spec: str) -> list[str]:
|
||||
return [p.replace(":", "-") for p in spec.split(",") if p]
|
||||
|
||||
|
||||
def _states(spec: str) -> list[str]:
|
||||
return [s.lower() for s in spec.split(",") if s]
|
||||
|
||||
|
||||
class _RuleTokens:
|
||||
"""The tokens of one ``-A`` line, read front to back."""
|
||||
|
||||
def __init__(self, tokens: list[str]):
|
||||
self.tokens = tokens
|
||||
self.i = 0
|
||||
|
||||
def more(self) -> bool:
|
||||
return self.i < len(self.tokens)
|
||||
|
||||
def peek(self, ahead: int = 0) -> str | None:
|
||||
j = self.i + ahead
|
||||
return self.tokens[j] if j < len(self.tokens) else None
|
||||
|
||||
def take(self) -> str:
|
||||
token = self.tokens[self.i]
|
||||
self.i += 1
|
||||
return token
|
||||
|
||||
|
||||
def _module_options(module: str, rt: _RuleTokens, matches: list[NetfilterMatchDict]) -> None:
|
||||
"""Read *module*'s options from *rt* into *matches*; stop at the next top-level option."""
|
||||
text = ["-m", module]
|
||||
unknown = module not in _KNOWN_MODULES
|
||||
while rt.more():
|
||||
token = rt.peek()
|
||||
if token in _TOP_LEVEL or (token == _NEGATION and rt.peek(1) in _TOP_LEVEL):
|
||||
break
|
||||
negated = False
|
||||
if token == _NEGATION:
|
||||
rt.take()
|
||||
negated = True
|
||||
text.append(_NEGATION)
|
||||
option = rt.take()
|
||||
value = rt.take() if rt.more() and not str(rt.peek()).startswith("-") else ""
|
||||
text += [option] + ([value] if value else [])
|
||||
if unknown:
|
||||
continue
|
||||
mapped = _module_match(module, option, value, negated)
|
||||
if mapped is None:
|
||||
unknown = True
|
||||
elif mapped is not False:
|
||||
matches.append(mapped)
|
||||
if unknown and module != "comment":
|
||||
matches.append(_unknown(" ".join(text)))
|
||||
|
||||
|
||||
def _module_match(module: str, option: str, value: str, negated: bool) -> Any:
|
||||
"""The match one module option makes.
|
||||
|
||||
False for one that matches nothing new; None if not read.
|
||||
"""
|
||||
if module == "comment":
|
||||
return False
|
||||
if module in ("tcp", "udp", "sctp", "udplite"):
|
||||
if option in ("--dport", "--destination-port"):
|
||||
return _match("dport", negated, _ports(value))
|
||||
if option in ("--sport", "--source-port"):
|
||||
return _match("sport", negated, _ports(value))
|
||||
return None
|
||||
if module == "multiport":
|
||||
if option in ("--dports", "--destination-ports"):
|
||||
return _match("dport", negated, _ports(value))
|
||||
if option in ("--sports", "--source-ports"):
|
||||
return _match("sport", negated, _ports(value))
|
||||
return None
|
||||
if module == "state" and option == "--state":
|
||||
return _match("ct_state", negated, _states(value))
|
||||
if module == "conntrack":
|
||||
if option == "--ctstate":
|
||||
return _match("ct_state", negated, _states(value))
|
||||
if option == "--ctorigdstport":
|
||||
return _match("ct_orig_dport", negated, _ports(value))
|
||||
return None
|
||||
|
||||
|
||||
def _target(name: str, chains: set[str]) -> tuple[str, str | None]:
|
||||
if name in _TERMINAL:
|
||||
return _TERMINAL[name], None
|
||||
if name in chains:
|
||||
return "jump", name
|
||||
if name in _NON_TERMINAL:
|
||||
return "continue", name
|
||||
return "other", name
|
||||
|
||||
|
||||
def _iptables_rule(line: str, chains: set[str]) -> tuple[str, NetfilterRuleDict]:
|
||||
"""``-A CHAIN ...`` as ``(chain, rule)``."""
|
||||
tokens = shlex.split(line)
|
||||
chain = tokens[1]
|
||||
rt = _RuleTokens(tokens[2:])
|
||||
matches: list[NetfilterMatchDict] = []
|
||||
verdict, target = "continue", None
|
||||
negated = False
|
||||
simple = {
|
||||
"-i": "iifname",
|
||||
"--in-interface": "iifname",
|
||||
"-o": "oifname",
|
||||
"--out-interface": "oifname",
|
||||
"-s": "saddr",
|
||||
"--source": "saddr",
|
||||
"-d": "daddr",
|
||||
"--destination": "daddr",
|
||||
"-p": "l4proto",
|
||||
"--protocol": "l4proto",
|
||||
}
|
||||
while rt.more():
|
||||
token = rt.take()
|
||||
if token == _NEGATION:
|
||||
negated = True
|
||||
continue
|
||||
if token in simple:
|
||||
field, value = simple[token], rt.take()
|
||||
if field in ("iifname", "oifname"):
|
||||
matches.append(_match(field, negated, [_iface(value)]))
|
||||
elif field == "l4proto":
|
||||
if value.lower() != "all":
|
||||
matches.append(_match(field, negated, [value.lower()]))
|
||||
else:
|
||||
matches.append(_match(field, negated, value.split(",")))
|
||||
elif token in ("-m", "--match"):
|
||||
_module_options(rt.take(), rt, matches)
|
||||
elif token in ("-j", "--jump"):
|
||||
verdict, target = _target(rt.take(), chains)
|
||||
break # the target's own options follow and match nothing
|
||||
elif token in ("-g", "--goto"):
|
||||
verdict, target = "goto", rt.take()
|
||||
break
|
||||
elif token in ("-c", "--set-counters"):
|
||||
rt.take(), rt.take()
|
||||
else:
|
||||
matches.append(_unknown(token))
|
||||
negated = False
|
||||
return chain, {"matches": matches, "verdict": verdict, "target": target, "text": line}
|
||||
|
||||
|
||||
def _iptables_variant(lines: list[str]) -> str | None:
|
||||
"""``iptables-nft`` or ``iptables-legacy`` from the header; None if nothing was saved."""
|
||||
for line in lines:
|
||||
if line.startswith("# Generated by"):
|
||||
return "iptables-nft" if "(nf_tables)" in line else "iptables-legacy"
|
||||
return None
|
||||
|
||||
|
||||
def _parse_iptables_save(lines: list[str], family: str) -> list[NetfilterChainDict]:
|
||||
chains: list[NetfilterChainDict] = []
|
||||
table: str | None = None
|
||||
by_name: dict[str, NetfilterChainDict] = {}
|
||||
for raw in lines:
|
||||
line = raw.strip()
|
||||
if line.startswith("*"):
|
||||
table, by_name = line[1:], {}
|
||||
elif line.startswith(":") and table is not None:
|
||||
name, policy = line[1:].split()[:2]
|
||||
kind, hook, prio = _BUILT_IN.get(table, {}).get(name, (None, None, None))
|
||||
chain: NetfilterChainDict = {
|
||||
"family": family,
|
||||
"table": table,
|
||||
"name": name,
|
||||
"type": kind,
|
||||
"hook": hook,
|
||||
"priority": prio,
|
||||
"policy": None if policy == "-" else policy.lower(),
|
||||
"rules": [],
|
||||
}
|
||||
by_name[name] = chain
|
||||
chains.append(chain)
|
||||
elif line.startswith("-A ") and table is not None:
|
||||
name, rule = _iptables_rule(line, set(by_name))
|
||||
if name in by_name:
|
||||
by_name[name]["rules"].append(rule)
|
||||
return chains
|
||||
|
||||
|
||||
# ── nft -j ───────────────────────────────────────────────────────────────────
|
||||
|
||||
_VERDICTS = {
|
||||
"accept": "accept",
|
||||
"drop": "drop",
|
||||
"reject": "reject",
|
||||
"return": "return",
|
||||
"continue": "continue",
|
||||
}
|
||||
#: Statements that act on a packet without deciding its fate.
|
||||
_NO_OPS = frozenset(
|
||||
{
|
||||
"counter",
|
||||
"log",
|
||||
"mangle",
|
||||
"notrack",
|
||||
"set",
|
||||
"flow",
|
||||
"ct helper",
|
||||
"synproxy",
|
||||
"tcp option",
|
||||
"dup",
|
||||
"fwd",
|
||||
}
|
||||
)
|
||||
#: Statements that pass only some packets on: what follows is not about every packet.
|
||||
_PARTIAL = frozenset({"limit", "quota", "numgen", "hash", "ct count"})
|
||||
#: Verdicts that hand the packet elsewhere, or that this does not model.
|
||||
_OTHER = frozenset({"queue", "dnat", "snat", "masquerade", "redirect", "tproxy", "vmap"})
|
||||
_TRANSPORTS = frozenset({"tcp", "udp", "sctp", "udplite", "dccp"})
|
||||
|
||||
|
||||
def _nft_scalar(value: Any) -> list[str] | None:
|
||||
"""A right-hand side as values; None for one this does not model."""
|
||||
if isinstance(value, bool):
|
||||
return None
|
||||
if isinstance(value, (int, str)):
|
||||
return [str(value)]
|
||||
if isinstance(value, dict):
|
||||
if "prefix" in value:
|
||||
return [f"{value['prefix']['addr']}/{value['prefix']['len']}"]
|
||||
if "range" in value:
|
||||
low, high = value["range"]
|
||||
return [f"{low}-{high}"]
|
||||
if "elem" in value:
|
||||
return _nft_scalar(value["elem"].get("val"))
|
||||
if "set" in value:
|
||||
return _nft_values(value["set"])
|
||||
if isinstance(value, list):
|
||||
return _nft_values(value)
|
||||
return None
|
||||
|
||||
|
||||
def _nft_values(items: Any) -> list[str] | None:
|
||||
items = items if isinstance(items, list) else [items]
|
||||
out: list[str] = []
|
||||
for item in items:
|
||||
values = _nft_scalar(item)
|
||||
if values is None:
|
||||
return None
|
||||
out += values
|
||||
return out
|
||||
|
||||
|
||||
def _left_text(left: Any) -> str:
|
||||
if isinstance(left, dict):
|
||||
if "payload" in left:
|
||||
return (
|
||||
f"{left['payload'].get('protocol', '')} {left['payload'].get('field', '')}".strip()
|
||||
)
|
||||
if "meta" in left:
|
||||
return f"meta {left['meta'].get('key', '')}"
|
||||
if "ct" in left:
|
||||
ct = left["ct"]
|
||||
return f"ct {ct.get('dir', '')} {ct.get('key', '')}".replace(" ", " ")
|
||||
key = next(iter(left), "")
|
||||
return str(key)
|
||||
return str(left)
|
||||
|
||||
|
||||
def _right_text(right: Any) -> str:
|
||||
values = _nft_scalar(right)
|
||||
if values is None:
|
||||
return json.dumps(right, sort_keys=True)
|
||||
return values[0] if len(values) == 1 else "{" + ", ".join(values) + "}"
|
||||
|
||||
|
||||
def _nft_field(left: Any) -> tuple[str | None, str | None]:
|
||||
"""``(field, implied l4proto)`` of a left-hand side; ``(None, None)`` if not modelled."""
|
||||
if not isinstance(left, dict):
|
||||
return None, None
|
||||
if "meta" in left:
|
||||
key = left["meta"].get("key")
|
||||
return {
|
||||
"iifname": "iifname",
|
||||
"iif": "iifname",
|
||||
"oifname": "oifname",
|
||||
"oif": "oifname",
|
||||
"l4proto": "l4proto",
|
||||
"nfproto": "nfproto",
|
||||
}.get(key), None
|
||||
if "payload" in left:
|
||||
proto, field = left["payload"].get("protocol"), left["payload"].get("field")
|
||||
if proto in ("ip", "ip6") and field in ("saddr", "daddr"):
|
||||
return field, None
|
||||
if (proto, field) in (("ip", "protocol"), ("ip6", "nexthdr")):
|
||||
return "l4proto", None
|
||||
if field in ("dport", "sport") and (proto in _TRANSPORTS or proto == "th"):
|
||||
return field, proto if proto in _TRANSPORTS else None
|
||||
return None, None
|
||||
if "ct" in left:
|
||||
key, direction = left["ct"].get("key"), left["ct"].get("dir")
|
||||
if key == "state":
|
||||
return "ct_state", None
|
||||
if key == "proto-dst" and direction == "original":
|
||||
return "ct_orig_dport", None
|
||||
return None, None
|
||||
|
||||
|
||||
def _nft_match(
|
||||
match: dict[str, Any], sets: dict[tuple[str, str, str], list[str]], where: tuple[str, str]
|
||||
) -> list[NetfilterMatchDict]:
|
||||
left, op, right = match.get("left"), match.get("op"), match.get("right")
|
||||
field, implied = _nft_field(left)
|
||||
raw_text = (
|
||||
f"{_left_text(left)} {'' if op in ('==', 'in') else str(op) + ' '}{_right_text(right)}"
|
||||
)
|
||||
if field is None or op not in ("==", "in", "!="):
|
||||
return [_unknown(raw_text)]
|
||||
if isinstance(right, str) and right.startswith("@"):
|
||||
values = sets.get((where[0], where[1], right[1:]))
|
||||
else:
|
||||
values = _nft_scalar(right)
|
||||
if values is None:
|
||||
return [_unknown(raw_text)]
|
||||
out: list[NetfilterMatchDict] = []
|
||||
if implied:
|
||||
out.append(_match("l4proto", False, [implied]))
|
||||
if field in ("iifname", "oifname", "l4proto", "ct_state", "nfproto"):
|
||||
values = [
|
||||
str(v).lower() if field != "iifname" and field != "oifname" else str(v) for v in values
|
||||
]
|
||||
out.append(_match(field, op == "!=", values))
|
||||
return out
|
||||
|
||||
|
||||
def _nft_rule(
|
||||
rule: dict[str, Any], sets: dict[tuple[str, str, str], list[str]]
|
||||
) -> NetfilterRuleDict:
|
||||
where = (rule.get("family", ""), rule.get("table", ""))
|
||||
matches: list[NetfilterMatchDict] = []
|
||||
verdict, target = "continue", None
|
||||
for statement in rule.get("expr") or []:
|
||||
if not isinstance(statement, dict) or not statement:
|
||||
continue
|
||||
key = next(iter(statement))
|
||||
body = statement[key]
|
||||
if key == "match":
|
||||
matches += _nft_match(body, sets, where)
|
||||
elif key in _VERDICTS:
|
||||
verdict = _VERDICTS[key]
|
||||
break
|
||||
elif key in ("jump", "goto"):
|
||||
verdict, target = key, str((body or {}).get("target"))
|
||||
break
|
||||
elif key == "xt":
|
||||
if (body or {}).get("type") == "target":
|
||||
verdict, target = "other", str(body.get("name"))
|
||||
break
|
||||
matches.append(_unknown(f"xt match {(body or {}).get('name')}"))
|
||||
elif key in _PARTIAL:
|
||||
matches.append(_unknown(key))
|
||||
elif key in _OTHER:
|
||||
verdict, target = "other", key
|
||||
break
|
||||
elif key not in _NO_OPS:
|
||||
matches.append(_unknown(key))
|
||||
entry: NetfilterRuleDict = {
|
||||
"matches": matches,
|
||||
"verdict": verdict,
|
||||
"target": target,
|
||||
"text": "",
|
||||
}
|
||||
entry["text"] = rule_text(entry)
|
||||
return entry
|
||||
|
||||
|
||||
def _parse_nft(text: str, skip: set[tuple[str, str]]) -> list[NetfilterChainDict]:
|
||||
start = text.find("{")
|
||||
if start < 0:
|
||||
raise ValueError("nft printed no JSON")
|
||||
objects = json.loads(text[start:]).get("nftables") or []
|
||||
sets: dict[tuple[str, str, str], list[str]] = {}
|
||||
for obj in objects:
|
||||
if "set" in obj and "elem" in obj["set"]:
|
||||
values = _nft_values(obj["set"]["elem"])
|
||||
if values is not None:
|
||||
s = obj["set"]
|
||||
sets[(s.get("family", ""), s.get("table", ""), s.get("name", ""))] = values
|
||||
chains: dict[tuple[str, str, str], NetfilterChainDict] = {}
|
||||
for obj in objects:
|
||||
if "chain" not in obj:
|
||||
continue
|
||||
c = obj["chain"]
|
||||
if (c.get("family"), c.get("table")) in skip:
|
||||
continue
|
||||
chains[(c["family"], c["table"], c["name"])] = {
|
||||
"family": c["family"],
|
||||
"table": c["table"],
|
||||
"name": c["name"],
|
||||
"type": c.get("type"),
|
||||
"hook": c.get("hook"),
|
||||
"priority": c.get("prio"),
|
||||
"policy": c.get("policy") if c.get("hook") else None,
|
||||
"rules": [],
|
||||
}
|
||||
for obj in objects:
|
||||
if "rule" not in obj:
|
||||
continue
|
||||
r = obj["rule"]
|
||||
chain = chains.get((r.get("family"), r.get("table"), r.get("chain")))
|
||||
if chain is not None:
|
||||
chain["rules"].append(_nft_rule(r, sets))
|
||||
return list(chains.values())
|
||||
|
||||
|
||||
# ── text ─────────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _family_of(values: list[str]) -> str:
|
||||
try:
|
||||
return (
|
||||
"ip6"
|
||||
if all(ipaddress.ip_network(v, strict=False).version == 6 for v in values)
|
||||
else "ip"
|
||||
)
|
||||
except ValueError:
|
||||
return "ip"
|
||||
|
||||
|
||||
def _values_text(values: list[str]) -> str:
|
||||
return values[0] if len(values) == 1 else "{" + ", ".join(values) + "}"
|
||||
|
||||
|
||||
def rule_text(rule: NetfilterRuleDict) -> str:
|
||||
"""A rule as one nft-style line, from its normalized matches and verdict."""
|
||||
protos = [
|
||||
v
|
||||
for m in rule["matches"]
|
||||
if m["field"] == "l4proto" and m["op"] == "=="
|
||||
for v in m["values"]
|
||||
]
|
||||
parts: list[str] = []
|
||||
for m in rule["matches"]:
|
||||
neg = "!= " if m["op"] == "!=" else ""
|
||||
field, values = m["field"], m["values"]
|
||||
if field == "unknown":
|
||||
parts.append(" ".join(values))
|
||||
continue
|
||||
keyword = {
|
||||
"saddr": f"{_family_of(values)} saddr",
|
||||
"daddr": f"{_family_of(values)} daddr",
|
||||
"l4proto": "meta l4proto",
|
||||
"nfproto": "meta nfproto",
|
||||
"ct_state": "ct state",
|
||||
"ct_orig_dport": "ct original proto-dst",
|
||||
"dport": f"{protos[0] if len(protos) == 1 else 'th'} dport",
|
||||
"sport": f"{protos[0] if len(protos) == 1 else 'th'} sport",
|
||||
}.get(field, field)
|
||||
parts.append(f"{keyword} {neg}{_values_text(values)}")
|
||||
verdict = rule["verdict"]
|
||||
if verdict in ("jump", "goto"):
|
||||
parts.append(f"{verdict} {rule['target']}")
|
||||
elif verdict == "other":
|
||||
parts.append(str(rule["target"] or "other"))
|
||||
elif verdict != "continue":
|
||||
parts.append(verdict)
|
||||
return " ".join(parts)
|
||||
|
||||
|
||||
# ── the reading ──────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _iptables_sections(
|
||||
sections: dict[str, tuple[list[str], int | None]],
|
||||
) -> Iterator[tuple[str, str, list[str]]]:
|
||||
"""``(variant, family, lines)`` for each iptables tool that saved something."""
|
||||
for tool, family in _IPTABLES_FAMILY.items():
|
||||
lines, rc = sections.get(tool, ([], None))
|
||||
if rc != 0:
|
||||
continue
|
||||
variant = _iptables_variant(lines)
|
||||
if variant is not None:
|
||||
yield variant, family, lines
|
||||
|
||||
|
||||
def parse_host_firewall(output: str) -> HostFirewallDict:
|
||||
"""The framed report of :data:`HOST_FIREWALL_COMMAND` as one ruleset.
|
||||
|
||||
:raises ValueError: if the report's frame is not intact.
|
||||
:raises HostFirewallUnavailable: if neither tool could be read.
|
||||
"""
|
||||
sections = _sections(_frame(output))
|
||||
sources: list[str] = []
|
||||
chains: list[NetfilterChainDict] = []
|
||||
covered: set[tuple[str, str]] = set()
|
||||
for variant, family, lines in _iptables_sections(sections):
|
||||
saved = _parse_iptables_save(lines, family)
|
||||
chains += saved
|
||||
if variant not in sources:
|
||||
sources.append(variant)
|
||||
if variant == "iptables-nft":
|
||||
covered |= {(c["family"], c["table"]) for c in saved}
|
||||
nft_lines, nft_rc = sections.get("nft", ([], None))
|
||||
if nft_rc == 0:
|
||||
try:
|
||||
nft_chains = _parse_nft("\n".join(nft_lines), covered)
|
||||
except ValueError:
|
||||
nft_chains = None
|
||||
if nft_chains is not None:
|
||||
sources.insert(0, "nftables")
|
||||
chains = nft_chains + chains
|
||||
if not sources:
|
||||
raise HostFirewallUnavailable("neither nft nor iptables-save could be read (root needed)")
|
||||
return {"sources": sources, "chains": chains}
|
||||
|
||||
|
||||
class HostFirewallMixin:
|
||||
"""Adds :meth:`get_host_firewall` to a driver that can run a command on a Linux host.
|
||||
|
||||
The template form (README, "Function classes"): the command and its parse
|
||||
are the same everywhere, so they are concrete here, and a driver supplies
|
||||
only :meth:`_run_host_firewall_command` -- how a command reaches its host,
|
||||
and how it gains root there. Mixed in by the drivers that can, so
|
||||
``hasattr(driver, "get_host_firewall")`` stays truthful.
|
||||
"""
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def _run_host_firewall_command(self, command: str, *, privileged: bool) -> str:
|
||||
"""Run *command* on the host and return what it printed; as root
|
||||
when *privileged*. The command is a single ``sh -c`` invocation."""
|
||||
...
|
||||
|
||||
def get_host_firewall(self) -> HostFirewallDict:
|
||||
"""
|
||||
Returns the host's netfilter rules: every chain with its hook, priority
|
||||
and policy, and every rule as matches and a verdict.
|
||||
|
||||
* sources (list) - ``nftables``, ``iptables-nft``, ``iptables-legacy``: what was read
|
||||
* chains (list) - see :class:`~napalm_device_types.models.NetfilterChainDict`
|
||||
|
||||
Example::
|
||||
|
||||
{
|
||||
"sources": ["nftables", "iptables-nft"],
|
||||
"chains": [
|
||||
{"family": "ip", "table": "filter", "name": "INPUT", "type": "filter",
|
||||
"hook": "input", "priority": 0, "policy": "accept",
|
||||
"rules": [
|
||||
{"matches": [{"field": "iifname", "op": "==", "values": ["ens6"]}],
|
||||
"verdict": "drop", "target": None,
|
||||
"text": "-A INPUT -i ens6 -j DROP"},
|
||||
]},
|
||||
],
|
||||
}
|
||||
|
||||
:raises HostFirewallUnavailable: if neither tool could be read -- no
|
||||
``nft``, no ``iptables-save``, or no root to run them.
|
||||
:raises ValueError: if the report came back cut short.
|
||||
"""
|
||||
command = f"sh -c {quote(HOST_FIREWALL_COMMAND)}"
|
||||
return parse_host_firewall(self._run_host_firewall_command(command, privileged=True))
|
||||
@@ -0,0 +1,32 @@
|
||||
"""Shutting the device itself down cleanly.
|
||||
|
||||
Declared under ``if TYPE_CHECKING``: a contract, not a placeholder. Only a
|
||||
driver that can shut its device down through the device's own means defines
|
||||
``shutdown_host``, so ``hasattr(driver, "shutdown_host")`` tells a caller
|
||||
whether to offer it. There is no forced variant here, by design: cutting the
|
||||
power is the BMC's business, not a driver's.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
|
||||
class HostShutdownMixin:
|
||||
if TYPE_CHECKING:
|
||||
|
||||
def shutdown_host(self) -> None:
|
||||
"""
|
||||
Shuts down the device this driver is connected to, cleanly.
|
||||
|
||||
Returns once the device has accepted the request; the session is
|
||||
usually gone right after. Whether the device really went off is
|
||||
the caller's business, e.g. through its BMC.
|
||||
|
||||
A driver that manages other machines shuts down its *own* host,
|
||||
never one of them: a hypervisor node shuts down itself, and its
|
||||
guests are stopped the way the node stops them on shutdown.
|
||||
|
||||
:raises RuntimeError: If the device refuses the request.
|
||||
"""
|
||||
...
|
||||
@@ -0,0 +1,365 @@
|
||||
"""Host status: does the host need a reboot, and does it patch itself?
|
||||
|
||||
A patch run that installed a new kernel or libc has not closed anything until
|
||||
the host restarts, so "reboot required" is part of being patched. Whether the
|
||||
host installs updates on its own (unattended-upgrades, dnf-automatic) decides
|
||||
how far netOrk's maintenance window reaches. Both are read the same way on every
|
||||
Linux host, so the command and its parse live here once and a driver only
|
||||
carries the command across.
|
||||
|
||||
**Reboot required** is any of:
|
||||
|
||||
- ``/var/run/reboot-required`` exists. Ubuntu always writes it; Debian does when
|
||||
update-notifier or unattended-upgrades is installed.
|
||||
- ``needs-restarting -r`` exits 1 (dnf-utils).
|
||||
- A kernel newer than the running one is installed, of the same flavour. A
|
||||
Raspberry Pi carries ``rpi-v8`` and ``rpi-2712`` builds side by side, and only
|
||||
the running one's counts.
|
||||
- On FreeBSD, ``freebsd-version -k`` (the installed kernel) differs from
|
||||
``freebsd-version -r`` (the running one): a patch replaced the kernel.
|
||||
|
||||
It is ``None`` when none of these could be read, for example in a container
|
||||
without a ``/lib/modules`` of its own.
|
||||
|
||||
**Auto updates** is apt's ``APT::Periodic::Unattended-Upgrade`` (set, not "0",
|
||||
``apt-daily-upgrade.timer`` not disabled, and unattended-upgrades installed: a
|
||||
package removed but not purged leaves its "1" behind) or an enabled
|
||||
``dnf-automatic-install.timer``. ``dnf-automatic.timer`` installs only with
|
||||
``apply_updates = yes`` in ``automatic.conf`` (dnf5's in ``dnf5-plugins/``
|
||||
wins); by default it only downloads. It is ``None`` on a host with neither apt
|
||||
nor dnf-automatic. When they last ran
|
||||
successfully is ``/var/lib/apt/periodic/upgrade-stamp``, which
|
||||
``apt.systemd.daily`` touches only after unattended-upgrade succeeded.
|
||||
``update-success-stamp`` would say nothing: every ``apt-get update`` touches it,
|
||||
whoever runs it. How often they are meant to run is the same setting, in apt's
|
||||
units (a bare number is days).
|
||||
|
||||
**An interrupted package manager** is dpkg left halfway, the way apt checks it:
|
||||
numbered files left in ``/var/lib/dpkg/updates``, or a package in a state only
|
||||
a run cut off leaves behind (half-installed, unpacked, half-configured, waiting
|
||||
for triggers, or flagged for reinstallation). apt then refuses to install
|
||||
anything until ``dpkg --configure -a`` has run, while ``apt.systemd.daily``
|
||||
exits quietly every day (NetOrk/netork#738). While dpkg, apt, aptitude or
|
||||
unattended-upgrade is running, packages are halfway on purpose -- also between
|
||||
two of apt's dpkg calls -- so the answer is then ``None``, as on a host without
|
||||
dpkg. unattended-upgrade is told from Ubuntu's idle shutdown helper by its
|
||||
command line (:data:`BUSY_CHECK`).
|
||||
|
||||
**Filesystems** are how full ``/``, ``/boot`` and ``/var`` are, from POSIX
|
||||
``df -Pk``: where an upgrade unpacks a kernel, builds its initrd and keeps the
|
||||
package files. A kernel upgrade that ran out of room there left dpkg halfway on
|
||||
a Proxmox host whose root had been 94 % full for a week (NetOrk/netork#862). A
|
||||
directory that is not a mount of its own is reported as the filesystem it lives
|
||||
on, so each filesystem appears once. ``available`` is what df says an ordinary
|
||||
user can still write, which on ext4 is less than total minus used. How full is
|
||||
too full is not decided here.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from napalm_device_types.models import (
|
||||
FilesystemUsageDict,
|
||||
HostStatusDict,
|
||||
InterruptedPackageDict,
|
||||
)
|
||||
from napalm_device_types.terminal import strip_terminal_codes
|
||||
|
||||
_BEGIN = "HSTAT_BEGIN"
|
||||
_END = "HSTAT_END"
|
||||
_REBOOT_FILE = "/var/run/reboot-required"
|
||||
_APT_TIMER = "apt-daily-upgrade.timer"
|
||||
_DNF_TIMER = "dnf-automatic.timer"
|
||||
_DNF_INSTALL_TIMER = "dnf-automatic-install.timer"
|
||||
_DNF_TIMERS = (_DNF_TIMER, _DNF_INSTALL_TIMER)
|
||||
#: dnf4's, then dnf5's: the last ``apply_updates`` printed is the one that holds.
|
||||
_DNF_AUTOMATIC_CONF = "/etc/dnf/automatic.conf /etc/dnf/dnf5-plugins/automatic.conf"
|
||||
_UPGRADE_STAMP = "/var/lib/apt/periodic/upgrade-stamp"
|
||||
_DPKG_JOURNAL = "/var/lib/dpkg/updates"
|
||||
#: Where an upgrade writes: the kernel's modules, its initrd, the package files.
|
||||
_FILESYSTEMS = "/ /boot /var"
|
||||
#: Prints ``[dpkg-busy]`` while a process that leaves packages halfway is working:
|
||||
#: dpkg, apt, apt-get or aptitude by name, unattended-upgrade by its command line.
|
||||
#: Its name is cut to 15 characters, ``unattended-upgr``, the same as that of
|
||||
#: ``unattended-upgrade-shutdown --wait-for-signal``, which Ubuntu keeps running all
|
||||
#: the time and which installs nothing.
|
||||
BUSY_CHECK = (
|
||||
"{ ps -e -o comm= 2>/dev/null | grep -qxE 'dpkg|apt|apt-get|aptitude' || "
|
||||
"ps -e -o args= 2>/dev/null | grep -qE '(^|[ /])unattended-upgrade( |$)'; } "
|
||||
"&& echo '[dpkg-busy]'; "
|
||||
)
|
||||
|
||||
#: One line, POSIX ``sh``, read-only, no privileges. The frame markers are
|
||||
#: printed in two halves so that an echoing transport does not show them early.
|
||||
#: Each timer is asked on its own: older systemd prints nothing for an unknown
|
||||
#: unit, which would shift a combined answer. Run through a pipe, so nothing in
|
||||
#: it sees a terminal and colours its output.
|
||||
HOST_STATUS_COMMAND = (
|
||||
"{ printf '%s%s\\n' HSTAT_ BEGIN; "
|
||||
f"[ -f {_REBOOT_FILE} ] && echo '[reboot-required]'; "
|
||||
"if command -v needs-restarting >/dev/null 2>&1; then echo '[needs-restarting]'; "
|
||||
"needs-restarting -r >/dev/null 2>&1; echo $?; fi; "
|
||||
"echo '[kernel]'; uname -r; echo '[modules]'; ls -1 /lib/modules 2>/dev/null; "
|
||||
"if command -v freebsd-version >/dev/null 2>&1; then "
|
||||
"echo '[freebsd-kernel]'; freebsd-version -k; "
|
||||
"echo '[freebsd-running]'; freebsd-version -r; fi; "
|
||||
"if command -v apt-config >/dev/null 2>&1; then echo '[apt-config]'; "
|
||||
"apt-config dump 2>/dev/null | grep '^APT::Periodic::Unattended-Upgrade '; "
|
||||
# Named through a variable: spelt out, the program name would match BUSY_CHECK
|
||||
# in this very command line.
|
||||
'uu=unattended-upgrade; { command -v "$uu" || [ -x "/usr/bin/$uu" ]; } >/dev/null 2>&1 '
|
||||
"&& echo '[unattended-upgrade]'; "
|
||||
f"echo '[upgrade-stamp]'; stat -c %Y {_UPGRADE_STAMP} 2>/dev/null; fi; "
|
||||
"if command -v dpkg-query >/dev/null 2>&1; then "
|
||||
f"{BUSY_CHECK}"
|
||||
f"echo '[dpkg-journal]'; ls -1 {_DPKG_JOURNAL} 2>/dev/null | head -n 20; "
|
||||
"echo '[dpkg-audit]'; dpkg-query -W -f='${db:Status-Abbrev} ${Package}\\n' 2>/dev/null "
|
||||
"| awk 'substr($0, 2, 1) ~ /[HUFWt]/ || substr($0, 3, 1) == \"R\"' | head -n 50; fi; "
|
||||
f"echo '[filesystems]'; df -Pk {_FILESYSTEMS}; "
|
||||
f"echo '[timers]'; for u in {_APT_TIMER} {' '.join(_DNF_TIMERS)}; do "
|
||||
'printf \'%s %s\\n\' "$u" "$(systemctl is-enabled "$u" 2>/dev/null)"; done; '
|
||||
"echo '[dnf-automatic]'; grep -h '^[[:space:]]*apply_updates[[:space:]]*=' "
|
||||
f"{_DNF_AUTOMATIC_CONF} 2>/dev/null; "
|
||||
"printf '%s%s\\n' HSTAT_ END; } 2>/dev/null | cat"
|
||||
)
|
||||
|
||||
_PERIODIC = re.compile(r'^APT::Periodic::Unattended-Upgrade\s+"([^"]*)"')
|
||||
_APPLY_UPDATES = re.compile(r"^apply_updates\s*=\s*(\S*)")
|
||||
#: What libdnf takes for a true boolean.
|
||||
_TRUE = frozenset({"1", "yes", "true", "on"})
|
||||
_OFF_STATES = frozenset({"disabled", "masked"})
|
||||
_INTERVAL = re.compile(r"(\d+)([smhd]?)")
|
||||
_UNIT_SECONDS = {"s": 1, "m": 60, "h": 3600, "d": 86400, "": 86400}
|
||||
_JOURNAL_ENTRY = re.compile(r"\d+")
|
||||
#: One line of ``df -P``: device, 1024-blocks, used, available, capacity, mountpoint.
|
||||
#: The device and the mountpoint may hold blanks; the four numbers between them do not.
|
||||
_DF_LINE = re.compile(r"^.+?\s+(\d+)\s+(\d+)\s+(\d+)\s+(?:\d+%|-)\s+(/.*)$")
|
||||
#: dpkg's second status letter -> its word, for the states a cut-off run leaves.
|
||||
_HALFWAY = {
|
||||
"H": "half-installed",
|
||||
"U": "unpacked",
|
||||
"F": "half-configured",
|
||||
"W": "triggers-awaited",
|
||||
"t": "triggers-pending",
|
||||
}
|
||||
|
||||
|
||||
def _sections(output: str) -> dict[str, list[str]]:
|
||||
lines = [line.strip() for line in strip_terminal_codes(output).splitlines()]
|
||||
try:
|
||||
start = lines.index(_BEGIN)
|
||||
end = lines.index(_END, start)
|
||||
except ValueError:
|
||||
raise ValueError("no intact host status report in the output") from None
|
||||
sections: dict[str, list[str]] = {}
|
||||
current: list[str] = []
|
||||
for line in lines[start + 1 : end]:
|
||||
if line.startswith("[") and line.endswith("]"):
|
||||
current = sections.setdefault(line[1:-1], [])
|
||||
elif line:
|
||||
current.append(line)
|
||||
return sections
|
||||
|
||||
|
||||
def _version_key(version: str) -> tuple[object, ...]:
|
||||
"""Natural order: 6.8.0-142 after 6.8.0-87, 7.0.14 after 7.0.2."""
|
||||
return tuple(int(part) if part.isdigit() else part for part in re.split(r"(\d+)", version))
|
||||
|
||||
|
||||
def kernel_reboot_pending(running: str, installed: list[str]) -> str | None:
|
||||
"""The newest installed kernel of the running flavour, if it is newer than the
|
||||
running one; otherwise None.
|
||||
|
||||
The flavour is what follows the last ``-`` (``generic``, ``amd64``, ``pve``,
|
||||
``v8``), or on an RPM kernel (``5.14.0-427.13.1.el9_4.x86_64``) the arch
|
||||
after the release; a kernel of another flavour is never a reason to reboot.
|
||||
"""
|
||||
version, flavour = _kernel_flavour(running)
|
||||
same = [k for k in installed if _kernel_flavour(k)[1] == flavour]
|
||||
if not same:
|
||||
return None
|
||||
newest = max(same, key=lambda k: _version_key(_kernel_flavour(k)[0]))
|
||||
if _version_key(_kernel_flavour(newest)[0]) > _version_key(version):
|
||||
return newest
|
||||
return None
|
||||
|
||||
|
||||
def _kernel_flavour(release: str) -> tuple[str, str]:
|
||||
"""``6.8.0-87-generic`` -> ``("6.8.0-87", "generic")``,
|
||||
``5.14.0-427.13.1.el9_4.x86_64`` -> ``("5.14.0-427.13.1.el9_4", "x86_64")``."""
|
||||
base, sep, tail = release.rpartition("-")
|
||||
if not sep:
|
||||
return release, ""
|
||||
if tail[:1].isdigit() and "." in tail:
|
||||
rel, _, arch = tail.rpartition(".")
|
||||
return f"{base}-{rel}", arch
|
||||
return base, tail
|
||||
|
||||
|
||||
def _freebsd_reboot(sections: dict[str, list[str]]) -> tuple[bool, str | None] | None:
|
||||
"""FreeBSD's answer from freebsd-version, or None on any other system."""
|
||||
installed = (sections.get("freebsd-kernel") or [""])[0]
|
||||
running = (sections.get("freebsd-running") or [""])[0]
|
||||
if not (installed and running):
|
||||
return None
|
||||
if installed != running:
|
||||
return True, f"kernel {installed} installed, {running} running"
|
||||
return False, None
|
||||
|
||||
|
||||
def _reboot(sections: dict[str, list[str]]) -> tuple[bool | None, str | None]:
|
||||
if "reboot-required" in sections:
|
||||
return True, f"{_REBOOT_FILE} is present"
|
||||
needs = sections.get("needs-restarting")
|
||||
if needs and needs[0] == "1":
|
||||
return True, "needs-restarting -r reports a reboot"
|
||||
freebsd = _freebsd_reboot(sections)
|
||||
if freebsd is not None:
|
||||
return freebsd
|
||||
running = (sections.get("kernel") or [""])[0]
|
||||
modules = sections.get("modules") or []
|
||||
newer = kernel_reboot_pending(running, modules) if running and modules else None
|
||||
if newer:
|
||||
return True, f"kernel {newer} installed, {running} running"
|
||||
if needs or modules:
|
||||
return False, None
|
||||
return None, None
|
||||
|
||||
|
||||
def _timer_states(sections: dict[str, list[str]]) -> dict[str, str]:
|
||||
states: dict[str, str] = {}
|
||||
for line in sections.get("timers") or []:
|
||||
unit, _, state = line.partition(" ")
|
||||
states[unit] = state.strip()
|
||||
return states
|
||||
|
||||
|
||||
def _periodic(sections: dict[str, list[str]]) -> str | None:
|
||||
"""The value of ``APT::Periodic::Unattended-Upgrade``, or None when it is not set."""
|
||||
lines = sections.get("apt-config") or []
|
||||
match = next(filter(None, (_PERIODIC.match(line) for line in lines)), None)
|
||||
return match.group(1) if match else None
|
||||
|
||||
|
||||
def _dnf_applies(sections: dict[str, list[str]]) -> bool:
|
||||
"""Whether dnf-automatic.timer installs what it downloads; dnf's default is no."""
|
||||
lines = sections.get("dnf-automatic") or []
|
||||
values = [m.group(1) for m in map(_APPLY_UPDATES.match, lines) if m]
|
||||
return bool(values) and values[-1].lower() in _TRUE
|
||||
|
||||
|
||||
def _auto_updates(sections: dict[str, list[str]]) -> bool | None:
|
||||
timers = _timer_states(sections)
|
||||
if timers.get(_DNF_INSTALL_TIMER) == "enabled":
|
||||
return True
|
||||
downloads_only = timers.get(_DNF_TIMER) == "enabled"
|
||||
if downloads_only and _dnf_applies(sections):
|
||||
return True
|
||||
if "apt-config" not in sections:
|
||||
return False if downloads_only else None
|
||||
switched_on = _periodic(sections) not in (None, "", "0")
|
||||
installed = "unattended-upgrade" in sections
|
||||
return switched_on and installed and timers.get(_APT_TIMER) not in _OFF_STATES
|
||||
|
||||
|
||||
def _interval(sections: dict[str, list[str]]) -> int | None:
|
||||
"""apt's interval in seconds, the way ``apt.systemd.daily`` reads it; 0 for "always"."""
|
||||
value = _periodic(sections)
|
||||
if value == "always":
|
||||
return 0
|
||||
match = _INTERVAL.fullmatch(value or "")
|
||||
if not match or int(match.group(1)) == 0:
|
||||
return None
|
||||
return int(match.group(1)) * _UNIT_SECONDS[match.group(2)]
|
||||
|
||||
|
||||
def _last_success(sections: dict[str, list[str]]) -> int | None:
|
||||
stamp = (sections.get("upgrade-stamp") or [""])[0]
|
||||
return int(stamp) if stamp.isdigit() else None
|
||||
|
||||
|
||||
def _halfway(line: str) -> InterruptedPackageDict | None:
|
||||
status, _, name = line.partition(" ")
|
||||
name = name.strip()
|
||||
state = "reinstall-required" if status[2:3] == "R" else _HALFWAY.get(status[1:2])
|
||||
return {"name": name, "state": state} if name and state else None
|
||||
|
||||
|
||||
def _dpkg(sections: dict[str, list[str]]) -> tuple[bool | None, list[InterruptedPackageDict]]:
|
||||
if "dpkg-audit" not in sections or "dpkg-busy" in sections:
|
||||
return None, []
|
||||
packages = [p for p in map(_halfway, sections["dpkg-audit"]) if p is not None]
|
||||
journal = any(_JOURNAL_ENTRY.fullmatch(f) for f in sections.get("dpkg-journal") or [])
|
||||
return bool(packages) or journal, packages
|
||||
|
||||
|
||||
def _filesystems(sections: dict[str, list[str]]) -> list[FilesystemUsageDict] | None:
|
||||
"""Each filesystem df named, once and in its order; None when it named none."""
|
||||
found: dict[str, FilesystemUsageDict] = {}
|
||||
for line in sections.get("filesystems") or []:
|
||||
match = _DF_LINE.match(line)
|
||||
if not match or int(match.group(1)) == 0:
|
||||
continue
|
||||
total, used, available = (int(match.group(i)) * 1024 for i in (1, 2, 3))
|
||||
mountpoint = match.group(4)
|
||||
found.setdefault(
|
||||
mountpoint,
|
||||
{"mountpoint": mountpoint, "total": total, "used": used, "available": available},
|
||||
)
|
||||
return list(found.values()) or None
|
||||
|
||||
|
||||
def parse_host_status(output: str) -> HostStatusDict:
|
||||
"""Parse what :data:`HOST_STATUS_COMMAND` printed.
|
||||
|
||||
:raises ValueError: when the output carries no intact report.
|
||||
"""
|
||||
sections = _sections(output)
|
||||
required, reason = _reboot(sections)
|
||||
interrupted, packages = _dpkg(sections)
|
||||
return {
|
||||
"reboot_required": required,
|
||||
"reboot_reason": reason,
|
||||
"auto_updates": _auto_updates(sections),
|
||||
"auto_updates_last_success": _last_success(sections),
|
||||
"auto_updates_interval": _interval(sections),
|
||||
"package_manager_interrupted": interrupted,
|
||||
"interrupted_packages": packages,
|
||||
"filesystems": _filesystems(sections),
|
||||
}
|
||||
|
||||
|
||||
class HostStatusMixin:
|
||||
"""Adds :meth:`get_host_status` to a driver that can run a command on a Linux host.
|
||||
|
||||
The template form, like :class:`~napalm_device_types.kernel.KernelFactsMixin`:
|
||||
the reading and its parse are the same everywhere, and a driver supplies only
|
||||
:meth:`_run_host_status_command`. Mixed in by the drivers that can, so
|
||||
``hasattr(driver, "get_host_status")`` stays a truthful answer.
|
||||
"""
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def _run_host_status_command(self, command: str) -> str:
|
||||
"""Run *command* on the host with ``sh`` and return what it printed."""
|
||||
...
|
||||
|
||||
def get_host_status(self) -> HostStatusDict:
|
||||
"""
|
||||
Returns whether the host needs a reboot, whether it patches itself, and
|
||||
whether its package manager was left halfway.
|
||||
|
||||
* reboot_required (bool or None)
|
||||
* reboot_reason (string or None)
|
||||
* auto_updates (bool or None)
|
||||
* auto_updates_last_success (Unix time or None)
|
||||
* auto_updates_interval (seconds or None)
|
||||
* package_manager_interrupted (bool or None)
|
||||
* interrupted_packages (list of {name, state})
|
||||
* filesystems (list of {mountpoint, total, used, available} in bytes, or None)
|
||||
|
||||
:raises ValueError: if the host's output carried no intact report.
|
||||
"""
|
||||
return parse_host_status(self._run_host_status_command(HOST_STATUS_COMMAND))
|
||||
@@ -10,23 +10,28 @@ Usage::
|
||||
...
|
||||
"""
|
||||
|
||||
from typing import Any, Dict, List, TYPE_CHECKING
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from napalm_device_types.base import DeviceTypeDriver
|
||||
from napalm_device_types.packages import PackageManagementMixin
|
||||
from napalm_device_types.clusters import ClusterDict
|
||||
from napalm_device_types.health_metrics import HealthMetricsMixin
|
||||
from napalm_device_types.models import (
|
||||
NICConfigDict,
|
||||
HypervisorHostDict,
|
||||
NetworkTargetDict,
|
||||
NICConfigDict,
|
||||
SnapshotDict,
|
||||
StorageTargetDict,
|
||||
StorageVolumeDict,
|
||||
VirtualNetworkDict,
|
||||
VMConfigDict,
|
||||
VMCpuTypeDict,
|
||||
VMDict,
|
||||
VMProvisionResultDict,
|
||||
VMStatusDict,
|
||||
VirtualNetworkDict,
|
||||
)
|
||||
from napalm_device_types.packages import PackageManagementMixin
|
||||
from napalm_device_types.provisioning import QEMU_GUEST_AGENTS, GuestAgents
|
||||
from napalm_device_types.vm_migration import MigrationPreflightDict
|
||||
|
||||
|
||||
class HypervisorDriver(PackageManagementMixin, HealthMetricsMixin, DeviceTypeDriver):
|
||||
@@ -44,12 +49,11 @@ class HypervisorDriver(PackageManagementMixin, HealthMetricsMixin, DeviceTypeDri
|
||||
ROLE: str = "hypervisor"
|
||||
TYPE_LABEL: str = "Hypervisor"
|
||||
|
||||
#: What cloud-init installs and starts on a VM provisioned through this
|
||||
#: driver, so the hypervisor can read the guest's IP address back.
|
||||
GUEST_AGENT_PACKAGES: tuple[str, ...] = ("qemu-guest-agent",)
|
||||
GUEST_AGENT_RUNCMD: tuple[str, ...] = ("systemctl enable --now qemu-guest-agent",)
|
||||
|
||||
|
||||
#: What cloud-init installs and runs on a VM provisioned through this
|
||||
#: driver, per guest OS, so the hypervisor can read the guest's IP
|
||||
#: address back. The guest operating systems listed here are the ones
|
||||
#: the driver can provision; netOrk reads this off the class.
|
||||
GUEST_AGENTS: GuestAgents = {"linux": QEMU_GUEST_AGENTS["linux"]}
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Virtual machines – read
|
||||
@@ -57,7 +61,7 @@ class HypervisorDriver(PackageManagementMixin, HealthMetricsMixin, DeviceTypeDri
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
||||
def get_vms(self) -> List[VMDict]:
|
||||
def get_vms(self) -> list[VMDict]:
|
||||
"""
|
||||
Returns a list of all virtual machines known to this hypervisor,
|
||||
including their runtime status.
|
||||
@@ -260,7 +264,7 @@ class HypervisorDriver(PackageManagementMixin, HealthMetricsMixin, DeviceTypeDri
|
||||
# Snapshots
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def get_vm_snapshots(self, name: str) -> List[SnapshotDict]:
|
||||
def get_vm_snapshots(self, name: str) -> list[SnapshotDict]:
|
||||
"""
|
||||
Returns all snapshots of a virtual machine.
|
||||
|
||||
@@ -299,8 +303,9 @@ class HypervisorDriver(PackageManagementMixin, HealthMetricsMixin, DeviceTypeDri
|
||||
"""
|
||||
...
|
||||
|
||||
def create_vm_snapshot(self, name: str, snapshot: str,
|
||||
description: str = "", include_memory: bool = False) -> None:
|
||||
def create_vm_snapshot(
|
||||
self, name: str, snapshot: str, description: str = "", include_memory: bool = False
|
||||
) -> None:
|
||||
"""
|
||||
Creates a snapshot of a virtual machine.
|
||||
|
||||
@@ -327,7 +332,8 @@ class HypervisorDriver(PackageManagementMixin, HealthMetricsMixin, DeviceTypeDri
|
||||
:param name: VM name or its ``vmid``.
|
||||
:param snapshot: Name of the snapshot to delete.
|
||||
:raises ValueError: If the VM or snapshot does not exist.
|
||||
:raises RuntimeError: If other snapshots depend on this one (must delete children first).
|
||||
:raises RuntimeError: If other snapshots depend on this one
|
||||
(must delete children first).
|
||||
|
||||
Example::
|
||||
|
||||
@@ -357,7 +363,7 @@ class HypervisorDriver(PackageManagementMixin, HealthMetricsMixin, DeviceTypeDri
|
||||
# Storage
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def get_vm_storage_pools(self) -> Dict[str, StorageVolumeDict]:
|
||||
def get_vm_storage_pools(self) -> dict[str, StorageVolumeDict]:
|
||||
"""
|
||||
Returns the storage pools / datastores configured on this hypervisor.
|
||||
|
||||
@@ -401,7 +407,7 @@ class HypervisorDriver(PackageManagementMixin, HealthMetricsMixin, DeviceTypeDri
|
||||
# Virtual networking
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def get_virtual_networks(self) -> Dict[str, VirtualNetworkDict]:
|
||||
def get_virtual_networks(self) -> dict[str, VirtualNetworkDict]:
|
||||
"""
|
||||
Returns the virtual networks / bridges defined on this hypervisor.
|
||||
|
||||
@@ -441,11 +447,6 @@ class HypervisorDriver(PackageManagementMixin, HealthMetricsMixin, DeviceTypeDri
|
||||
# Package management (hypervisor extensions / plugins)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Virtual machine provisioning
|
||||
# ------------------------------------------------------------------
|
||||
@@ -457,28 +458,26 @@ class HypervisorDriver(PackageManagementMixin, HealthMetricsMixin, DeviceTypeDri
|
||||
image_url: str,
|
||||
cpu: int,
|
||||
memory: int,
|
||||
nics: List[NICConfigDict],
|
||||
cloud_init_config: Dict[str, Any],
|
||||
nics: list[NICConfigDict],
|
||||
cloud_init_config: dict[str, Any],
|
||||
image_checksum: str | None = None,
|
||||
ssh_public_keys: List[str] | None = None,
|
||||
ssh_public_keys: list[str] | None = None,
|
||||
disk_resize_gb: int | None = None,
|
||||
storage: str | None = None,
|
||||
cpu_type: str | None = None,
|
||||
guest_os: str = "linux",
|
||||
node: str | None = None,
|
||||
download_timeout: int = 300,
|
||||
timeout: int = 180,
|
||||
) -> VMProvisionResultDict:
|
||||
"""
|
||||
Create a new virtual machine from a cloud image via Cloud-Init.
|
||||
|
||||
Downloads the cloud image (qcow2/raw) directly on the hypervisor if not
|
||||
already cached there, creates a new VM shell, imports the image as its
|
||||
root disk, configures virtual network interfaces, and injects Cloud-Init
|
||||
configuration via a storage snippet or similar mechanism. The resulting
|
||||
VM is left in a running state.
|
||||
|
||||
Implementations should cache downloaded images by URL/filename on the
|
||||
hypervisor so repeated provisioning from the same image does not
|
||||
re-download it every time.
|
||||
Downloads the cloud image (qcow2/raw) directly on the hypervisor unless it is
|
||||
cached there already (implementations cache by URL/filename), creates a new VM
|
||||
shell, imports the image as its root disk, configures virtual network interfaces,
|
||||
and injects Cloud-Init configuration via a storage snippet or similar mechanism.
|
||||
The resulting VM is left in a running state.
|
||||
|
||||
Args:
|
||||
name (string) - new VM display name
|
||||
@@ -497,10 +496,8 @@ class HypervisorDriver(PackageManagementMixin, HealthMetricsMixin, DeviceTypeDri
|
||||
image_checksum (string | None) - expected checksum of the downloaded
|
||||
image (e.g. "sha256:<hex>"). If given, verified after download;
|
||||
mismatch raises RuntimeError. If None, no verification is performed.
|
||||
ssh_public_keys (list[str] | None) - SSH public keys to inject into guest.
|
||||
If None or empty, no SSH key injection is performed.
|
||||
disk_resize_gb (int | None) - resize root disk to this size in GB.
|
||||
If None, disk remains the downloaded image's native size. Default None.
|
||||
ssh_public_keys (list[str] | None) - SSH public keys to inject (None: none).
|
||||
disk_resize_gb (int | None) - resize root disk to this size in GB (None: keep).
|
||||
storage (string | None) - name of the storage pool to place the root
|
||||
disk on (a name returned by ``get_image_storages()``). If None,
|
||||
the driver auto-detects the first enabled, node-available storage
|
||||
@@ -512,6 +509,16 @@ class HypervisorDriver(PackageManagementMixin, HealthMetricsMixin, DeviceTypeDri
|
||||
value other than None with ValueError; one that has it raises
|
||||
ValueError for a name it does not list or that is not
|
||||
``available`` on this node, before creating anything.
|
||||
guest_os (string) - the operating system in the image, a key of
|
||||
``GUEST_AGENTS`` (``"linux"``, ``"freebsd"``, ``"openbsd"``). The
|
||||
driver sets up the VM's hardware for it (OS type, how the guest
|
||||
agent is attached) and, where the guest needs it, writes the
|
||||
network-config itself (``provisioning.network_config``). A
|
||||
guest_os the driver does not list raises ValueError before
|
||||
anything is created. Default ``"linux"``.
|
||||
node (string | None) - a node ``name`` from ``get_clusters()`` to create
|
||||
the VM on; None leaves it to the driver. One that only creates on its
|
||||
own node raises ValueError for another, before creating anything.
|
||||
download_timeout (int) - maximum seconds to wait for the image download
|
||||
(skipped entirely if already cached on the hypervisor). Default 300.
|
||||
timeout (int) - maximum seconds to wait for the remaining provisioning
|
||||
@@ -524,6 +531,8 @@ class HypervisorDriver(PackageManagementMixin, HealthMetricsMixin, DeviceTypeDri
|
||||
Raises:
|
||||
RuntimeError - if provisioning fails (download failure, checksum
|
||||
mismatch, storage unavailable, invalid config, timeout, etc.)
|
||||
VMProvisionIncomplete - a RuntimeError for a failure after the VM was created,
|
||||
with that VM's ``vmid`` and ``node`` for ``destroy_vm`` and the failure as cause.
|
||||
"""
|
||||
...
|
||||
|
||||
@@ -592,7 +601,7 @@ class HypervisorDriver(PackageManagementMixin, HealthMetricsMixin, DeviceTypeDri
|
||||
"""
|
||||
...
|
||||
|
||||
def get_network_targets(self) -> List[NetworkTargetDict]:
|
||||
def get_network_targets(self) -> list[NetworkTargetDict]:
|
||||
"""
|
||||
List the network targets a new VM's NIC may attach to.
|
||||
|
||||
@@ -606,11 +615,13 @@ class HypervisorDriver(PackageManagementMixin, HealthMetricsMixin, DeviceTypeDri
|
||||
List[NetworkTargetDict] - each entry's ``vlan_aware`` flag tells the
|
||||
caller whether a ``NICConfigDict.vlan_tag`` may additionally be set
|
||||
for a NIC using that target (see ``NetworkTargetDict`` for the
|
||||
per-kind rules).
|
||||
per-kind rules). ``uplinks``, where the driver can tell, names
|
||||
the physical NICs a bridge leaves the host through, grouped:
|
||||
a bond's slaves in one group, a single NIC in its own.
|
||||
"""
|
||||
...
|
||||
|
||||
def get_image_storages(self) -> List[StorageTargetDict]:
|
||||
def get_image_storages(self) -> list[StorageTargetDict]:
|
||||
"""
|
||||
List the storage pools a new VM's root disk may be placed on, scoped to
|
||||
the specific node the VM will be created on.
|
||||
@@ -628,7 +639,71 @@ class HypervisorDriver(PackageManagementMixin, HealthMetricsMixin, DeviceTypeDri
|
||||
"""
|
||||
...
|
||||
|
||||
def get_vm_cpu_types(self) -> List[VMCpuTypeDict]:
|
||||
def get_clusters(self) -> list[ClusterDict]:
|
||||
"""
|
||||
List the clusters this connection can see, each with its nodes, its
|
||||
storages and its networks.
|
||||
|
||||
A cluster member (a Proxmox node) reports its one cluster, the whole of
|
||||
it: every node, offline ones included with ``status`` ``"offline"``, and
|
||||
``local`` set on the node it is. A manager (vCenter) reports each
|
||||
cluster it runs, and each standalone host it runs as a cluster of one.
|
||||
A standalone host reports itself as a cluster of one, ``name`` None.
|
||||
|
||||
The VMs ``get_vms()`` lists name their node in ``node``; that name is
|
||||
the node's ``name`` here. A storage or network lists the nodes it is
|
||||
available on, so a caller can tell where a VM may go. A value the
|
||||
platform does not report is None, never a guess.
|
||||
|
||||
Facts only: which node is a good place for a VM is the caller's to
|
||||
decide.
|
||||
|
||||
Returns:
|
||||
List[ClusterDict]
|
||||
"""
|
||||
...
|
||||
|
||||
def get_vm_migration_preflight(self, name: str) -> MigrationPreflightDict:
|
||||
"""Which nodes of its cluster the guest could move to, and what blocks it
|
||||
(see ``vm_migration``).
|
||||
|
||||
:raises ValueError: If no VM with the given name/ID exists.
|
||||
"""
|
||||
...
|
||||
|
||||
def migrate_vm(
|
||||
self,
|
||||
name: str,
|
||||
target_node: str,
|
||||
online: bool = True,
|
||||
target_storage: str | None = None,
|
||||
timeout: int = 3600,
|
||||
) -> None:
|
||||
"""Move the guest to another node of its cluster; block until the
|
||||
platform's task ended. ``online`` keeps a running guest running.
|
||||
|
||||
:raises ValueError: Unknown VM, or a target that is not another
|
||||
member of the cluster.
|
||||
:raises RuntimeError: The migration failed, with the platform's cause.
|
||||
"""
|
||||
...
|
||||
|
||||
def set_node_maintenance(self, node: str, enabled: bool) -> None:
|
||||
"""Switch HA maintenance mode of a cluster node; HA guests leave it
|
||||
while it is on.
|
||||
|
||||
:raises ValueError: If ``node`` is not a member of the cluster.
|
||||
:raises RuntimeError: If the platform refuses or cannot be asked.
|
||||
"""
|
||||
...
|
||||
|
||||
def get_hypervisor_hosts(self) -> list[HypervisorHostDict]:
|
||||
"""The hosts this hypervisor or its manager runs, and how each stands:
|
||||
connected, in maintenance, with configuration issues, under a central
|
||||
manager. Facts; what is a problem is the caller's to decide."""
|
||||
...
|
||||
|
||||
def get_vm_cpu_types(self) -> list[VMCpuTypeDict]:
|
||||
"""
|
||||
List the virtual CPU models a new VM may be given on the node it will
|
||||
be created on.
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Dropping interfaces that are noise rather than signal.
|
||||
|
||||
An access point exposes radio PHYs and a loopback alongside its real
|
||||
@@ -9,19 +8,19 @@ attributes are the customisation point.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any, ClassVar, Dict, FrozenSet, Tuple
|
||||
from typing import Any, ClassVar
|
||||
|
||||
|
||||
class InterfaceFilterMixin:
|
||||
"""Adds :meth:`_filter_interfaces` for drivers that report raw interface dicts."""
|
||||
|
||||
#: Interface names dropped outright.
|
||||
_EXCLUDED_INTERFACES: ClassVar[FrozenSet[str]] = frozenset({"lo"})
|
||||
_EXCLUDED_INTERFACES: ClassVar[frozenset[str]] = frozenset({"lo"})
|
||||
|
||||
#: Name prefixes dropped -- ``phy*`` are radio devices, not links.
|
||||
_EXCLUDED_INTERFACE_PREFIXES: ClassVar[Tuple[str, ...]] = ("phy",)
|
||||
_EXCLUDED_INTERFACE_PREFIXES: ClassVar[tuple[str, ...]] = ("phy",)
|
||||
|
||||
def _filter_interfaces(self, interfaces: Dict[str, Any]) -> Dict[str, Any]:
|
||||
def _filter_interfaces(self, interfaces: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Remove loopback and radio-device (phy*) interfaces from an interface dict."""
|
||||
return {
|
||||
name: data
|
||||
|
||||
@@ -0,0 +1,171 @@
|
||||
"""What the running kernel has built and loaded.
|
||||
|
||||
A kernel CVE's exploitability often hangs on code that is simply not there --
|
||||
a module that is neither loaded nor shipped, a subsystem the kernel was built
|
||||
without. Reading that is identical on every Linux host, so the command and its
|
||||
parse live here once and a driver only carries the command across: SSH,
|
||||
an API's exec endpoint, whatever it has.
|
||||
|
||||
The command is read-only and needs no privileges. It frames its report and
|
||||
sends it gzipped and base64-encoded, for two reasons: nothing in the payload can
|
||||
then look like a shell prompt to a screen-scraping transport, and a kernel's
|
||||
build configuration (~300 kB on a distribution kernel) crosses as a fifth of
|
||||
that.
|
||||
|
||||
What the four lists mean for a module, and why "not loaded" alone is never
|
||||
"absent": a module that is not loaded can still be loaded on demand -- by an
|
||||
attacker too, where autoloading reaches it. Only a module that is neither
|
||||
loaded, nor compiled in, nor shipped for this kernel is one it cannot have.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import binascii
|
||||
import gzip
|
||||
import zlib
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from napalm_device_types.models import KernelFactsDict
|
||||
|
||||
_BEGIN = "KFACTS_BEGIN"
|
||||
_END = "KFACTS_END"
|
||||
|
||||
#: One line, POSIX ``sh``, read-only. The frame markers are printed in two
|
||||
#: halves so that a transport which echoes the command does not show them early.
|
||||
KERNEL_FACTS_COMMAND = (
|
||||
"r=$(uname -r); m=/lib/modules/$r; "
|
||||
"printf '%s%s\\n' KFACTS_ BEGIN; "
|
||||
"{ echo '[release]'; echo \"$r\"; "
|
||||
"if [ -r /proc/modules ]; then echo '[loaded]'; cut -d' ' -f1 /proc/modules; fi; "
|
||||
"if [ -r $m/modules.builtin ]; then echo '[builtin]'; cat $m/modules.builtin; fi; "
|
||||
"if [ -r $m/modules.dep ]; then echo '[available]'; cut -d: -f1 $m/modules.dep; fi; "
|
||||
"if [ -r /boot/config-$r ]; then echo '[config]'; grep '^CONFIG_' /boot/config-$r; "
|
||||
"elif [ -r /proc/config.gz ]; then echo '[config]'; "
|
||||
"zcat /proc/config.gz | grep '^CONFIG_'; fi; "
|
||||
"} 2>/dev/null | gzip -c | base64; "
|
||||
"printf '%s%s\\n' KFACTS_ END"
|
||||
)
|
||||
|
||||
|
||||
def module_name(raw: str) -> str:
|
||||
"""A module as the kernel names it: no path, no ``.ko`` suffix, ``_`` for ``-``.
|
||||
|
||||
``kernel/net/can/can-raw.ko.zst`` and ``can_raw`` are the same module; the
|
||||
kernel itself treats dash and underscore alike.
|
||||
"""
|
||||
base = raw.strip().rsplit("/", 1)[-1]
|
||||
suffix = base.find(".ko")
|
||||
if suffix != -1:
|
||||
base = base[:suffix]
|
||||
return base.replace("-", "_").lower()
|
||||
|
||||
|
||||
def _report(output: str) -> str:
|
||||
lines = [line.strip() for line in output.splitlines()]
|
||||
try:
|
||||
start = lines.index(_BEGIN)
|
||||
end = lines.index(_END, start)
|
||||
except ValueError:
|
||||
raise ValueError("no kernel facts in the output") from None
|
||||
try:
|
||||
packed = base64.b64decode("".join(lines[start + 1 : end]), validate=True)
|
||||
report = gzip.decompress(packed).decode()
|
||||
except (binascii.Error, OSError, EOFError, zlib.error, UnicodeDecodeError) as exc:
|
||||
raise ValueError(f"the kernel facts could not be decoded: {exc}") from exc
|
||||
if "[release]" not in report.splitlines():
|
||||
raise ValueError("the kernel facts are empty: gzip or base64 is missing on the host")
|
||||
return report
|
||||
|
||||
|
||||
def _sections(report: str) -> dict[str, list[str]]:
|
||||
sections: dict[str, list[str]] = {}
|
||||
current: list[str] | None = None
|
||||
for line in report.splitlines():
|
||||
line = line.strip()
|
||||
if line.startswith("[") and line.endswith("]"):
|
||||
current = sections.setdefault(line[1:-1], [])
|
||||
elif line and current is not None:
|
||||
current.append(line)
|
||||
return sections
|
||||
|
||||
|
||||
def _config(lines: list[str]) -> dict[str, str]:
|
||||
config: dict[str, str] = {}
|
||||
for line in lines:
|
||||
option, sep, value = line.partition("=")
|
||||
if not sep:
|
||||
continue
|
||||
if len(value) >= 2 and value[0] == value[-1] == '"':
|
||||
value = value[1:-1]
|
||||
config[option] = value
|
||||
return config
|
||||
|
||||
|
||||
def parse_kernel_facts(output: str) -> KernelFactsDict:
|
||||
"""Parse what :data:`KERNEL_FACTS_COMMAND` printed.
|
||||
|
||||
A section the command did not print -- the file was missing or unreadable --
|
||||
comes back ``None``, never empty.
|
||||
|
||||
:raises ValueError: when the output carries no intact report.
|
||||
"""
|
||||
sections = _sections(_report(output))
|
||||
|
||||
def names(key: str) -> list[str] | None:
|
||||
if key not in sections:
|
||||
return None
|
||||
return sorted({module_name(line) for line in sections[key]})
|
||||
|
||||
release = sections.get("release") or [""]
|
||||
return {
|
||||
"release": release[0],
|
||||
"loaded": names("loaded"),
|
||||
"builtin": names("builtin"),
|
||||
"available": names("available"),
|
||||
"config": _config(sections["config"]) if "config" in sections else None,
|
||||
}
|
||||
|
||||
|
||||
class KernelFactsMixin:
|
||||
"""Adds :meth:`get_kernel_facts` to a driver that can run a command on a Linux host.
|
||||
|
||||
The template form (README, "Function classes"): the reading and its parse are
|
||||
the same everywhere, so they are concrete here, and a driver supplies only
|
||||
:meth:`_run_kernel_facts_command` -- how a command reaches its host. Mixed in
|
||||
by the drivers that can, not by :class:`~napalm_device_types.os.OSDriver`:
|
||||
a Windows host is an OS driver too and has no Linux kernel to read, and
|
||||
``hasattr(driver, "get_kernel_facts")`` has to stay a truthful answer.
|
||||
"""
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def _run_kernel_facts_command(self, command: str) -> str:
|
||||
"""Run *command* on the host with ``sh`` and return what it printed."""
|
||||
...
|
||||
|
||||
def get_kernel_facts(self) -> KernelFactsDict:
|
||||
"""
|
||||
Returns what the running kernel has built and loaded.
|
||||
|
||||
* release (string) - ``uname -r``
|
||||
* loaded (list or None) - loaded modules, from ``/proc/modules``
|
||||
* builtin (list or None) - modules compiled into the kernel image
|
||||
* available (list or None) - modules shipped for this kernel
|
||||
* config (dict or None) - the build configuration's set options
|
||||
|
||||
``None`` means the source could not be read.
|
||||
|
||||
Example::
|
||||
|
||||
{
|
||||
"release": "6.1.0-25-amd64",
|
||||
"loaded": ["nf_tables", "tipc"],
|
||||
"builtin": ["tcp_cubic"],
|
||||
"available": ["can_raw", "nf_tables", "tipc"],
|
||||
"config": {"CONFIG_TIPC": "m", "CONFIG_HZ": "250"},
|
||||
}
|
||||
|
||||
:raises ValueError: if the host's output carried no intact report.
|
||||
"""
|
||||
return parse_kernel_facts(self._run_kernel_facts_command(KERNEL_FACTS_COMMAND))
|
||||
@@ -1,4 +1,3 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Logical LAG entries for ``get_interfaces()``, built from their member ports.
|
||||
|
||||
Some switches list only physical ports, each tagged with the trunk it belongs
|
||||
@@ -10,17 +9,17 @@ the same for every vendor, so it lives here once; a driver only has to set
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from typing import Any, Dict, List, Optional
|
||||
from typing import Any
|
||||
|
||||
|
||||
def _port_order(name: str) -> List[Any]:
|
||||
def _port_order(name: str) -> list[Any]:
|
||||
return [int(p) if p.isdigit() else p for p in re.split(r"(\d+)", name)]
|
||||
|
||||
|
||||
def add_lag_interfaces(
|
||||
interfaces: Dict[str, Dict[str, Any]],
|
||||
lag_modes: Optional[Dict[str, str]] = None,
|
||||
) -> Dict[str, Dict[str, Any]]:
|
||||
interfaces: dict[str, dict[str, Any]],
|
||||
lag_modes: dict[str, str] | None = None,
|
||||
) -> dict[str, dict[str, Any]]:
|
||||
"""Return *interfaces* plus one logical entry per ``trunk_group``.
|
||||
|
||||
The LAG entry is up/enabled if any member is, its speed is the members'
|
||||
@@ -31,7 +30,7 @@ def add_lag_interfaces(
|
||||
mode gets no ``lag_mode`` key rather than a guessed one.
|
||||
"""
|
||||
result = dict(interfaces)
|
||||
groups: Dict[str, List[str]] = {}
|
||||
groups: dict[str, list[str]] = {}
|
||||
for name, iface in interfaces.items():
|
||||
group = iface.get("trunk_group")
|
||||
if group:
|
||||
@@ -41,12 +40,12 @@ def add_lag_interfaces(
|
||||
if group in result:
|
||||
continue
|
||||
members = sorted(members, key=_port_order)
|
||||
lag: Dict[str, Any] = {
|
||||
lag: dict[str, Any] = {
|
||||
"is_up": any(interfaces[m].get("is_up") for m in members),
|
||||
"is_enabled": any(interfaces[m].get("is_enabled") for m in members),
|
||||
"description": f"LAG ({', '.join(members)})",
|
||||
"last_flapped": -1.0,
|
||||
"speed": sum(float(interfaces[m].get("speed") or 0) for m in members),
|
||||
"speed": sum(max(float(interfaces[m].get("speed") or 0), 0.0) for m in members),
|
||||
"mtu": -1,
|
||||
"mac_address": "",
|
||||
"lag_members": members,
|
||||
|
||||
@@ -0,0 +1,31 @@
|
||||
"""Licenses and subscriptions a device reports.
|
||||
|
||||
A fact for the caller (docs/STRUCTURE.md): whether a free edition, a missing
|
||||
subscription or a license a feature needs is a problem is decided above the
|
||||
driver. Declared only; a driver that can read licenses implements it.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from napalm_device_types.models import LicenseDict
|
||||
|
||||
|
||||
class LicenseMixin:
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def get_licenses(self) -> list[LicenseDict]:
|
||||
"""Every license or subscription the device knows of, with its status.
|
||||
|
||||
Example::
|
||||
|
||||
[{"name": "pve-subscription", "status": "none"},
|
||||
{"name": "vSphere", "status": "active", "edition": "Free"},
|
||||
{"name": "DCMS", "status": "missing",
|
||||
"withholds": ["drives", "firmware inventory"]}]
|
||||
|
||||
``withholds`` names what a device keeps back for want of a license while
|
||||
the rest of it answers; a missing license without it may keep back all.
|
||||
"""
|
||||
...
|
||||
@@ -0,0 +1,279 @@
|
||||
"""Listening sockets: what listens on which address, and which service it is.
|
||||
|
||||
Whether a service is reachable from outside its host is decided by what it
|
||||
listens on -- ``0.0.0.0:5432`` is, ``127.0.0.1:5432`` is not -- and that is read
|
||||
the same way on every Linux host. So the command and its parse live here once,
|
||||
and a driver only carries the command across.
|
||||
|
||||
**One round trip.** ``ss -lntup`` lists every listening TCP and bound UDP
|
||||
socket with the processes holding it; for each of those processes,
|
||||
``/proc/<pid>/cgroup`` says which systemd service or container it runs in. The
|
||||
report is framed, and a report whose end is missing raises: a list cut short
|
||||
must never read as sockets that closed.
|
||||
|
||||
**Root, and without it.** Only root sees every process behind a socket.
|
||||
The whole script therefore goes to the host as one ``sh -c`` argument -- a
|
||||
driver that prefixes ``sudo -n`` would otherwise run only its first command as
|
||||
root. When that call brings no report back (no sudo, a wrong password), the
|
||||
command runs again without privilege: the sockets are still worth having, and
|
||||
the reading says it is not ``attributed``.
|
||||
|
||||
**No ``-H``.** iproute2 before 4.10 has no option to leave out the header and
|
||||
fails on it, which would read as nothing listening. The parse skips the header
|
||||
instead.
|
||||
|
||||
**Without ``ss``, ``netstat``.** OpenWrt's busybox and old net-tools hosts have
|
||||
no ``ss``; ``netstat -lntup`` lists the same sockets with ``PID/Program``, and
|
||||
the cgroups are read for its PIDs alike. On OpenWrt the cgroup names the procd
|
||||
service (``/services/<name>/<instance>``), a jailed one too, whose PID is not the
|
||||
one procd reports. A host with neither raises :class:`ListeningSocketsUnavailable`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from shlex import quote
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from napalm_device_types.models import ListeningSocketDict, ListeningSocketsDict
|
||||
from napalm_device_types.terminal import strip_terminal_codes
|
||||
|
||||
_BEGIN = "SOCK_BEGIN"
|
||||
_END = "SOCK_END"
|
||||
_NO_SS = "no-ss"
|
||||
_RC_RE = re.compile(r"^__SS_RC=(\d+)$")
|
||||
|
||||
#: One line, POSIX ``sh``, read-only. The frame markers are printed in two
|
||||
#: halves so that a transport which echoes the command does not show them early.
|
||||
#: ``ss`` is in ``/usr/sbin`` on some systems, outside a login user's ``PATH``.
|
||||
#: ``netstat`` names a socket's process as ``PID/Program``, ``ss`` as ``pid=PID``.
|
||||
LISTENING_SOCKETS_COMMAND = (
|
||||
"PATH=$PATH:/usr/sbin:/sbin; "
|
||||
"printf '%s%s\\n' SOCK_ BEGIN; "
|
||||
"if command -v ss >/dev/null 2>&1; then "
|
||||
"t=ss; s=$(ss -lntup 2>&1); r=$?; "
|
||||
"pids=$(printf '%s\\n' \"$s\" | grep -o 'pid=[0-9]*' | cut -d= -f2); "
|
||||
"elif command -v netstat >/dev/null 2>&1; then "
|
||||
"t=netstat; s=$(netstat -lntup 2>&1); r=$?; "
|
||||
"pids=$(printf '%s\\n' \"$s\" | grep -o ' [0-9][0-9]*/' | tr -d ' /'); "
|
||||
"else t=; fi; "
|
||||
'if [ -n "$t" ]; then '
|
||||
'echo "[$t]"; printf \'%s\\n\' "$s"; echo "__SS_RC=$r"; '
|
||||
"echo '[cgroups]'; "
|
||||
"for p in $(printf '%s\\n' \"$pids\" | sort -u); do "
|
||||
'sed "s|^|$p |" /proc/$p/cgroup 2>/dev/null; done; '
|
||||
"else echo '[no-ss]'; fi; "
|
||||
"printf '%s%s\\n' SOCK_ END"
|
||||
)
|
||||
|
||||
_PROTOCOLS = frozenset({"tcp", "udp"})
|
||||
#: netstat names the IPv6 sockets of net-tools ``tcp6``/``udp6``; busybox does not.
|
||||
_NETSTAT_PROTOCOLS = {"tcp": "tcp", "tcp6": "tcp", "udp": "udp", "udp6": "udp"}
|
||||
#: ``1604/dropbear``; net-tools prints ``700/sshd: /usr/sbin``.
|
||||
_PROGRAM_RE = re.compile(r"^(\d+)/(\S*)")
|
||||
#: ``users:(("nginx",pid=901,fd=6),("nginx",pid=900,fd=6))``
|
||||
_USER_RE = re.compile(r'\("((?:[^"\\]|\\.)*)",pid=(\d+),fd=\d+\)')
|
||||
#: The service a cgroup path runs in: its deepest ``*.service`` component.
|
||||
_SERVICE_RE = re.compile(r"/([^/]+)\.service(?=/|$)")
|
||||
#: OpenWrt's procd: ``/services/<name>/<instance>``.
|
||||
_PROCD_RE = re.compile(r"^/services/([^/]+)(?:/|$)")
|
||||
#: A container's cgroup: ``docker-<id>.scope`` (systemd driver), ``/docker/<id>`` (cgroupfs).
|
||||
_CONTAINER_RE = re.compile(
|
||||
r"(?:docker|libpod)-([0-9a-f]{64})\.scope|/(?:docker|libpod)/([0-9a-f]{64})(?=/|$)"
|
||||
)
|
||||
|
||||
|
||||
class ListeningSocketsUnavailable(NotImplementedError):
|
||||
"""The host has neither ``ss`` nor ``netstat``; nothing to read, nothing to retry."""
|
||||
|
||||
|
||||
def _frame(output: str) -> list[str]:
|
||||
lines = [line.strip() for line in strip_terminal_codes(output).splitlines()]
|
||||
try:
|
||||
start = lines.index(_BEGIN)
|
||||
end = lines.index(_END, start)
|
||||
except ValueError:
|
||||
raise ValueError("no intact listening socket report in the output") from None
|
||||
return lines[start + 1 : end]
|
||||
|
||||
|
||||
def _sections(lines: list[str]) -> dict[str, list[str]]:
|
||||
sections: dict[str, list[str]] = {}
|
||||
current: list[str] = []
|
||||
for line in lines:
|
||||
if line.startswith("[") and line.endswith("]") and " " not in line:
|
||||
current = sections.setdefault(line[1:-1], [])
|
||||
else:
|
||||
current.append(line)
|
||||
return sections
|
||||
|
||||
|
||||
def _split_local(local: str) -> tuple[str, str | None, int] | None:
|
||||
"""``[fe80::1]%eth0:546`` (or ``[fe80::1%eth0]:546``) -> ``("fe80::1", "eth0", 546)``;
|
||||
None if no port."""
|
||||
host, sep, port = local.rpartition(":")
|
||||
if not sep or not port.isdigit():
|
||||
return None
|
||||
address, _, zone = host.partition("%")
|
||||
zone = zone.rstrip("]")
|
||||
if address.startswith("["):
|
||||
address = address[1:].rstrip("]")
|
||||
return address or "*", zone or None, int(port)
|
||||
|
||||
|
||||
def _cgroup_paths(lines: list[str]) -> dict[int, str]:
|
||||
"""Each process's cgroup path: the unified hierarchy, or systemd's under v1."""
|
||||
paths: dict[int, str] = {}
|
||||
for line in lines:
|
||||
pid, _, entry = line.partition(" ")
|
||||
parts = entry.split(":", 2)
|
||||
if not pid.isdigit() or len(parts) != 3:
|
||||
continue
|
||||
hierarchy, controllers, path = parts
|
||||
if (hierarchy == "0" and controllers == "") or controllers == "name=systemd":
|
||||
paths[int(pid)] = path
|
||||
return paths
|
||||
|
||||
|
||||
def _unit(path: str | None) -> str | None:
|
||||
services = _SERVICE_RE.findall(path or "")
|
||||
if services:
|
||||
return str(services[-1])
|
||||
procd = _PROCD_RE.match(path or "")
|
||||
return str(procd.group(1)) if procd else None
|
||||
|
||||
|
||||
def _container(path: str | None) -> str | None:
|
||||
match = _CONTAINER_RE.search(path or "")
|
||||
return (match.group(1) or match.group(2)) if match else None
|
||||
|
||||
|
||||
def _entry(
|
||||
proto: str,
|
||||
local: tuple[str, str | None, int],
|
||||
process: str | None,
|
||||
pid: int | None,
|
||||
paths: dict[int, str],
|
||||
) -> ListeningSocketDict:
|
||||
address, interface, port = local
|
||||
path = paths.get(pid) if pid is not None else None
|
||||
return {
|
||||
"proto": proto,
|
||||
"address": address,
|
||||
"port": port,
|
||||
"interface": interface,
|
||||
"process": process,
|
||||
"pid": pid,
|
||||
"unit": _unit(path),
|
||||
"container_id": _container(path),
|
||||
}
|
||||
|
||||
|
||||
def _ss_socket(line: str, paths: dict[int, str]) -> ListeningSocketDict | None:
|
||||
parts = line.split()
|
||||
if len(parts) < 5 or parts[0] not in _PROTOCOLS:
|
||||
return None
|
||||
local = _split_local(parts[4])
|
||||
if local is None:
|
||||
return None
|
||||
users = _USER_RE.findall(line)
|
||||
process, pid = (users[0][0], int(users[0][1])) if users else (None, None)
|
||||
return _entry(parts[0], local, process, pid, paths)
|
||||
|
||||
|
||||
def _netstat_socket(line: str, paths: dict[int, str]) -> ListeningSocketDict | None:
|
||||
"""``Proto Recv-Q Send-Q Local Foreign [State] PID/Program`` -- a UDP line
|
||||
has no state, and ``-`` is a socket without a process."""
|
||||
parts = line.split()
|
||||
proto = _NETSTAT_PROTOCOLS.get(parts[0]) if parts else None
|
||||
if proto is None or len(parts) < 6:
|
||||
return None
|
||||
local = _split_local(parts[3])
|
||||
if local is None:
|
||||
return None
|
||||
process, pid = None, None
|
||||
for token in parts[5:7]:
|
||||
program = _PROGRAM_RE.match(token)
|
||||
if program:
|
||||
pid, process = int(program.group(1)), program.group(2).rstrip(":") or None
|
||||
break
|
||||
return _entry(proto, local, process, pid, paths)
|
||||
|
||||
|
||||
_PARSERS = {"ss": _ss_socket, "netstat": _netstat_socket}
|
||||
|
||||
|
||||
def parse_listening_sockets(output: str) -> list[ListeningSocketDict]:
|
||||
"""Parse what :data:`LISTENING_SOCKETS_COMMAND` printed, sorted by protocol,
|
||||
port and address.
|
||||
|
||||
:raises ListeningSocketsUnavailable: when the host has neither ``ss`` nor ``netstat``.
|
||||
:raises ValueError: when the output carries no intact report, or the tool failed.
|
||||
"""
|
||||
sections = _sections(_frame(output))
|
||||
if _NO_SS in sections:
|
||||
raise ListeningSocketsUnavailable("the host has neither ss nor netstat")
|
||||
tool = "netstat" if "netstat" in sections else "ss"
|
||||
lines = sections.get(tool, [])
|
||||
statuses = [m.group(1) for m in map(_RC_RE.match, lines) if m]
|
||||
if not statuses or statuses[-1] != "0":
|
||||
detail = " ".join(line for line in lines if not _RC_RE.match(line))[:200]
|
||||
raise ValueError(f"{tool} did not list the sockets: {detail or 'no exit status'}")
|
||||
paths = _cgroup_paths(sections.get("cgroups", []))
|
||||
parse = _PARSERS[tool]
|
||||
sockets = [s for s in (parse(line, paths) for line in lines) if s is not None]
|
||||
return sorted(
|
||||
sockets, key=lambda s: (s["proto"], s["port"], s["address"], s["interface"] or "")
|
||||
)
|
||||
|
||||
|
||||
class ListeningSocketsMixin:
|
||||
"""Adds :meth:`get_listening_sockets` to a driver that can run a command on a Linux host.
|
||||
|
||||
With ``ss``, or ``netstat`` where there is none (OpenWrt's busybox).
|
||||
|
||||
The template form (README, "Function classes"): the command and its parse
|
||||
are the same everywhere, so they are concrete here, and a driver supplies
|
||||
only :meth:`_run_listening_sockets_command` -- how a command reaches its
|
||||
host, and how it gains root there. Mixed in by the drivers that can, not by
|
||||
:class:`~napalm_device_types.os.OSDriver`: a Windows host is an OS driver
|
||||
too, and ``hasattr(driver, "get_listening_sockets")`` has to stay truthful.
|
||||
"""
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def _run_listening_sockets_command(self, command: str, *, privileged: bool) -> str:
|
||||
"""Run *command* on the host and return what it printed; as root
|
||||
when *privileged*. The command is a single ``sh -c`` invocation."""
|
||||
...
|
||||
|
||||
def get_listening_sockets(self) -> ListeningSocketsDict:
|
||||
"""
|
||||
Returns every listening TCP and bound UDP socket, with the process,
|
||||
systemd service and container behind it.
|
||||
|
||||
* attributed (bool) - read as root, so every process is named
|
||||
* sockets (list) - see :class:`~napalm_device_types.models.ListeningSocketDict`
|
||||
|
||||
Example::
|
||||
|
||||
{
|
||||
"attributed": True,
|
||||
"sockets": [
|
||||
{"proto": "tcp", "address": "0.0.0.0", "port": 5432,
|
||||
"interface": None, "process": "postgres", "pid": 812,
|
||||
"unit": "postgresql@16-main", "container_id": None},
|
||||
],
|
||||
}
|
||||
|
||||
:raises ListeningSocketsUnavailable: if the host has neither ``ss`` nor ``netstat``.
|
||||
:raises ValueError: if neither reading carried an intact report.
|
||||
"""
|
||||
command = f"sh -c {quote(LISTENING_SOCKETS_COMMAND)}"
|
||||
try:
|
||||
output = self._run_listening_sockets_command(command, privileged=True)
|
||||
return {"attributed": True, "sockets": parse_listening_sockets(output)}
|
||||
except ValueError:
|
||||
pass
|
||||
output = self._run_listening_sockets_command(command, privileged=False)
|
||||
return {"attributed": False, "sockets": parse_listening_sockets(output)}
|
||||
@@ -0,0 +1,97 @@
|
||||
"""LLDP neighbours as rows, and the two shapes NAPALM asks for.
|
||||
|
||||
A driver reads its neighbours one row per neighbour -- from ``lldpctl``
|
||||
(:func:`~napalm_device_types.lldpctl.parse_lldpctl_keyvalue`), an API or a CLI
|
||||
-- and ``get_lldp_neighbors`` and ``get_lldp_neighbors_detail`` are the same
|
||||
projection of those rows for every driver. A port with several neighbours (a
|
||||
phone and the PC behind it, an unmanaged switch) is several rows, and stays
|
||||
several entries in both shapes.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from collections.abc import Iterable
|
||||
from typing import NotRequired, TypedDict
|
||||
|
||||
from napalm.base.models import LLDPNeighborDetailDict
|
||||
|
||||
#: A chassis id of type ``mac`` that really is a MAC address.
|
||||
_MAC_RE = re.compile(r"^([0-9A-Fa-f]{2}:){5}[0-9A-Fa-f]{2}$")
|
||||
|
||||
|
||||
class LldpNeighborRow(TypedDict):
|
||||
"""One LLDP neighbour, as the local port heard it.
|
||||
|
||||
``chassis_id_type`` and ``port_id_type`` name what the id is: ``mac``,
|
||||
``ip``, ``ifname``, ``ifalias``, ``local`` or ``unhandled`` (an id type
|
||||
lldpd does not decode), ``""`` when the neighbour sent no id.
|
||||
``capabilities`` and ``enabled_capabilities`` use NAPALM's names
|
||||
(``bridge``, ``router``, ``wlan-access-point``, ``telephone``, ...).
|
||||
"""
|
||||
|
||||
local_port: str
|
||||
chassis_id: str
|
||||
chassis_id_type: str
|
||||
port_id: str
|
||||
port_id_type: str
|
||||
port_description: str
|
||||
system_name: str
|
||||
system_description: str
|
||||
management_addresses: list[str]
|
||||
capabilities: list[str]
|
||||
enabled_capabilities: list[str]
|
||||
|
||||
|
||||
class LldpNeighborDict(TypedDict):
|
||||
"""NAPALM's ``get_lldp_neighbors`` entry (``hostname``, ``port``), plus the chassis MAC.
|
||||
|
||||
``mac`` (lower case) is there only when the neighbour names its chassis by a
|
||||
MAC address; an application can match the neighbour by it, where a name
|
||||
may be ambiguous.
|
||||
"""
|
||||
|
||||
hostname: str
|
||||
port: str
|
||||
mac: NotRequired[str]
|
||||
|
||||
|
||||
def lldp_neighbors(rows: Iterable[LldpNeighborRow]) -> dict[str, list[LldpNeighborDict]]:
|
||||
"""``get_lldp_neighbors``: per local port, every neighbour's name, port and chassis MAC."""
|
||||
neighbors: dict[str, list[LldpNeighborDict]] = {}
|
||||
for row in rows:
|
||||
entry: LldpNeighborDict = {"hostname": row["system_name"], "port": row["port_id"]}
|
||||
if row["chassis_id_type"] == "mac" and _MAC_RE.match(row["chassis_id"]):
|
||||
entry["mac"] = row["chassis_id"].lower()
|
||||
neighbors.setdefault(row["local_port"], []).append(entry)
|
||||
return neighbors
|
||||
|
||||
|
||||
def lldp_neighbors_detail(
|
||||
rows: Iterable[LldpNeighborRow], interface: str = ""
|
||||
) -> dict[str, list[LLDPNeighborDetailDict]]:
|
||||
"""``get_lldp_neighbors_detail``: per local port, every neighbour in full.
|
||||
|
||||
With ``interface``, that port only. ``parent_interface`` stays empty: a row
|
||||
does not know which aggregate its port belongs to.
|
||||
"""
|
||||
details: dict[str, list[LLDPNeighborDetailDict]] = {}
|
||||
for row in rows:
|
||||
if interface and row["local_port"] != interface:
|
||||
continue
|
||||
details.setdefault(row["local_port"], []).append(
|
||||
{
|
||||
"parent_interface": "",
|
||||
"remote_port": row["port_id"],
|
||||
"remote_port_description": row["port_description"],
|
||||
"remote_chassis_id": row["chassis_id"],
|
||||
"remote_system_name": row["system_name"],
|
||||
"remote_system_description": row["system_description"],
|
||||
"remote_system_capab": list(row["capabilities"]),
|
||||
"remote_system_enable_capab": list(row["enabled_capabilities"]),
|
||||
}
|
||||
)
|
||||
return details
|
||||
|
||||
|
||||
__all__ = ["LldpNeighborDict", "LldpNeighborRow", "lldp_neighbors", "lldp_neighbors_detail"]
|
||||
@@ -0,0 +1,204 @@
|
||||
"""``lldpctl -f keyvalue``: lldpd's neighbours, one row each.
|
||||
|
||||
Every driver for a host that runs lldpd reads its neighbours the same way, so
|
||||
the command and its parse live here once and a driver only carries the command
|
||||
across. lldpd's keyvalue writer (``src/client/kv_writer.c``) prints one
|
||||
``key=value`` line per field, the key being the field's path joined by dots
|
||||
(a newline inside a value becomes a space)::
|
||||
|
||||
lldp.eth0.1.via=LLDP
|
||||
lldp.eth0.1.chassis.mac=00:11:22:33:44:01
|
||||
lldp.eth0.1.chassis.Tel.enabled=on
|
||||
lldp.eth0.1.port.local=gi1/0/7
|
||||
|
||||
**A neighbour is a block, not a port.** Every neighbour on a port prints the
|
||||
same keys, so grouping the lines by interface merges them. A neighbour opens
|
||||
with ``via`` (after the local interface's ``alias``, when it has one); the next
|
||||
``via`` on the same port is the next neighbour.
|
||||
|
||||
**An interface name may hold dots** (``eth0.1``, ``br-lan.10``). It is
|
||||
everything between ``lldp.`` and the neighbour's first field, never the second
|
||||
dotted part of the key.
|
||||
|
||||
**An id is printed under its type**, ``chassis.<type>=`` and ``port.<type>=``,
|
||||
with lldpd's names for the type (``mac``, ``ip``, ``ifname``, ``ifalias``,
|
||||
``local``, ``unhandled``); there is no ``chassis.id`` key.
|
||||
|
||||
**A failed read raises.** lldpctl exits non-zero when it cannot reach lldpd;
|
||||
:data:`LLDPCTL_KEYVALUE_COMMAND` brings that status and lldpctl's message back,
|
||||
and the parse raises on it instead of reading it as no neighbours. Output that
|
||||
carries no exit status (a driver that runs plain ``lldpctl -f keyvalue``) raises
|
||||
when it holds text but not a single neighbour line, as a shell's ``lldpctl: not
|
||||
found`` does.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
|
||||
from napalm_device_types.lldp import LldpNeighborRow
|
||||
|
||||
_RC = "__LLDPCTL_RC"
|
||||
|
||||
#: One line, POSIX ``sh``, read-only. Without arguments lldpctl shows the
|
||||
#: neighbours in detail; its error text and exit status come back with them.
|
||||
LLDPCTL_KEYVALUE_COMMAND = f"lldpctl -f keyvalue 2>&1; echo {_RC}=$?"
|
||||
|
||||
_RC_RE = re.compile(rf"^{_RC}=(\d+)$")
|
||||
_PREFIX = "lldp."
|
||||
#: The fields a neighbour opens with: the local interface's alias, then ``via``.
|
||||
_OPENERS = frozenset({"alias", "via"})
|
||||
#: The first part after the interface name of every key lldpd prints for a
|
||||
#: neighbour (``src/client/display.c``, ``display_interface``).
|
||||
_CATEGORIES = _OPENERS | {
|
||||
"rid",
|
||||
"age",
|
||||
"status",
|
||||
"chassis",
|
||||
"port",
|
||||
"ttl",
|
||||
"vlan",
|
||||
"ppvid",
|
||||
"pi",
|
||||
"lldp-med",
|
||||
"unknown-tlvs",
|
||||
}
|
||||
#: lldpd's id types (``src/lib/atoms/chassis.c``, ``port.c``), in the order read.
|
||||
_CHASSIS_ID_TYPES = ("mac", "local", "ip", "ifname", "ifalias", "unhandled")
|
||||
_PORT_ID_TYPES = ("ifname", "mac", "local", "ip", "ifalias", "unhandled")
|
||||
#: lldpd's capability symbols in NAPALM's names; one NAPALM lacks is lowercased.
|
||||
_CAPABILITIES = {
|
||||
"Other": "other",
|
||||
"Repeater": "repeater",
|
||||
"Bridge": "bridge",
|
||||
"Router": "router",
|
||||
"Wlan": "wlan-access-point",
|
||||
"Tel": "telephone",
|
||||
"Docsis": "docsis-cable-device",
|
||||
"Station": "station",
|
||||
}
|
||||
_SWITCH = {"on": True, "off": False}
|
||||
|
||||
_Fields = dict[str, list[str]]
|
||||
|
||||
|
||||
def parse_lldpctl_keyvalue(output: str) -> list[LldpNeighborRow]:
|
||||
"""One row per neighbour, in the order lldpctl listed them.
|
||||
|
||||
:raises ValueError: when lldpctl failed -- a non-zero exit status, or text
|
||||
without a single neighbour line and no status at all.
|
||||
"""
|
||||
return [_row(port, fields) for port, fields in _neighbours(_pairs(output))]
|
||||
|
||||
|
||||
def _pairs(output: str) -> list[tuple[str, str]]:
|
||||
"""``(key without "lldp.", value)`` of every neighbour line; raises on a failed run."""
|
||||
pairs: list[tuple[str, str]] = []
|
||||
other: list[str] = []
|
||||
status: int | None = None
|
||||
for line in output.splitlines():
|
||||
line = line.strip()
|
||||
key, sep, value = line.partition("=")
|
||||
if _RC_RE.match(line):
|
||||
status = int(value)
|
||||
elif sep and key.startswith(_PREFIX):
|
||||
pairs.append((key[len(_PREFIX) :], value))
|
||||
elif line:
|
||||
other.append(line)
|
||||
_raise_on_failure(status, other, bool(pairs))
|
||||
return pairs
|
||||
|
||||
|
||||
def _raise_on_failure(status: int | None, other: list[str], listed: bool) -> None:
|
||||
"""A non-zero status, or text without a status and without a neighbour line."""
|
||||
if status or (status is None and other and not listed):
|
||||
said = " ".join(other)[:200] or f"exit status {status}"
|
||||
raise ValueError(f"lldpctl did not list the neighbours: {said}")
|
||||
|
||||
|
||||
def _neighbours(pairs: list[tuple[str, str]]) -> list[tuple[str, _Fields]]:
|
||||
"""Split the lines into neighbours: ``(local interface, {field: [values]})``."""
|
||||
neighbours: list[tuple[str, _Fields]] = []
|
||||
port: str | None = None
|
||||
fields: _Fields = {}
|
||||
for key, value in pairs:
|
||||
split = _split(key, port)
|
||||
if split is None:
|
||||
continue
|
||||
name, field = split
|
||||
if name != port or field == "alias" or (field == "via" and "via" in fields):
|
||||
port, fields = name, {}
|
||||
neighbours.append((port, fields))
|
||||
fields.setdefault(field, []).append(value)
|
||||
return neighbours
|
||||
|
||||
|
||||
def _split(key: str, port: str | None) -> tuple[str, str] | None:
|
||||
"""``(interface, field)`` of a key, or None when it is no neighbour field.
|
||||
|
||||
``via`` and ``alias`` end their key, so they name the interface exactly;
|
||||
the keys after them continue that interface. Only a key that does neither
|
||||
ends its interface name at the first known field.
|
||||
"""
|
||||
head, _, last = key.rpartition(".")
|
||||
if head and last in _OPENERS:
|
||||
return head, last
|
||||
if port is not None and key.startswith(port + "."):
|
||||
field = key[len(port) + 1 :]
|
||||
if field.split(".", 1)[0] in _CATEGORIES:
|
||||
return port, field
|
||||
parts = key.split(".")
|
||||
for i in range(1, len(parts)):
|
||||
if parts[i] in _CATEGORIES:
|
||||
return ".".join(parts[:i]), ".".join(parts[i:])
|
||||
return None
|
||||
|
||||
|
||||
def _row(port: str, fields: _Fields) -> LldpNeighborRow:
|
||||
chassis_type, chassis_id = _id(fields, "chassis", _CHASSIS_ID_TYPES)
|
||||
port_type, port_id = _id(fields, "port", _PORT_ID_TYPES)
|
||||
capabilities, enabled = _capabilities(fields)
|
||||
return {
|
||||
"local_port": port,
|
||||
"chassis_id": chassis_id,
|
||||
"chassis_id_type": chassis_type,
|
||||
"port_id": port_id,
|
||||
"port_id_type": port_type,
|
||||
"port_description": _first(fields, "port.descr"),
|
||||
"system_name": _first(fields, "chassis.name"),
|
||||
"system_description": _first(fields, "chassis.descr"),
|
||||
"management_addresses": list(fields.get("chassis.mgmt-ip", [])),
|
||||
"capabilities": capabilities,
|
||||
"enabled_capabilities": enabled,
|
||||
}
|
||||
|
||||
|
||||
def _first(fields: _Fields, field: str) -> str:
|
||||
return fields.get(field, [""])[0]
|
||||
|
||||
|
||||
def _id(fields: _Fields, category: str, types: tuple[str, ...]) -> tuple[str, str]:
|
||||
"""``(type, id)`` of the chassis or the port; ``("", "")`` when none was sent."""
|
||||
for kind in types:
|
||||
if f"{category}.{kind}" in fields:
|
||||
return kind, _first(fields, f"{category}.{kind}")
|
||||
return "", ""
|
||||
|
||||
|
||||
def _capabilities(fields: _Fields) -> tuple[list[str], list[str]]:
|
||||
"""``chassis.<Symbol>.enabled=on|off``: the capabilities offered, and those enabled."""
|
||||
offered: list[str] = []
|
||||
enabled: list[str] = []
|
||||
for field, values in fields.items():
|
||||
parts = field.split(".")
|
||||
switch = _SWITCH.get(values[0])
|
||||
if len(parts) != 3 or parts[0] != "chassis" or parts[2] != "enabled" or switch is None:
|
||||
continue
|
||||
name = _CAPABILITIES.get(parts[1], parts[1].lower())
|
||||
offered.append(name)
|
||||
if switch:
|
||||
enabled.append(name)
|
||||
return offered, enabled
|
||||
|
||||
|
||||
__all__ = ["LLDPCTL_KEYVALUE_COMMAND", "parse_lldpctl_keyvalue"]
|
||||
@@ -0,0 +1,915 @@
|
||||
"""Personal login accounts on a host: reading them, adding one, locking one again.
|
||||
|
||||
A person who is to work on a host gets an account of their own: they log in
|
||||
with their public key and, where they are to administer the host, may use
|
||||
sudo with their own password. When they leave, the account is locked, not
|
||||
deleted: their key goes, sudo goes, the login is refused, and the home stays.
|
||||
|
||||
**What the person changes follows them** (NetOrk/netork#2013). A key they
|
||||
replace goes from the account as the new one comes in (``replaces``); a key
|
||||
they delete goes without a new one (``remove_local_account_keys``). Either way
|
||||
only the keys named go: one the person added on the host stays, and the
|
||||
account stays open. A password they remove goes from the account too
|
||||
(``clear_password``): it logs in with a key only and has no sudo, as the sudo
|
||||
rule asks for that password.
|
||||
|
||||
:class:`LocalAccountsMixin` holds what is the same on every POSIX host -- the
|
||||
checks, reading an account back, the order of the steps, the key file and the
|
||||
sudo rule. A driver supplies the commands that differ between operating
|
||||
systems: creating, opening and locking an account, reading its lock, and
|
||||
taking it out of a group.
|
||||
|
||||
**Every command root runs is one fixed command**, with no shell syntax of its
|
||||
own -- ``useradd ...``, ``install -m 440 ...``, ``rm -f -- <path>`` -- so a
|
||||
sudoers allowlist of commands can name each one. What goes into a file is
|
||||
staged first, without privilege; root never reads stdin, where a sudo that
|
||||
does not prompt would leave the password.
|
||||
|
||||
**Nothing root runs follows the person's files.** The key files live in a home
|
||||
the person owns, where a symbolic link may point anywhere, so they are read and
|
||||
written as that person (``sudo -n -u <name>``). Only the existence of a key
|
||||
file is asked as root.
|
||||
|
||||
**The sudo rule is a file of its own** in the drop-in directory sudo includes,
|
||||
``<name> ALL=(ALL) ALL``: the person's own password, as on a new VM. A group
|
||||
alone is not enough, as some distributions grant ``%wheel`` and others
|
||||
``%sudo``. ``visudo`` checks the rule before it is installed, and ``sudo -l``
|
||||
afterwards shows whether it took effect. A rule elsewhere -- cloud-init's
|
||||
``90-cloud-init-users`` -- is not edited; it is reported.
|
||||
|
||||
**The login user's own account and system accounts are never touched**:
|
||||
locking the first would lock the caller out.
|
||||
|
||||
**netOrk's own account** (``add_service_account``) is made the same way,
|
||||
through the account it logs in with today: a key and no password, and a rule
|
||||
of its own -- ``<name> ALL=(ALL) NOPASSWD: ALL`` in ``85-service-<name>`` --
|
||||
so it gains root without a password that could end up anywhere. Once netOrk
|
||||
logs in with it, the person's account it used before is one like any other.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import shlex
|
||||
from collections.abc import Iterable, Sequence
|
||||
from typing import TYPE_CHECKING, ClassVar
|
||||
|
||||
from typing_extensions import TypedDict
|
||||
|
||||
from napalm_device_types.fixes import FixStep, run_fix_steps
|
||||
from napalm_device_types.models import FixResultDict, FixStepDict, FixStepOutcomeDict
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - for type checkers only
|
||||
from napalm_device_types.channel import CommandResult
|
||||
|
||||
#: A login name every POSIX host takes as it is: lowercase, a letter or an
|
||||
#: underscore first, at most 32 characters. It is also safe in a root command.
|
||||
ACCOUNT_NAME_RE = re.compile(r"[a-z_][a-z0-9_.-]{0,31}")
|
||||
|
||||
#: The crypt(3) prefixes of each password scheme a driver may take.
|
||||
PASSWORD_SCHEMES: dict[str, tuple[str, ...]] = {
|
||||
"sha512": ("$6$",),
|
||||
"bcrypt": ("$2b$", "$2a$", "$2y$"),
|
||||
}
|
||||
_CRYPT_RE = re.compile(r"\$[0-9a-z]+\$[./0-9A-Za-z$]+")
|
||||
|
||||
#: The public key types OpenSSH takes in a key file.
|
||||
_KEY_TYPE_RE = re.compile(
|
||||
r"(ssh-ed25519|ssh-rsa|ecdsa-sha2-nistp(256|384|521)"
|
||||
r"|sk-ssh-ed25519@openssh\.com|sk-ecdsa-sha2-nistp256@openssh\.com)"
|
||||
)
|
||||
_KEY_BLOB_RE = re.compile(r"[A-Za-z0-9+/]+={0,3}")
|
||||
|
||||
#: Before the rule's file name; after cloud-init's ``90-`` it would win, so before it.
|
||||
SUDOERS_FILE_PREFIX = "80-account-"
|
||||
#: Before the file name of netOrk's own account's rule; after a person's, before cloud-init's.
|
||||
SERVICE_SUDOERS_FILE_PREFIX = "85-service-"
|
||||
|
||||
#: A line of ``sudo -l`` that lets the account run every command as root.
|
||||
_RUN_ALL_RE = re.compile(r"^\s*\(ALL(\s*:\s*ALL)?\)\s+([A-Z_]+:\s*)*ALL\s*$", re.MULTILINE)
|
||||
#: The same, without a password.
|
||||
_RUN_ALL_NOPASSWD_RE = re.compile(
|
||||
r"^\s*\(ALL(\s*:\s*ALL)?\)\s+([A-Z_]+:\s*)*NOPASSWD:\s*([A-Z_]+:\s*)*ALL\s*$",
|
||||
re.MULTILINE,
|
||||
)
|
||||
|
||||
#: Login shells that refuse every login: an account with one belongs to a service.
|
||||
_NO_LOGIN_SHELLS = frozenset({"nologin", "false"})
|
||||
|
||||
#: Stages stdin as a file anyone may read -- the person reads the key file
|
||||
#: from it -- and prints its path. The content is a public key or a sudo rule.
|
||||
_STAGE = 't=$(mktemp) && chmod 644 "$t" && cat > "$t" && printf %s "$t"'
|
||||
|
||||
|
||||
class LocalAccountDict(TypedDict):
|
||||
"""A person's account as the host has it (``get_local_accounts``)."""
|
||||
|
||||
name: str
|
||||
uid: int
|
||||
home: str
|
||||
shell: str
|
||||
#: Primary group first.
|
||||
groups: list[str]
|
||||
#: No login, with a key or a password.
|
||||
locked: bool
|
||||
#: ``sudo -l`` lets it run every command as root.
|
||||
admin: bool
|
||||
#: The lines of its key files; None when they could not be read (a locked
|
||||
#: account cannot run a command any more).
|
||||
authorized_keys: list[str] | None
|
||||
|
||||
|
||||
class LocalAccountChangeDict(FixResultDict):
|
||||
"""The result of ``add_local_account`` and ``lock_local_account``."""
|
||||
|
||||
#: The account read back after the change; None when it is not there or
|
||||
#: could not be read.
|
||||
account: LocalAccountDict | None
|
||||
|
||||
|
||||
def sudoers_rule(name: str) -> str:
|
||||
"""The rule that lets *name* run everything as root, with its own password."""
|
||||
return f"{name} ALL=(ALL) ALL\n"
|
||||
|
||||
|
||||
def sudoers_rule_path(directory: str, name: str) -> str:
|
||||
"""Where *name*'s rule lives in *directory*.
|
||||
|
||||
sudo skips a file whose name holds a dot, so a dot becomes ``_2e``, and an
|
||||
underscore ``__``: no two names share a file.
|
||||
"""
|
||||
return f"{directory}/{SUDOERS_FILE_PREFIX}{_rule_file_name(name)}"
|
||||
|
||||
|
||||
def service_sudoers_rule(name: str) -> str:
|
||||
"""The rule that lets netOrk's own account *name* run everything as root, no password."""
|
||||
return f"{name} ALL=(ALL) NOPASSWD: ALL\n"
|
||||
|
||||
|
||||
def service_sudoers_rule_path(directory: str, name: str) -> str:
|
||||
"""Where the rule of netOrk's own account *name* lives in *directory*."""
|
||||
return f"{directory}/{SERVICE_SUDOERS_FILE_PREFIX}{_rule_file_name(name)}"
|
||||
|
||||
|
||||
def _rule_file_name(name: str) -> str:
|
||||
return name.replace("_", "__").replace(".", "_2e")
|
||||
|
||||
|
||||
def authorized_keys_paths(home: str) -> tuple[str, str]:
|
||||
"""The key files sshd reads in *home* unless configured otherwise."""
|
||||
return f"{home}/.ssh/authorized_keys", f"{home}/.ssh/authorized_keys2"
|
||||
|
||||
|
||||
def check_account_name(name: str) -> None:
|
||||
""":raises ValueError: when *name* is no login name every host takes."""
|
||||
if not ACCOUNT_NAME_RE.fullmatch(name):
|
||||
raise ValueError(f"not an account name: {name!r}")
|
||||
|
||||
|
||||
def _key_blob(line: str) -> str | None:
|
||||
"""The base64 part of a key file line, options and comment aside."""
|
||||
words = line.split()
|
||||
for i, word in enumerate(words[:-1]):
|
||||
if _KEY_TYPE_RE.fullmatch(word):
|
||||
return words[i + 1]
|
||||
return None
|
||||
|
||||
|
||||
def check_public_key(key: str) -> None:
|
||||
""":raises ValueError: when *key* is not one OpenSSH public key on one line."""
|
||||
words = key.split()
|
||||
if (
|
||||
"\n" in key.strip()
|
||||
or len(words) < 2
|
||||
or not _KEY_TYPE_RE.fullmatch(words[0])
|
||||
or not _KEY_BLOB_RE.fullmatch(words[1])
|
||||
):
|
||||
raise ValueError("not a public key: expected '<type> <base64> [comment]' on one line")
|
||||
|
||||
|
||||
def _checked_keys(keys: Sequence[str]) -> list[str]:
|
||||
"""*keys*, each one stripped.
|
||||
|
||||
:raises ValueError: when *keys* is one string, or one of them no public key.
|
||||
"""
|
||||
if isinstance(keys, str):
|
||||
raise ValueError("expected a list of public keys, not one string")
|
||||
for key in keys:
|
||||
check_public_key(key)
|
||||
return [key.strip() for key in keys]
|
||||
|
||||
|
||||
def _command(argv: Sequence[str]) -> str:
|
||||
return shlex.join(argv)
|
||||
|
||||
|
||||
def _as(name: str, *argv: str) -> str:
|
||||
"""*argv* run as *name*, from root: nothing it does reaches beyond that account."""
|
||||
return _command(["sudo", "-n", "-u", name, *argv])
|
||||
|
||||
|
||||
def _said(result: CommandResult) -> str:
|
||||
return (result.stderr or result.stdout).strip() or f"exit {result.exit_code}"
|
||||
|
||||
|
||||
class LocalAccountsMixin:
|
||||
"""Reads, adds and locks personal accounts; a driver supplies the OS's commands.
|
||||
|
||||
Mixed in by the drivers that can, so ``callable(getattr(driver,
|
||||
"add_local_account", None))`` says whether a driver can.
|
||||
"""
|
||||
|
||||
#: The login shell of a new account.
|
||||
ACCOUNT_SHELL: ClassVar[str] = "/bin/sh"
|
||||
#: The password scheme the host's accounts take, a key of :data:`PASSWORD_SCHEMES`.
|
||||
ACCOUNT_PASSWORD_SCHEME: ClassVar[str] = "sha512"
|
||||
#: Groups that make an account an administrator; a person without sudo leaves them.
|
||||
ACCOUNT_ADMIN_GROUPS: ClassVar[tuple[str, ...]] = ("wheel",)
|
||||
#: The lowest uid of a person's account; below are the system's.
|
||||
ACCOUNT_MIN_UID: ClassVar[int] = 1000
|
||||
#: The drop-in directory sudo includes.
|
||||
SUDOERS_DIR: ClassVar[str] = "/etc/sudoers.d"
|
||||
#: The group of root's files.
|
||||
ROOT_GROUP: ClassVar[str] = "root"
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def run_command(
|
||||
self,
|
||||
command: str,
|
||||
*,
|
||||
privileged: bool = False,
|
||||
timeout: float = 60,
|
||||
stdin: bytes | None = None,
|
||||
) -> CommandResult: ...
|
||||
|
||||
# -- what a driver supplies ------------------------------------------------------
|
||||
#
|
||||
# ``_account_*`` are the driver's: the five below it must supply, the two after
|
||||
# them (``_account_create_with_uid``, ``_account_clear_password``) it may, and
|
||||
# the key store (``_account_key_files``, ``_account_read_keys``,
|
||||
# ``_account_add_key``, ``_account_drop_keys``, ``_account_remove_keys``) it may
|
||||
# replace. ``_accounts_*`` are this mixin's own.
|
||||
|
||||
def _account_create(self, name: str, password_hash: str | None) -> list[list[str]]:
|
||||
"""The commands that create *name* with a home and :attr:`ACCOUNT_SHELL`.
|
||||
|
||||
Without *password_hash* the account takes no password, only a key.
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
def _account_open(
|
||||
self, name: str, password_hash: str | None, *, locked: bool
|
||||
) -> list[list[str]]:
|
||||
"""The commands that let *name* log in again, with *password_hash* if given."""
|
||||
raise NotImplementedError
|
||||
|
||||
def _account_lock(self, name: str, *, locked: bool) -> list[list[str]]:
|
||||
"""The commands that refuse *name* every login, a key's too; none when *locked*."""
|
||||
raise NotImplementedError
|
||||
|
||||
def _account_locked(self, name: str, passwd: list[str]) -> bool:
|
||||
"""Whether *name* is locked; *passwd* is its ``getent passwd`` line, split.
|
||||
|
||||
:raises RuntimeError: when that cannot be read.
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
def _account_leave_group(self, name: str, group: str, groups: list[str]) -> list[str]:
|
||||
"""The command that takes *name* out of *group*; *groups* are all of its own."""
|
||||
raise NotImplementedError
|
||||
|
||||
def _account_create_with_uid(self, name: str, uid: int) -> list[list[str]]:
|
||||
"""The commands that create *name* as :meth:`_account_create` does, without a
|
||||
password, and with the uid *uid*.
|
||||
|
||||
Only netOrk's own account asks for one (``add_service_account(uid=)``).
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
def _account_clear_password(self, name: str) -> list[list[str]]:
|
||||
"""The commands that take *name*'s password away; its key login stays as it is.
|
||||
|
||||
Not an empty password -- PAM's ``nullok`` lets anyone in without one --
|
||||
and not a lock or an expiry, which sshd or PAM apply to a key too.
|
||||
Only ``add_local_account(clear_password=True)`` and
|
||||
``remove_local_account_keys(clear_password=True)`` ask for it.
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
# -- reading ---------------------------------------------------------------------
|
||||
|
||||
def get_local_accounts(self, names: Iterable[str]) -> list[LocalAccountDict]:
|
||||
"""The accounts of *names* the host has, in that order; a missing one is left out.
|
||||
|
||||
:raises ValueError: for a name that is no account name.
|
||||
:raises RuntimeError: when an account cannot be read.
|
||||
"""
|
||||
wanted = list(names)
|
||||
for name in wanted:
|
||||
check_account_name(name)
|
||||
found = (self._accounts_read(name) for name in wanted)
|
||||
return [account for account in found if account is not None]
|
||||
|
||||
def _accounts_passwd(self, name: str) -> list[str] | None:
|
||||
result = self.run_command(_command(["getent", "passwd", name]), timeout=30)
|
||||
if result.exit_code == 2:
|
||||
return None
|
||||
fields = result.stdout.strip().split(":")
|
||||
if result.exit_code != 0 or len(fields) < 7 or fields[0] != name:
|
||||
raise RuntimeError(f"cannot read the account {name}: {_said(result)}")
|
||||
return fields
|
||||
|
||||
def _accounts_read(self, name: str) -> LocalAccountDict | None:
|
||||
passwd = self._accounts_passwd(name)
|
||||
if passwd is None:
|
||||
return None
|
||||
groups = self.run_command(_command(["id", "-Gn", name]), timeout=30)
|
||||
if groups.exit_code != 0:
|
||||
raise RuntimeError(f"cannot read the groups of {name}: {_said(groups)}")
|
||||
home = passwd[5]
|
||||
return {
|
||||
"name": name,
|
||||
"uid": int(passwd[2]),
|
||||
"home": home,
|
||||
"shell": passwd[6],
|
||||
"groups": groups.stdout.split(),
|
||||
"locked": self._account_locked(name, passwd),
|
||||
"admin": self._accounts_admin(name),
|
||||
"authorized_keys": self._account_read_keys(name, home),
|
||||
}
|
||||
|
||||
def _accounts_admin(self, name: str) -> bool:
|
||||
result = self.run_command(_command(["sudo", "-l", "-U", name]), privileged=True, timeout=30)
|
||||
if result.exit_code != 0:
|
||||
raise RuntimeError(f"cannot ask sudo about {name}: {_said(result)}")
|
||||
return bool(_RUN_ALL_RE.search(result.stdout))
|
||||
|
||||
def _account_key_files(self, name: str, home: str) -> tuple[str, ...]:
|
||||
"""Every file sshd reads *name*'s keys from; a host that adds one of its own says so."""
|
||||
return authorized_keys_paths(home)
|
||||
|
||||
def _account_read_keys(self, name: str, home: str) -> list[str] | None:
|
||||
"""The lines of :meth:`_account_key_files`, read as *name*; None when it cannot."""
|
||||
lines: list[str] = []
|
||||
for path in self._account_key_files(name, home):
|
||||
there = self.run_command(_command(["test", "-e", path]), privileged=True, timeout=30)
|
||||
if there.exit_code != 0:
|
||||
continue
|
||||
result = self.run_command(_as(name, "cat", "--", path), privileged=True, timeout=30)
|
||||
if result.exit_code != 0:
|
||||
return None
|
||||
lines += [ln for ln in result.stdout.splitlines() if ln.strip()]
|
||||
return lines
|
||||
|
||||
# -- changing ----------------------------------------------------------------------
|
||||
|
||||
def add_local_account(
|
||||
self,
|
||||
name: str,
|
||||
public_key: str,
|
||||
*,
|
||||
password_hash: str | None = None,
|
||||
admin: bool = False,
|
||||
clear_password: bool = False,
|
||||
replaces: Sequence[str] = (),
|
||||
) -> LocalAccountChangeDict:
|
||||
"""Give *name* an account that logs in with *public_key*, or open it again.
|
||||
|
||||
An account that is there is opened again and keeps its other keys; a
|
||||
new one gets a home. *password_hash* (crypt(3), in
|
||||
:attr:`ACCOUNT_PASSWORD_SCHEME`) becomes its password; *admin* gives it
|
||||
the sudo rule, which asks for that password -- so an administrator
|
||||
needs one. Without *admin*, the rule and :attr:`ACCOUNT_ADMIN_GROUPS`
|
||||
are taken away. Without *password_hash* the account keeps the password
|
||||
it has, unless *clear_password* takes it away: the account then logs
|
||||
in with a key only, and can have no sudo.
|
||||
|
||||
*replaces* are keys *public_key* takes the place of -- the person's
|
||||
old ones. They go from the account, matched by their base64 whatever
|
||||
a line's options or comment; every other key stays.
|
||||
|
||||
Steps: ``account``, then ``sudo``, ``password`` (with
|
||||
*clear_password*), ``keys`` and ``old keys`` (with *replaces*), each of
|
||||
which runs when another failed: an old key goes even when the new one
|
||||
could not be added.
|
||||
|
||||
:raises ValueError: for a name, key or hash no host takes, a replaced
|
||||
key that is none, *clear_password* with a *password_hash* or with
|
||||
*admin*, the login user's own account, or a system account.
|
||||
"""
|
||||
check_account_name(name)
|
||||
check_public_key(public_key)
|
||||
new = _key_blob(public_key)
|
||||
old = [key for key in _checked_keys(replaces) if _key_blob(key) != new]
|
||||
self._accounts_check_password(password_hash, admin=admin, clear=clear_password)
|
||||
existing = self._accounts_changeable(name)
|
||||
first = run_fix_steps(
|
||||
[("account", lambda: self._accounts_open_or_create(name, existing, password_hash))]
|
||||
)
|
||||
parts = [first]
|
||||
if first["success"]:
|
||||
steps = self._accounts_after_open(name, public_key, admin, clear_password, old)
|
||||
parts += [run_fix_steps([step]) for step in steps]
|
||||
return self._accounts_result(name, parts)
|
||||
|
||||
def _accounts_after_open(
|
||||
self, name: str, public_key: str, admin: bool, clear_password: bool, old: list[str]
|
||||
) -> list[FixStep]:
|
||||
"""The steps of ``add_local_account`` once the account is there, each on its own."""
|
||||
steps: list[FixStep] = [("sudo", lambda: self._accounts_set_sudo(name, admin=admin))]
|
||||
if clear_password:
|
||||
steps.append(("password", lambda: self._accounts_clear_password(name)))
|
||||
steps.append(("keys", lambda: self._accounts_add_key(name, public_key)))
|
||||
if old:
|
||||
steps.append(("old keys", lambda: self._accounts_drop_keys(name, old)))
|
||||
return steps
|
||||
|
||||
def remove_local_account_keys(
|
||||
self, name: str, keys: Sequence[str], *, clear_password: bool = False
|
||||
) -> LocalAccountChangeDict:
|
||||
"""Take *keys* off *name*'s account, for a person who deleted them.
|
||||
|
||||
Only those keys go, matched by their base64 whatever a line's options
|
||||
or comment; the other keys stay, and the account stays open -- to get
|
||||
a new key once there is one. A locked account has none left to
|
||||
remove: the lock took them all. Without *clear_password* the password
|
||||
and sudo stay too; with it the password goes, and so do the sudo rule
|
||||
and :attr:`ACCOUNT_ADMIN_GROUPS`, as sudo asks for that password -- for
|
||||
a person whose password went with their key.
|
||||
|
||||
Steps: ``keys``, then ``password`` and ``sudo`` (with *clear_password*),
|
||||
each of which runs when another failed.
|
||||
|
||||
:raises ValueError: for a name no host takes, no keys or one that is
|
||||
none, the login user's own account, or a system account.
|
||||
"""
|
||||
check_account_name(name)
|
||||
wanted = _checked_keys(keys)
|
||||
if not wanted:
|
||||
raise ValueError("no public key to remove")
|
||||
if self._accounts_changeable(name) is None:
|
||||
step: FixStepDict = {"step": "keys", "ok": False, "output": f"no account {name}"}
|
||||
return {**_joined([step]), "account": None}
|
||||
steps: list[FixStep] = [("keys", lambda: self._accounts_drop_keys(name, wanted))]
|
||||
if clear_password:
|
||||
steps.append(("password", lambda: self._accounts_clear_password(name)))
|
||||
steps.append(("sudo", lambda: self._accounts_set_sudo(name, admin=False)))
|
||||
return self._accounts_result(name, [run_fix_steps([step]) for step in steps])
|
||||
|
||||
def lock_local_account(self, name: str) -> LocalAccountChangeDict:
|
||||
"""Lock *name*: its keys go, every login is refused, sudo goes; its home stays.
|
||||
|
||||
Steps ``keys``, ``lock`` and ``sudo``, each run when one before failed:
|
||||
the lock matters most. A sudo rule outside the drop-in directory is
|
||||
reported, not edited -- with the account locked it can no longer be used.
|
||||
|
||||
:raises ValueError: for a name no host takes, the login user's own
|
||||
account, or a system account.
|
||||
"""
|
||||
check_account_name(name)
|
||||
if self._accounts_changeable(name) is None:
|
||||
step: FixStepDict = {"step": "account", "ok": False, "output": f"no account {name}"}
|
||||
return {**_joined([step]), "account": None}
|
||||
parts = [
|
||||
run_fix_steps([("keys", lambda: self._accounts_remove_keys(name))]),
|
||||
run_fix_steps([("lock", lambda: self._accounts_lock(name))]),
|
||||
run_fix_steps(
|
||||
[("sudo", lambda: self._accounts_set_sudo(name, admin=False, locked=True))]
|
||||
),
|
||||
]
|
||||
return self._accounts_result(name, parts)
|
||||
|
||||
def add_service_account(
|
||||
self, name: str, public_key: str, *, uid: int | None = None
|
||||
) -> LocalAccountChangeDict:
|
||||
"""Make *name* netOrk's own account: it logs in with *public_key*, and gains
|
||||
root without a password.
|
||||
|
||||
Created through the account the driver logs in with now, or opened
|
||||
again when it is there; its other keys stay. It takes no password. Run
|
||||
again with the account the driver already logs in with, it only adds
|
||||
what is missing.
|
||||
|
||||
A new account gets *uid* when one is given -- the same on every host --
|
||||
once no other account has it; an account that is there keeps its own,
|
||||
as a uid is never changed under files that carry it.
|
||||
|
||||
Steps as in :meth:`add_local_account`: ``account``, then ``sudo`` and
|
||||
``keys``. The new login itself is for the caller to try.
|
||||
|
||||
An account that is there and cannot log in (``nologin``, ``false``) is
|
||||
another service's -- a satellite's ``netork``, say -- and is left alone.
|
||||
|
||||
:raises ValueError: for a name or key no host takes, root, a uid of 0 or
|
||||
less, or another service's account.
|
||||
:raises RuntimeError: when the login user cannot be told.
|
||||
"""
|
||||
check_public_key(public_key)
|
||||
login, existing = self._accounts_service_target(name, uid)
|
||||
if name == login:
|
||||
step: FixStepDict = {
|
||||
"step": "account",
|
||||
"ok": True,
|
||||
"output": f"this driver logs in as {name} already",
|
||||
}
|
||||
first: FixResultDict = _joined([step])
|
||||
else:
|
||||
first = run_fix_steps(
|
||||
[("account", lambda: self._accounts_service_account(name, existing, uid))]
|
||||
)
|
||||
parts = [first]
|
||||
if first["success"]:
|
||||
parts += [
|
||||
run_fix_steps([("sudo", lambda: self._accounts_grant_service_sudo(name))]),
|
||||
run_fix_steps([("keys", lambda: self._accounts_add_key(name, public_key))]),
|
||||
]
|
||||
return self._accounts_result(name, parts)
|
||||
|
||||
def _accounts_service_target(self, name: str, uid: int | None) -> tuple[str, list[str] | None]:
|
||||
"""The login user and *name*'s passwd line, once *name* may become netOrk's.
|
||||
|
||||
:raises ValueError: as :meth:`add_service_account` says.
|
||||
"""
|
||||
check_account_name(name)
|
||||
if uid is not None and uid <= 0:
|
||||
raise ValueError(f"not a uid for netOrk's own account: {uid}")
|
||||
login = self._accounts_login()
|
||||
existing = self._accounts_passwd(name)
|
||||
if existing is not None and int(existing[2]) == 0:
|
||||
raise ValueError(f"{name} is root")
|
||||
if existing is not None and existing[6].rsplit("/", 1)[-1] in _NO_LOGIN_SHELLS:
|
||||
raise ValueError(
|
||||
f"{name} is there and cannot log in ({existing[6]}): another service's account"
|
||||
)
|
||||
return login, existing
|
||||
|
||||
def _accounts_service_account(
|
||||
self, name: str, passwd: list[str] | None, uid: int | None
|
||||
) -> FixStepOutcomeDict:
|
||||
"""netOrk's own account opened or made, a new one with *uid* when given."""
|
||||
if uid is None:
|
||||
return self._accounts_open_or_create(name, passwd, None)
|
||||
if passwd is not None:
|
||||
outcome = self._accounts_open_or_create(name, passwd, None)
|
||||
if outcome["ok"] and int(passwd[2]) != uid:
|
||||
outcome["output"] += f"; it keeps its uid {passwd[2]}, not {uid}"
|
||||
return outcome
|
||||
owner = self._accounts_passwd_of_uid(uid)
|
||||
if owner is not None:
|
||||
return {"ok": False, "output": f"uid {uid} is the account {owner}'s"}
|
||||
try:
|
||||
create = self._account_create_with_uid(name, uid)
|
||||
except NotImplementedError:
|
||||
return {"ok": False, "output": f"this driver cannot give an account the uid {uid}"}
|
||||
failed = self._accounts_run(create)
|
||||
if failed is not None:
|
||||
return {"ok": False, "output": _said(failed)}
|
||||
return {"ok": True, "output": f"account {name} created, uid {uid}"}
|
||||
|
||||
def _accounts_passwd_of_uid(self, uid: int) -> str | None:
|
||||
"""The name of the account with *uid*, or None."""
|
||||
result = self.run_command(_command(["getent", "passwd", str(uid)]), timeout=30)
|
||||
if result.exit_code == 2:
|
||||
return None
|
||||
if result.exit_code != 0:
|
||||
raise RuntimeError(f"cannot ask who has uid {uid}: {_said(result)}")
|
||||
return result.stdout.split(":", 1)[0].strip() or None
|
||||
|
||||
def _accounts_check_password(
|
||||
self, password_hash: str | None, *, admin: bool, clear: bool
|
||||
) -> None:
|
||||
""":raises ValueError: for a hash the host does not take, or a password to
|
||||
clear that is to be set, or that sudo would ask for."""
|
||||
if clear and password_hash is not None:
|
||||
raise ValueError(
|
||||
"a password to clear cannot be set too: clear_password or password_hash"
|
||||
)
|
||||
if clear and admin:
|
||||
raise ValueError("an account without a password cannot use sudo: its rule asks for it")
|
||||
if password_hash is not None:
|
||||
self._accounts_check_hash(password_hash)
|
||||
|
||||
def _accounts_check_hash(self, password_hash: str) -> None:
|
||||
prefixes = PASSWORD_SCHEMES[self.ACCOUNT_PASSWORD_SCHEME]
|
||||
if not _CRYPT_RE.fullmatch(password_hash) or not password_hash.startswith(prefixes):
|
||||
raise ValueError(f"not a {self.ACCOUNT_PASSWORD_SCHEME} password hash")
|
||||
|
||||
def _accounts_changeable(self, name: str) -> list[str] | None:
|
||||
"""*name*'s passwd line, None when there is none.
|
||||
|
||||
:raises ValueError: for the login user's own account or a system account.
|
||||
:raises RuntimeError: when the login user cannot be told.
|
||||
"""
|
||||
if name == self._accounts_login():
|
||||
raise ValueError(f"{name} is the account this driver logs in with")
|
||||
passwd = self._accounts_passwd(name)
|
||||
if passwd is not None and int(passwd[2]) < self.ACCOUNT_MIN_UID:
|
||||
raise ValueError(f"{name} is a system account (uid {passwd[2]})")
|
||||
return passwd
|
||||
|
||||
def _accounts_login(self) -> str:
|
||||
""":raises RuntimeError: when the login user cannot be told."""
|
||||
login = self.run_command("id -un", timeout=30).stdout.strip()
|
||||
if not login:
|
||||
raise RuntimeError("cannot tell which account this driver logs in with")
|
||||
return login
|
||||
|
||||
def _accounts_result(self, name: str, parts: list[FixResultDict]) -> LocalAccountChangeDict:
|
||||
steps = [s for part in parts for s in part["steps"]]
|
||||
result = _joined(steps)
|
||||
account: LocalAccountDict | None = None
|
||||
try:
|
||||
account = self._accounts_read(name)
|
||||
except RuntimeError as exc:
|
||||
result["output"] += f"\n[read back] {exc}"
|
||||
return {**result, "account": account}
|
||||
|
||||
def _accounts_root(self, argv: Sequence[str]) -> CommandResult:
|
||||
return self.run_command(_command(argv), privileged=True, timeout=60)
|
||||
|
||||
def _accounts_run(self, commands: list[list[str]]) -> CommandResult | None:
|
||||
"""Run *commands* as root in order; the first that failed, or None."""
|
||||
for argv in commands:
|
||||
result = self._accounts_root(argv)
|
||||
if result.exit_code != 0:
|
||||
return result
|
||||
return None
|
||||
|
||||
def _accounts_open_or_create(
|
||||
self, name: str, passwd: list[str] | None, password_hash: str | None
|
||||
) -> FixStepOutcomeDict:
|
||||
if passwd is None:
|
||||
failed = self._accounts_run(self._account_create(name, password_hash))
|
||||
done = f"account {name} created"
|
||||
else:
|
||||
locked = self._account_locked(name, passwd)
|
||||
failed = self._accounts_run(self._account_open(name, password_hash, locked=locked))
|
||||
done = f"account {name} opened again" if locked else f"account {name} is there"
|
||||
if failed is not None:
|
||||
return {"ok": False, "output": _said(failed)}
|
||||
return {"ok": True, "output": done + (", password set" if password_hash else "")}
|
||||
|
||||
def _accounts_clear_password(self, name: str) -> FixStepOutcomeDict:
|
||||
try:
|
||||
commands = self._account_clear_password(name)
|
||||
except NotImplementedError:
|
||||
return {"ok": False, "output": "this driver cannot remove a password"}
|
||||
failed = self._accounts_run(commands)
|
||||
if failed is not None:
|
||||
return {"ok": False, "output": _said(failed)}
|
||||
return {"ok": True, "output": f"{name} has no password: it logs in with a key only"}
|
||||
|
||||
def _accounts_lock(self, name: str) -> FixStepOutcomeDict:
|
||||
passwd = self._accounts_passwd(name)
|
||||
if passwd is None:
|
||||
return {"ok": False, "output": f"no account {name}"}
|
||||
locked = self._account_locked(name, passwd)
|
||||
failed = self._accounts_run(self._account_lock(name, locked=locked))
|
||||
if failed is not None:
|
||||
return {"ok": False, "output": _said(failed)}
|
||||
if not self._account_locked(name, passwd):
|
||||
return {"ok": False, "output": f"{name} is still not locked"}
|
||||
return {"ok": True, "output": f"{name} was locked already" if locked else f"{name} locked"}
|
||||
|
||||
# -- sudo --------------------------------------------------------------------------
|
||||
|
||||
def _accounts_set_sudo(
|
||||
self, name: str, *, admin: bool, locked: bool = False
|
||||
) -> FixStepOutcomeDict:
|
||||
path = sudoers_rule_path(self.SUDOERS_DIR, name)
|
||||
if admin:
|
||||
return self._accounts_grant_sudo(name, path)
|
||||
said: list[str] = []
|
||||
removed = self._accounts_root(["rm", "-f", "--", path])
|
||||
if removed.exit_code != 0:
|
||||
return {"ok": False, "output": f"{path}: {_said(removed)}"}
|
||||
groups = self.run_command(_command(["id", "-Gn", name]), timeout=30).stdout.split()
|
||||
for group in self.ACCOUNT_ADMIN_GROUPS:
|
||||
if group not in groups[1:]:
|
||||
continue
|
||||
left = self._accounts_root(self._account_leave_group(name, group, groups))
|
||||
if left.exit_code != 0:
|
||||
return {"ok": False, "output": f"cannot take {name} out of {group}: {_said(left)}"}
|
||||
groups.remove(group)
|
||||
said.append(f"{name} left {group}")
|
||||
if self._accounts_admin(name):
|
||||
rest = f"another rule than {path} still lets {name} run everything as root"
|
||||
if locked:
|
||||
return {"ok": True, "output": f"{rest}; the account is locked, so it cannot use it"}
|
||||
return {"ok": False, "output": rest}
|
||||
said.append(f"{name} has no sudo")
|
||||
return {"ok": True, "output": "\n".join(said)}
|
||||
|
||||
def _accounts_grant_sudo(self, name: str, path: str) -> FixStepOutcomeDict:
|
||||
failed = self._accounts_install_rule(sudoers_rule(name), path)
|
||||
if failed is not None:
|
||||
return failed
|
||||
if not self._accounts_admin(name):
|
||||
return {
|
||||
"ok": False,
|
||||
"output": f"{path} is written, but sudo does not read it: "
|
||||
f"does sudoers include {self.SUDOERS_DIR}?",
|
||||
}
|
||||
return {"ok": True, "output": f"{name} may use sudo with its password ({path})"}
|
||||
|
||||
def _accounts_grant_service_sudo(self, name: str) -> FixStepOutcomeDict:
|
||||
path = service_sudoers_rule_path(self.SUDOERS_DIR, name)
|
||||
failed = self._accounts_install_rule(service_sudoers_rule(name), path)
|
||||
if failed is not None:
|
||||
return failed
|
||||
result = self.run_command(_command(["sudo", "-l", "-U", name]), privileged=True, timeout=30)
|
||||
if result.exit_code != 0:
|
||||
return {"ok": False, "output": f"cannot ask sudo about {name}: {_said(result)}"}
|
||||
if not _RUN_ALL_NOPASSWD_RE.search(result.stdout):
|
||||
return {
|
||||
"ok": False,
|
||||
"output": f"{path} is written, but sudo does not read it: "
|
||||
f"does sudoers include {self.SUDOERS_DIR}?",
|
||||
}
|
||||
return {"ok": True, "output": f"{name} may use sudo without a password ({path})"}
|
||||
|
||||
def _accounts_install_rule(self, rule: str, path: str) -> FixStepOutcomeDict | None:
|
||||
"""Check *rule* with visudo and install it at *path*; the failure, or None."""
|
||||
staged = self._accounts_stage(rule)
|
||||
try:
|
||||
checked = self._accounts_root(["visudo", "-cf", staged])
|
||||
if checked.exit_code != 0:
|
||||
return {"ok": False, "output": f"visudo refuses the rule: {_said(checked)}"}
|
||||
installed = self._accounts_root(
|
||||
["install", "-m", "440", "-o", "root", "-g", self.ROOT_GROUP, "--", staged, path]
|
||||
)
|
||||
if installed.exit_code != 0:
|
||||
return {"ok": False, "output": f"{path}: {_said(installed)}"}
|
||||
finally:
|
||||
self._accounts_unstage(staged)
|
||||
return None
|
||||
|
||||
# -- keys --------------------------------------------------------------------------
|
||||
|
||||
def _accounts_stage(self, content: str) -> str:
|
||||
"""*content* in a fresh file of the login user's; its path.
|
||||
|
||||
:raises RuntimeError: when it cannot be staged.
|
||||
"""
|
||||
result = self.run_command(_STAGE, timeout=30, stdin=content.encode())
|
||||
path = result.stdout.strip().splitlines()[-1:] or [""]
|
||||
if result.exit_code != 0 or not path[0].startswith("/"):
|
||||
raise RuntimeError(f"cannot stage a file: {_said(result)}")
|
||||
return path[0]
|
||||
|
||||
def _accounts_unstage(self, path: str) -> None:
|
||||
self.run_command(_command(["rm", "-f", "--", path]), timeout=30)
|
||||
|
||||
def _accounts_home(self, name: str) -> str | None:
|
||||
passwd = self._accounts_passwd(name)
|
||||
return passwd[5] if passwd is not None else None
|
||||
|
||||
def _accounts_add_key(self, name: str, public_key: str) -> FixStepOutcomeDict:
|
||||
home = self._accounts_home(name)
|
||||
if home is None:
|
||||
return {"ok": False, "output": f"no account {name}"}
|
||||
return self._account_add_key(name, home, public_key)
|
||||
|
||||
def _accounts_remove_keys(self, name: str) -> FixStepOutcomeDict:
|
||||
home = self._accounts_home(name)
|
||||
if home is None:
|
||||
return {"ok": False, "output": f"no account {name}"}
|
||||
return self._account_remove_keys(name, home)
|
||||
|
||||
def _account_add_key(self, name: str, home: str, public_key: str) -> FixStepOutcomeDict:
|
||||
"""Add *public_key* to *name*'s first key file, as *name*; the others stay.
|
||||
|
||||
A driver whose host keeps keys elsewhere overrides this, with
|
||||
:meth:`_account_read_keys` and :meth:`_account_remove_keys`.
|
||||
"""
|
||||
path = authorized_keys_paths(home)[0]
|
||||
current = self._account_read_keys(name, home)
|
||||
if current is None:
|
||||
return {"ok": False, "output": f"cannot read the key files of {name}"}
|
||||
blob = _key_blob(public_key)
|
||||
if any(_key_blob(line) == blob for line in current):
|
||||
return {"ok": True, "output": f"the key is already in {path}"}
|
||||
mine = self._accounts_key_file(name, path)
|
||||
if mine is None:
|
||||
return {"ok": False, "output": f"cannot read {path}"}
|
||||
failed = self._accounts_write_key_file(name, home, path, [*mine, public_key.strip()])
|
||||
if failed is not None:
|
||||
return {"ok": False, "output": f"{path}: {_said(failed)}"}
|
||||
return {"ok": True, "output": f"key added to {path}"}
|
||||
|
||||
def _accounts_write_key_file(
|
||||
self, name: str, home: str, path: str, lines: list[str]
|
||||
) -> CommandResult | None:
|
||||
"""*lines* as *name*'s key file *path*, written as *name*; None, or what failed.
|
||||
|
||||
Without lines the file goes: nothing is staged from an empty stdin.
|
||||
"""
|
||||
if not lines:
|
||||
return self._accounts_run_as(name, [["rm", "-f", "--", path]])
|
||||
staged = self._accounts_stage("".join(f"{line}\n" for line in lines))
|
||||
try:
|
||||
return self._accounts_run_as(
|
||||
name,
|
||||
[
|
||||
["mkdir", "-p", "-m", "700", "--", f"{home}/.ssh"],
|
||||
["install", "-m", "600", "--", staged, path],
|
||||
],
|
||||
)
|
||||
finally:
|
||||
self._accounts_unstage(staged)
|
||||
|
||||
def _accounts_drop_from(
|
||||
self, name: str, home: str, path: str, blobs: set[str | None]
|
||||
) -> FixStepOutcomeDict:
|
||||
"""The keys of *blobs* out of *name*'s key file *path*; no output when none was in it."""
|
||||
lines = self._accounts_key_file(name, path)
|
||||
if lines is None:
|
||||
return {"ok": False, "output": f"cannot read {path}"}
|
||||
kept = [line for line in lines if _key_blob(line) not in blobs]
|
||||
if len(kept) == len(lines):
|
||||
return {"ok": True, "output": ""}
|
||||
failed = self._accounts_write_key_file(name, home, path, kept)
|
||||
if failed is not None:
|
||||
return {"ok": False, "output": f"{path}: {_said(failed)}"}
|
||||
return {"ok": True, "output": f"{len(lines) - len(kept)} removed from {path}"}
|
||||
|
||||
def _accounts_drop_keys(self, name: str, keys: list[str]) -> FixStepOutcomeDict:
|
||||
home = self._accounts_home(name)
|
||||
if home is None:
|
||||
return {"ok": False, "output": f"no account {name}"}
|
||||
return self._account_drop_keys(name, home, keys)
|
||||
|
||||
def _account_drop_keys(self, name: str, home: str, keys: list[str]) -> FixStepOutcomeDict:
|
||||
"""Remove exactly *keys* from *name*'s key files in *home*, as *name*.
|
||||
|
||||
A key is matched by its base64, whatever a line's options or comment;
|
||||
every other line stays. Afterwards none of *keys* may be left in
|
||||
:meth:`_account_key_files`. A driver whose host keeps keys elsewhere
|
||||
overrides this, with :meth:`_account_add_key`.
|
||||
"""
|
||||
blobs = {_key_blob(key) for key in keys}
|
||||
said: list[str] = []
|
||||
for path in authorized_keys_paths(home):
|
||||
outcome = self._accounts_drop_from(name, home, path, blobs)
|
||||
if not outcome["ok"]:
|
||||
return outcome
|
||||
said += [outcome["output"]] if outcome["output"] else []
|
||||
left = self._account_read_keys(name, home)
|
||||
if left is None:
|
||||
return {"ok": False, "output": f"cannot read the key files of {name}"}
|
||||
if any(_key_blob(line) in blobs for line in left):
|
||||
return {"ok": False, "output": f"{name} still has a key that was to go"}
|
||||
return {"ok": True, "output": "\n".join(said) or f"none of the keys is in {home}/.ssh"}
|
||||
|
||||
def _accounts_key_file(self, name: str, path: str) -> list[str] | None:
|
||||
there = self.run_command(_command(["test", "-e", path]), privileged=True, timeout=30)
|
||||
if there.exit_code != 0:
|
||||
return []
|
||||
result = self.run_command(_as(name, "cat", "--", path), privileged=True, timeout=30)
|
||||
if result.exit_code != 0:
|
||||
return None
|
||||
return [line for line in result.stdout.splitlines() if line.strip()]
|
||||
|
||||
def _accounts_run_as(self, name: str, commands: list[list[str]]) -> CommandResult | None:
|
||||
for argv in commands:
|
||||
result = self.run_command(_as(name, *argv), privileged=True, timeout=60)
|
||||
if result.exit_code != 0:
|
||||
return result
|
||||
return None
|
||||
|
||||
def _account_remove_keys(self, name: str, home: str) -> FixStepOutcomeDict:
|
||||
"""Remove *name*'s key files in *home*, as *name*; then no key may be left."""
|
||||
paths = authorized_keys_paths(home)
|
||||
failed = self._accounts_run_as(name, [["rm", "-f", "--", *paths]])
|
||||
if failed is not None:
|
||||
return {"ok": False, "output": f"cannot remove the keys of {name}: {_said(failed)}"}
|
||||
if self._account_read_keys(name, home):
|
||||
return {"ok": False, "output": f"{name} still has keys"}
|
||||
return {"ok": True, "output": f"keys of {name} removed"}
|
||||
|
||||
|
||||
def _joined(steps: list[FixStepDict]) -> FixResultDict:
|
||||
return {
|
||||
"success": bool(steps) and all(s["ok"] for s in steps),
|
||||
"steps": steps,
|
||||
"output": "\n".join(f"[{s['step']}] {s['output']}".rstrip() for s in steps),
|
||||
}
|
||||
|
||||
|
||||
__all__ = [
|
||||
"ACCOUNT_NAME_RE",
|
||||
"PASSWORD_SCHEMES",
|
||||
"SERVICE_SUDOERS_FILE_PREFIX",
|
||||
"SUDOERS_FILE_PREFIX",
|
||||
"LocalAccountChangeDict",
|
||||
"LocalAccountDict",
|
||||
"LocalAccountsMixin",
|
||||
"authorized_keys_paths",
|
||||
"check_account_name",
|
||||
"check_public_key",
|
||||
"service_sudoers_rule",
|
||||
"service_sudoers_rule_path",
|
||||
"sudoers_rule",
|
||||
"sudoers_rule_path",
|
||||
]
|
||||
@@ -1,4 +1,3 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""MAC-based access control, per SSID on an AP and per port on a switch.
|
||||
|
||||
The reader is identical either way; only the writer is AP-specific so far.
|
||||
@@ -10,7 +9,7 @@ this class in can never shadow a working implementation from a sibling base.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Dict, List, TYPE_CHECKING
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from napalm_device_types.models import MACACLDict
|
||||
|
||||
@@ -18,13 +17,13 @@ from napalm_device_types.models import MACACLDict
|
||||
class MacAclMixin:
|
||||
if TYPE_CHECKING:
|
||||
|
||||
def get_mac_acl(self) -> Dict[str, MACACLDict]:
|
||||
def get_mac_acl(self) -> dict[str, MACACLDict]:
|
||||
"""
|
||||
Returns the MAC-address-based access control lists configured per SSID.
|
||||
|
||||
Keys are SSID names. Each value contains:
|
||||
|
||||
* ssid (string) - SSID name (repeated for convenience)
|
||||
* name (string) - SSID name (repeated for convenience)
|
||||
* policy (string) - ACL mode:
|
||||
|
||||
* ``"allow"`` – whitelist: only listed MACs may associate
|
||||
@@ -44,22 +43,25 @@ class MacAclMixin:
|
||||
"name": "CorpWiFi",
|
||||
"policy": "allow",
|
||||
"entries": [
|
||||
{"mac": "AA:BB:CC:DD:EE:01", "action": "allow", "description": "CEO-Laptop"},
|
||||
{"mac": "AA:BB:CC:DD:EE:02", "action": "allow", "description": "CFO-Laptop"},
|
||||
{"mac": "AA:BB:CC:DD:EE:01", "action": "allow",
|
||||
"description": "CEO-Laptop"},
|
||||
{"mac": "AA:BB:CC:DD:EE:02", "action": "allow",
|
||||
"description": "CFO-Laptop"},
|
||||
],
|
||||
},
|
||||
"GuestNet": {
|
||||
"name": "GuestNet",
|
||||
"policy": "deny",
|
||||
"entries": [
|
||||
{"mac": "DE:AD:BE:EF:00:01", "action": "deny", "description": "blocked device"},
|
||||
{"mac": "DE:AD:BE:EF:00:01", "action": "deny",
|
||||
"description": "blocked device"},
|
||||
],
|
||||
},
|
||||
}
|
||||
"""
|
||||
...
|
||||
|
||||
def push_mac_acl(self, ssid_name: str, mode: str, macs: List[str]) -> None:
|
||||
def push_mac_acl(self, ssid_name: str, mode: str, macs: list[str]) -> None:
|
||||
"""
|
||||
Rewrites the MAC-address access control list for a single SSID.
|
||||
|
||||
@@ -73,7 +75,8 @@ class MacAclMixin:
|
||||
|
||||
Example::
|
||||
|
||||
driver.push_mac_acl("CorpWiFi", "whitelist", ["AA:BB:CC:DD:EE:01", "AA:BB:CC:DD:EE:02"])
|
||||
macs = ["AA:BB:CC:DD:EE:01", "AA:BB:CC:DD:EE:02"]
|
||||
driver.push_mac_acl("CorpWiFi", "whitelist", macs)
|
||||
driver.push_mac_acl("GuestNet", "off", [])
|
||||
"""
|
||||
...
|
||||
@@ -0,0 +1,151 @@
|
||||
"""
|
||||
Abstract base class for server baseboard management controllers (BMCs).
|
||||
|
||||
Usage::
|
||||
|
||||
from napalm_device_types import ManagementControllerDriver
|
||||
|
||||
class IdracDriver(ManagementControllerDriver):
|
||||
...
|
||||
"""
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from napalm_device_types.base import DeviceTypeDriver
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - for type checkers only
|
||||
from napalm_device_types.models import (
|
||||
EventLogEntryDict,
|
||||
FirmwareDict,
|
||||
HardwareInventoryDict,
|
||||
ManagementProtocolsDict,
|
||||
PowerAction,
|
||||
PowerStateDict,
|
||||
)
|
||||
|
||||
|
||||
class ManagementControllerDriver(DeviceTypeDriver):
|
||||
"""
|
||||
Abstract intermediate driver for the management controller of a server
|
||||
(e.g. Supermicro BMC, Dell iDRAC, HPE iLO, Lenovo XCC), reached over
|
||||
Redfish or IPMI rather than SSH.
|
||||
|
||||
A BMC is not the server it manages. ``reboot_host`` restarts the BMC
|
||||
itself (Redfish ``Manager.Reset``), never the managed server; the server is
|
||||
switched on, off and restarted through :meth:`set_power_state`.
|
||||
|
||||
The BMC's sensors (temperatures, fans, PSU power) come through NAPALM's
|
||||
``get_environment``, with ``cpu`` and ``memory`` empty: a BMC does not know
|
||||
the load of the operating system on its server. There is no
|
||||
``HealthMetricsMixin`` here -- that is SNMP's -- and
|
||||
``CommandChannelMixin`` stays unimplemented: a BMC offers no shell to run a
|
||||
command in.
|
||||
"""
|
||||
|
||||
#: Stable key netOrk exposes as ``device_class``. The order in which a
|
||||
#: driver lists its role bases is the ranking; see
|
||||
#: :func:`napalm_device_types.roles.primary_role_of`.
|
||||
ROLE: str = "management_controller"
|
||||
TYPE_LABEL: str = "Management Controller"
|
||||
USES_SSH: bool = False
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
||||
def get_power_state(self) -> PowerStateDict:
|
||||
"""
|
||||
Returns the managed server's power state.
|
||||
|
||||
* state (string) - ``"On"``, ``"Off"``, ``"PoweringOn"`` or ``"PoweringOff"``
|
||||
* actions (list) - the :data:`~napalm_device_types.models.PowerAction`
|
||||
values the BMC accepts right now
|
||||
|
||||
Example::
|
||||
|
||||
{"state": "On", "actions": ["GracefulShutdown", "ForceOff", "ForceRestart"]}
|
||||
"""
|
||||
...
|
||||
|
||||
def set_power_state(self, action: PowerAction) -> None:
|
||||
"""
|
||||
Switches, shuts down or restarts the managed server.
|
||||
|
||||
Returns once the BMC has accepted the request; the server takes its
|
||||
time. ``GracefulShutdown`` and ``GracefulRestart`` ask the operating
|
||||
system (ACPI) and may be ignored by it.
|
||||
|
||||
:param action: One of ``On``, ``GracefulShutdown``, ``ForceOff``,
|
||||
``GracefulRestart``, ``ForceRestart`` or ``PowerCycle``.
|
||||
:raises ValueError: If *action* is not one of them.
|
||||
:raises RuntimeError: If the BMC refuses it, e.g. ``On`` for a
|
||||
server that is already on.
|
||||
"""
|
||||
...
|
||||
|
||||
def get_hardware_inventory(self) -> HardwareInventoryDict:
|
||||
"""
|
||||
Returns the managed server's hardware: CPUs, DIMMs, drives, power
|
||||
supplies and NICs, with the server's serial number, its SMBIOS
|
||||
system UUID and the MAC address of each NIC -- what ties the BMC to
|
||||
the operating system running on the server.
|
||||
|
||||
Example::
|
||||
|
||||
{
|
||||
"manufacturer": "Dell Inc.",
|
||||
"model": "PowerEdge R650",
|
||||
"serial_number": "7XK2Q93",
|
||||
"system_uuid": "4c4c4544-0058-4b10-8032-b7c04f513933",
|
||||
"cpus": [{"socket": "CPU.Socket.1", "model": "Intel(R) Xeon(R) Silver 4314",
|
||||
"cores": 16, "threads": 32, "status": "OK"}],
|
||||
"memory": [...],
|
||||
"drives": [...],
|
||||
"power_supplies": [...],
|
||||
"nics": [{"name": "NIC.Integrated.1-1", "mac_address": "b0:7b:25:de:1a:40",
|
||||
"speed_mbps": 10000, "link_up": True}],
|
||||
}
|
||||
"""
|
||||
...
|
||||
|
||||
def get_event_log(self, limit: int = 500) -> list[EventLogEntryDict]:
|
||||
"""
|
||||
Returns the BMC's system event log (SEL, Lifecycle Log, IML),
|
||||
newest first.
|
||||
|
||||
:param limit: At most this many entries.
|
||||
"""
|
||||
...
|
||||
|
||||
def get_firmware_inventory(self) -> list[FirmwareDict]:
|
||||
"""
|
||||
Returns every firmware component the BMC reports: its own, the
|
||||
BIOS, storage controllers, NICs, power supplies.
|
||||
|
||||
Example::
|
||||
|
||||
[
|
||||
{"name": "Integrated Dell Remote Access Controller",
|
||||
"version": "7.00.00.172", "updateable": True},
|
||||
{"name": "BIOS", "version": "1.13.2", "updateable": True},
|
||||
]
|
||||
"""
|
||||
...
|
||||
|
||||
def get_management_protocols(self) -> ManagementProtocolsDict:
|
||||
"""
|
||||
Returns what the BMC itself listens on: IPMI, SSH, HTTP, HTTPS,
|
||||
SNMP, the KVM console and virtual media, each with whether it is
|
||||
enabled and its port.
|
||||
|
||||
Example::
|
||||
|
||||
{
|
||||
"ipmi": {"enabled": False, "port": 623},
|
||||
"ssh": {"enabled": True, "port": 22},
|
||||
"http": {"enabled": False, "port": 80},
|
||||
"https": {"enabled": True, "port": 443},
|
||||
"snmp": {"enabled": False, "port": 161},
|
||||
"kvm": {"enabled": True, "port": 5900},
|
||||
"virtual_media": {"enabled": True, "port": None},
|
||||
}
|
||||
"""
|
||||
...
|
||||
@@ -1,4 +1,3 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
Abstract base class for networked media players.
|
||||
|
||||
@@ -10,7 +9,7 @@ Usage::
|
||||
...
|
||||
"""
|
||||
|
||||
from typing import TYPE_CHECKING, Any, Dict, List
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from napalm_device_types.base import DeviceTypeDriver
|
||||
|
||||
@@ -34,7 +33,7 @@ class MediaDriver(DeviceTypeDriver):
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
||||
def get_playback_state(self) -> Dict[str, Any]:
|
||||
def get_playback_state(self) -> dict[str, Any]:
|
||||
"""
|
||||
Returns the transport state of the renderer.
|
||||
|
||||
@@ -51,7 +50,7 @@ class MediaDriver(DeviceTypeDriver):
|
||||
"""Sets the output volume, 0-100."""
|
||||
...
|
||||
|
||||
def get_zone_info(self) -> List[Dict[str, Any]]:
|
||||
def get_zone_info(self) -> list[dict[str, Any]]:
|
||||
"""
|
||||
Returns the zones or groups this renderer participates in.
|
||||
|
||||
|
||||
+685
-184
File diff suppressed because it is too large.
Load diff
@@ -1,4 +1,3 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Address translation and VPN tunnels.
|
||||
|
||||
A home gateway does a subset of what a firewall does, and these readers are
|
||||
@@ -11,7 +10,7 @@ this class in can never shadow a working implementation from a sibling base.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Dict, List, TYPE_CHECKING
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from napalm_device_types.models import NATTranslationDict, PortForwardDict, VPNTunnelDict
|
||||
|
||||
@@ -19,7 +18,7 @@ from napalm_device_types.models import NATTranslationDict, PortForwardDict, VPNT
|
||||
class NatVpnMixin:
|
||||
if TYPE_CHECKING:
|
||||
|
||||
def get_nat_translations(self) -> List[NATTranslationDict]:
|
||||
def get_nat_translations(self) -> list[NATTranslationDict]:
|
||||
"""
|
||||
Returns a list of active NAT translation entries.
|
||||
|
||||
@@ -47,7 +46,7 @@ class NatVpnMixin:
|
||||
"""
|
||||
...
|
||||
|
||||
def get_port_forwards(self) -> List[PortForwardDict]:
|
||||
def get_port_forwards(self) -> list[PortForwardDict]:
|
||||
"""
|
||||
Returns the port forwards that let traffic in from the WAN.
|
||||
|
||||
@@ -87,7 +86,7 @@ class NatVpnMixin:
|
||||
"""
|
||||
...
|
||||
|
||||
def get_vpn_tunnels(self) -> Dict[str, VPNTunnelDict]:
|
||||
def get_vpn_tunnels(self) -> dict[str, VPNTunnelDict]:
|
||||
"""
|
||||
Returns the status of VPN tunnels.
|
||||
|
||||
|
||||
+18
-136
@@ -10,23 +10,24 @@ Usage::
|
||||
...
|
||||
"""
|
||||
|
||||
from typing import List, Optional, TYPE_CHECKING
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from napalm_device_types.base import DeviceTypeDriver
|
||||
from napalm_device_types.updates import UpdateMixin
|
||||
from napalm_device_types.services import ServiceControlMixin
|
||||
from napalm_device_types.packages import PackageManagementMixin
|
||||
from napalm_device_types.health_metrics import HealthMetricsMixin
|
||||
from napalm_device_types.models import (
|
||||
CronJobDict,
|
||||
DeviceActionResultDict,
|
||||
DockerInfoDict,
|
||||
ProcessDict,
|
||||
SNMPConfigDict,
|
||||
UserDict,
|
||||
)
|
||||
from napalm_device_types.packages import PackageManagementMixin
|
||||
from napalm_device_types.services import ServiceControlMixin
|
||||
from napalm_device_types.updates import UpdateMixin
|
||||
|
||||
|
||||
class OSDriver(UpdateMixin, ServiceControlMixin, PackageManagementMixin, HealthMetricsMixin, DeviceTypeDriver):
|
||||
class OSDriver(
|
||||
UpdateMixin, ServiceControlMixin, PackageManagementMixin, HealthMetricsMixin, DeviceTypeDriver
|
||||
):
|
||||
"""
|
||||
Abstract intermediate driver for general-purpose operating systems
|
||||
(e.g. Linux, BSD, macOS).
|
||||
@@ -37,33 +38,28 @@ class OSDriver(UpdateMixin, ServiceControlMixin, PackageManagementMixin, HealthM
|
||||
|
||||
#: Stable key netOrk exposes as ``device_class``. The order in which a
|
||||
#: driver lists its role bases is the ranking; see
|
||||
#: :func:`napalm_device_types.roles.primary_role_of`.
|
||||
#: :func:`napalm_device_types.roles.primary_role_of`. A driver for another
|
||||
#: OS sets its own (``ROLE = "windows"``), which replaces this one.
|
||||
ROLE: str = "linux"
|
||||
TYPE_LABEL: str = "OS"
|
||||
|
||||
# Interfaces matching this pattern are excluded from health-metric collection.
|
||||
# Set True for drivers where the out-error counter actually reports drops (e.g. OPNsense).
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Package management
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
||||
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Service management
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Users
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def get_users(self) -> List[UserDict]:
|
||||
def get_os_users(self) -> list[UserDict]:
|
||||
"""
|
||||
Returns a list of local OS user accounts.
|
||||
|
||||
@@ -95,7 +91,7 @@ class OSDriver(UpdateMixin, ServiceControlMixin, PackageManagementMixin, HealthM
|
||||
# Processes
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def get_processes(self) -> List[ProcessDict]:
|
||||
def get_processes(self) -> list[ProcessDict]:
|
||||
"""
|
||||
Returns a snapshot of currently running processes.
|
||||
|
||||
@@ -134,39 +130,11 @@ class OSDriver(UpdateMixin, ServiceControlMixin, PackageManagementMixin, HealthM
|
||||
"""
|
||||
...
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Cron jobs
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def get_cron_jobs(self) -> List[CronJobDict]:
|
||||
"""
|
||||
Returns scheduled cron tasks from all user crontabs and ``/etc/cron.d``.
|
||||
|
||||
Each entry contains:
|
||||
|
||||
* user (string) - owner of the crontab entry
|
||||
* schedule (string) - five-field cron expression (e.g. ``"0 * * * *"``)
|
||||
* command (string) - shell command to execute
|
||||
* description (string, optional) - inline comment text if present
|
||||
|
||||
Example::
|
||||
|
||||
[
|
||||
{
|
||||
"user": "root",
|
||||
"schedule": "0 4 * * *",
|
||||
"command": "/usr/local/bin/backup.sh",
|
||||
"description": "nightly backup",
|
||||
},
|
||||
]
|
||||
"""
|
||||
...
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# SNMP
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def get_snmp_config(self) -> Optional[SNMPConfigDict]:
|
||||
def get_snmp_config(self) -> SNMPConfigDict | None:
|
||||
"""
|
||||
Returns the SNMP agent configuration currently active on the device,
|
||||
or ``None`` if no SNMP daemon is running or detectable.
|
||||
@@ -193,97 +161,6 @@ class OSDriver(UpdateMixin, ServiceControlMixin, PackageManagementMixin, HealthM
|
||||
"""
|
||||
...
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Docker
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def get_docker_info(self) -> DockerInfoDict:
|
||||
"""
|
||||
Returns information about the local Docker environment.
|
||||
|
||||
If Docker is not installed or the current user lacks access to the
|
||||
Docker socket, returns ``{"available": False}``. When the user has
|
||||
no socket permission, ``permission_denied`` is additionally set to
|
||||
``True``.
|
||||
|
||||
When Docker is available the dict contains:
|
||||
|
||||
* available (bool) - always ``True``
|
||||
* version (string) - Docker Engine version string
|
||||
* containers (list) - all containers (running and stopped), each with:
|
||||
|
||||
* id (string) - short container ID
|
||||
* name (string) - container name(s)
|
||||
* image (string) - image reference
|
||||
* image_version (string) - OCI ``org.opencontainers.image.version`` label; empty if absent
|
||||
* command (string) - entrypoint / command string
|
||||
* created (string) - creation timestamp string
|
||||
* status (string) - human-readable status (e.g. ``"Up 3 hours"``)
|
||||
* ports (string) - port mapping string
|
||||
* state (string) - ``"running"``, ``"exited"``, ``"paused"``, etc.
|
||||
|
||||
* images (list) - local images, each with:
|
||||
|
||||
* id (string) - short image ID
|
||||
* repository (string) - image repository
|
||||
* tag (string) - image tag
|
||||
* size (string) - human-readable size string (e.g. ``"187MB"``)
|
||||
* created (string) - creation timestamp string
|
||||
* version (string) - OCI ``org.opencontainers.image.version`` label; empty if absent
|
||||
|
||||
* volumes (list) - Docker volumes, each with:
|
||||
|
||||
* name (string) - volume name
|
||||
* driver (string) - volume driver
|
||||
* mountpoint (string) - host filesystem path
|
||||
* scope (string) - ``"local"`` or ``"global"``
|
||||
|
||||
* networks (list) - Docker networks, each with:
|
||||
|
||||
* id (string) - short network ID
|
||||
* name (string) - network name
|
||||
* driver (string) - network driver (e.g. ``"bridge"``, ``"host"``, ``"overlay"``)
|
||||
* scope (string) - network scope
|
||||
* ipv6 (string) - ``"true"`` if IPv6 is enabled
|
||||
* internal (string) - ``"true"`` if the network is internal
|
||||
|
||||
* outdated_images (list of strings) - image names where the local digest
|
||||
differs from the latest remote digest; empty list if all images are
|
||||
current or update checks could not be performed.
|
||||
|
||||
Example::
|
||||
|
||||
# Docker not installed:
|
||||
{"available": False}
|
||||
|
||||
# Docker installed, no socket permission:
|
||||
{"available": False, "permission_denied": True}
|
||||
|
||||
# Docker available:
|
||||
{
|
||||
"available": True,
|
||||
"version": "Docker version 27.3.1, build ce12230",
|
||||
"containers": [
|
||||
{
|
||||
"id": "a1b2c3d4e5f6",
|
||||
"name": "my-app",
|
||||
"image": "nginx:latest",
|
||||
"image_version": "1.27.0",
|
||||
"command": "nginx -g 'daemon off;'",
|
||||
"created": "2026-05-28 10:00:00 +0000 UTC",
|
||||
"status": "Up 3 days",
|
||||
"ports": "0.0.0.0:80->80/tcp",
|
||||
"state": "running",
|
||||
},
|
||||
],
|
||||
"images": [...],
|
||||
"volumes": [],
|
||||
"networks": [...],
|
||||
"outdated_images": ["nginx:latest"],
|
||||
}
|
||||
"""
|
||||
...
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Generic device actions
|
||||
# ------------------------------------------------------------------
|
||||
@@ -292,6 +169,11 @@ class OSDriver(UpdateMixin, ServiceControlMixin, PackageManagementMixin, HealthM
|
||||
"""
|
||||
Executes a named administrative action on the device.
|
||||
|
||||
**Deprecated** (docs/STRUCTURE.md): the actions become fix templates
|
||||
in :mod:`napalm_device_types.fixes`, whose values the caller supplies --
|
||||
``configure_snmp_agent``, ``configure_package_proxy``,
|
||||
``grant_container_engine_access``, ``ensure_package``. Goes in 5.0.
|
||||
|
||||
This method is an extensibility point for driver-specific one-off
|
||||
operations that do not fit any other NAPALM API method. Each driver
|
||||
documents the action names it supports.
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
"""A device's OSPF state, in neutral terms.
|
||||
|
||||
The routing daemon's own output (FRR, Quagga, a vendor's CLI) is the driver's
|
||||
to translate; whether an interface without neighbours or without
|
||||
authentication is a problem is the caller's to decide.
|
||||
Declared only; a driver whose device runs OSPF implements it.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from napalm_device_types.models import OspfDict
|
||||
|
||||
|
||||
class OspfMixin:
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def get_ospf(self) -> OspfDict:
|
||||
"""Router ID, areas and OSPF per interface.
|
||||
|
||||
Example::
|
||||
|
||||
{"router_id": "10.255.0.1", "areas": ["0.0.0.0"],
|
||||
"interfaces": {"wg0": {"address": "10.255.0.1", "prefix_length": 24,
|
||||
"area": "0.0.0.0", "network_type": "POINTOPOINT",
|
||||
"authentication": "MessageDigest", "neighbors": 1,
|
||||
"neighbors_adjacent": 1, "cost": 10}}}
|
||||
"""
|
||||
...
|
||||
@@ -0,0 +1,120 @@
|
||||
"""Pending package updates: which package, from where, and whether it is a security fix.
|
||||
|
||||
A patch deadline -- "security updates within 14 days" -- needs to know which
|
||||
pending update is a security update. apt says so in the suite a candidate comes
|
||||
from (``noble-security``, ``stable-security``), dnf in its update advisories.
|
||||
Reading that is the same for every driver whose host runs apt or dnf, so the
|
||||
parsers live here once and a driver only carries the command across.
|
||||
|
||||
apt: the suites a candidate comes from are its ``origin``; any suite ending in
|
||||
``-security`` makes it a security update. A security fix that a later
|
||||
``-updates`` build superseded shows only ``-updates`` and counts as not
|
||||
security -- netOrk's CVE matching is what catches those.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
|
||||
from napalm_device_types.models import UpdateDict
|
||||
from napalm_device_types.terminal import strip_terminal_codes
|
||||
|
||||
#: Read-only, no root needed, in a fixed language so the parse holds, and
|
||||
#: through a pipe: without a terminal apt draws no progress and no terminal
|
||||
#: codes, one of which (``ESC >``) a screen-scraping transport took for a shell
|
||||
#: prompt and stopped reading at. Its exit status is printed inside the group,
|
||||
#: so it is apt's, not cat's.
|
||||
APT_UPGRADABLE_COMMAND = "{ LC_ALL=C apt list --upgradable 2>/dev/null; echo __APT_RC=$?; } | cat"
|
||||
|
||||
#: Read-only. Lists the packages that a pending security advisory covers. A
|
||||
#: failed dnf prints nothing, the same as no advisories, so its exit status is
|
||||
#: printed inside the group, as apt's is. It reads the cached metadata only, as
|
||||
#: apt reads its local index: an expired cache otherwise makes dnf download the
|
||||
#: repositories' metadata first, which can take minutes (#76).
|
||||
DNF_SECURITY_COMMAND = (
|
||||
"{ LC_ALL=C dnf --cacheonly updateinfo list --security --quiet 2>/dev/null;"
|
||||
" echo __DNF_RC=$?; } | cat"
|
||||
)
|
||||
|
||||
# openssl/noble-updates,noble-security 3.0.13-0ubuntu3.6 amd64 [upgradable from: 3.0.13-0ubuntu3.5]
|
||||
_APT_LINE = re.compile(r"^(\S+)/(\S+)\s+(\S+)\s+\S+\s+\[upgradable from:\s+(\S+)\]")
|
||||
_SECURITY_SUITE = "-security"
|
||||
_APT_STATUS = re.compile(r"^__APT_RC=(\d+)\s*$", re.MULTILINE)
|
||||
_DNF_STATUS = re.compile(r"^__DNF_RC=(\d+)\s*$", re.MULTILINE)
|
||||
|
||||
|
||||
def _joined_lines(output: str) -> list[str]:
|
||||
"""Lines as apt printed them: a terminal wraps long ones, and the
|
||||
continuation starts with a space."""
|
||||
lines: list[str] = []
|
||||
for line in output.splitlines():
|
||||
if line.startswith(" ") and lines:
|
||||
lines[-1] += line[1:].rstrip()
|
||||
else:
|
||||
lines.append(line)
|
||||
return lines
|
||||
|
||||
|
||||
def _listing(output: str, status: re.Pattern[str], what: str) -> str:
|
||||
"""The listing without its exit status, or ``ValueError`` when the command
|
||||
failed or the output was cut short -- "could not read" must never look like
|
||||
"nothing pending"."""
|
||||
text = strip_terminal_codes(output)
|
||||
statuses = status.findall(text)
|
||||
if not statuses:
|
||||
raise ValueError(f"{what} reported no exit status; the output was cut short")
|
||||
if statuses[-1] != "0":
|
||||
raise ValueError(f"{what} failed with exit status {statuses[-1]}")
|
||||
return status.sub("", text)
|
||||
|
||||
|
||||
def parse_apt_upgradable(output: str) -> list[UpdateDict]:
|
||||
"""Parse :data:`APT_UPGRADABLE_COMMAND`'s output, one entry per package.
|
||||
|
||||
A package listed for several architectures (``libc6`` for amd64 and i386)
|
||||
is one entry; it counts as a security update if any of its lines does.
|
||||
|
||||
:raises ValueError: when apt failed or its exit status never arrived.
|
||||
"""
|
||||
by_name: dict[str, UpdateDict] = {}
|
||||
for line in _joined_lines(_listing(output, _APT_STATUS, "apt list --upgradable")):
|
||||
match = _APT_LINE.match(line)
|
||||
if not match:
|
||||
continue
|
||||
name, listed, new_version, current_version = match.groups()
|
||||
suites = list(dict.fromkeys(listed.split(","))) # apt may list a suite twice
|
||||
security = any(suite.endswith(_SECURITY_SUITE) for suite in suites)
|
||||
seen = by_name.get(name)
|
||||
if seen is not None:
|
||||
seen["security"] = bool(seen.get("security")) or security
|
||||
continue
|
||||
by_name[name] = {
|
||||
"name": name,
|
||||
"current_version": current_version,
|
||||
"new_version": new_version,
|
||||
"origin": ",".join(suites),
|
||||
"security": security,
|
||||
}
|
||||
return list(by_name.values())
|
||||
|
||||
|
||||
def nevra_name(nevra: str) -> str:
|
||||
"""The package name of an RPM ``name-[epoch:]version-release.arch``."""
|
||||
without_arch = nevra.rsplit(".", 1)[0]
|
||||
return without_arch.rsplit("-", 2)[0]
|
||||
|
||||
|
||||
def parse_dnf_security(output: str) -> set[str]:
|
||||
"""The names of the packages a pending security advisory covers.
|
||||
|
||||
Parses :data:`DNF_SECURITY_COMMAND`'s ``ADVISORY SEVERITY/Sec. NEVRA`` lines.
|
||||
|
||||
:raises ValueError: when dnf failed or its exit status never arrived -- every
|
||||
update would otherwise read as not security.
|
||||
"""
|
||||
names: set[str] = set()
|
||||
for line in _listing(output, _DNF_STATUS, "dnf updateinfo list --security").splitlines():
|
||||
parts = line.split()
|
||||
if len(parts) >= 3 and parts[1].endswith("/Sec."):
|
||||
names.add(nevra_name(parts[-1]))
|
||||
return names
|
||||
@@ -1,4 +1,3 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Listing and managing installed software.
|
||||
|
||||
Identical on every device type that has a package manager -- this was five
|
||||
@@ -12,7 +11,7 @@ this class in can never shadow a working implementation from a sibling base.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any, Dict, List, TYPE_CHECKING
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from napalm_device_types.models import PackageDict
|
||||
|
||||
@@ -20,7 +19,7 @@ from napalm_device_types.models import PackageDict
|
||||
class PackageManagementMixin:
|
||||
if TYPE_CHECKING:
|
||||
|
||||
def get_packages(self) -> List[PackageDict]:
|
||||
def get_packages(self) -> list[PackageDict]:
|
||||
"""
|
||||
Returns all packages currently known to the device's package manager
|
||||
(e.g. ``opkg`` on OpenWrt, ``apk`` on Alpine-based APs).
|
||||
@@ -68,7 +67,6 @@ class PackageManagementMixin:
|
||||
:param name: Package name as known to the package manager.
|
||||
:param version: Exact version to install. An empty string (default)
|
||||
installs the latest available version.
|
||||
:raises NotImplementedError: If the driver does not support package management.
|
||||
:raises ValueError: If the package name is unknown or the version does
|
||||
not exist in any configured feed.
|
||||
:raises RuntimeError: If the installation fails on the device side
|
||||
@@ -88,7 +86,6 @@ class PackageManagementMixin:
|
||||
The method blocks until the removal is complete.
|
||||
|
||||
:param name: Package name to remove.
|
||||
:raises NotImplementedError: If the driver does not support package management.
|
||||
:raises ValueError: If the package is not currently installed.
|
||||
:raises RuntimeError: If the removal fails on the device side
|
||||
(e.g. other packages depend on it).
|
||||
@@ -99,13 +96,12 @@ class PackageManagementMixin:
|
||||
"""
|
||||
...
|
||||
|
||||
def get_package_config(self, name: str) -> Dict[str, Any]:
|
||||
def get_package_config(self, name: str) -> dict[str, Any]:
|
||||
"""
|
||||
Returns the current configuration of an installed package as a
|
||||
dictionary. The structure is package-specific.
|
||||
|
||||
:param name: Package name.
|
||||
:raises NotImplementedError: If the driver does not support package management.
|
||||
:raises ValueError: If the package is not installed.
|
||||
|
||||
Example::
|
||||
@@ -126,7 +122,7 @@ class PackageManagementMixin:
|
||||
"""
|
||||
...
|
||||
|
||||
def set_package_config(self, name: str, config: Dict[str, Any]) -> None:
|
||||
def set_package_config(self, name: str, config: dict[str, Any]) -> None:
|
||||
"""
|
||||
Writes a new configuration for an installed package.
|
||||
|
||||
@@ -140,7 +136,6 @@ class PackageManagementMixin:
|
||||
|
||||
:param name: Package name.
|
||||
:param config: New configuration as a nested dictionary.
|
||||
:raises NotImplementedError: If the driver does not support package management.
|
||||
:raises ValueError: If the package is not installed or the configuration
|
||||
contains invalid values.
|
||||
:raises RuntimeError: If the device rejects the configuration.
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
Abstract base class for IP telephony endpoints.
|
||||
|
||||
@@ -10,7 +9,7 @@ Usage::
|
||||
...
|
||||
"""
|
||||
|
||||
from typing import TYPE_CHECKING, Any, Dict, List
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from napalm_device_types.base import DeviceTypeDriver
|
||||
|
||||
@@ -36,7 +35,7 @@ class PhoneDriver(DeviceTypeDriver):
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
||||
def get_sip_accounts(self) -> List[Dict[str, Any]]:
|
||||
def get_sip_accounts(self) -> list[dict[str, Any]]:
|
||||
"""
|
||||
Returns the SIP registrations configured on the phone.
|
||||
|
||||
@@ -60,7 +59,7 @@ class PhoneDriver(DeviceTypeDriver):
|
||||
"""
|
||||
...
|
||||
|
||||
def get_call_status(self) -> Dict[str, Any]:
|
||||
def get_call_status(self) -> dict[str, Any]:
|
||||
"""
|
||||
Returns what the phone is doing right now.
|
||||
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Generic ICMP sweep on top of the NAPALM-standard ``ping()``.
|
||||
|
||||
Sweeping a range of addresses is orchestration, not device mechanics: the
|
||||
@@ -21,7 +20,8 @@ target list (see ``PING_SWEEP_MAX_TARGETS``).
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING, Any, Callable, ClassVar, Dict, Iterable, List, Optional
|
||||
from collections.abc import Callable, Iterable
|
||||
from typing import TYPE_CHECKING, Any, ClassVar
|
||||
|
||||
from napalm.base import NetworkDriver
|
||||
|
||||
@@ -60,7 +60,7 @@ class PingSweepMixin:
|
||||
|
||||
#: ``False`` opts a driver out of ping sweeps even though it implements
|
||||
#: ``ping()``. ``None`` (the default) means "decide by introspection".
|
||||
SUPPORTS_PING: ClassVar[Optional[bool]] = None
|
||||
SUPPORTS_PING: ClassVar[bool | None] = None
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
@@ -73,7 +73,7 @@ class PingSweepMixin:
|
||||
size: int = 100,
|
||||
count: int = 5,
|
||||
vrf: str = "",
|
||||
) -> Dict[str, Any]: ...
|
||||
) -> dict[str, Any]: ...
|
||||
|
||||
@classmethod
|
||||
def supports_ping(cls) -> bool:
|
||||
@@ -86,9 +86,9 @@ class PingSweepMixin:
|
||||
*,
|
||||
count: int = 1,
|
||||
timeout: int = 1,
|
||||
max_targets: Optional[int] = None,
|
||||
on_progress: Optional[Callable[[int, int], None]] = None,
|
||||
should_stop: Optional[Callable[[], bool]] = None,
|
||||
max_targets: int | None = None,
|
||||
on_progress: Callable[[int, int], None] | None = None,
|
||||
should_stop: Callable[[], bool] | None = None,
|
||||
) -> PingSweepResultDict:
|
||||
"""Ping every address in *destinations* and report who answered.
|
||||
|
||||
@@ -115,7 +115,7 @@ class PingSweepMixin:
|
||||
targets = targets[:limit]
|
||||
|
||||
total = len(targets)
|
||||
entries: List[PingSweepEntryDict] = []
|
||||
entries: list[PingSweepEntryDict] = []
|
||||
for done, destination in enumerate(targets, start=1):
|
||||
if should_stop is not None and should_stop():
|
||||
break
|
||||
@@ -150,6 +150,13 @@ class PingSweepMixin:
|
||||
return {"ip": destination, "alive": False, "rtt_ms": None, "error": error}
|
||||
|
||||
success = reply.get("success") or {}
|
||||
if not isinstance(success, dict):
|
||||
return {
|
||||
"ip": destination,
|
||||
"alive": False,
|
||||
"rtt_ms": None,
|
||||
"error": "malformed ping reply",
|
||||
}
|
||||
probes_sent = _as_int(success.get("probes_sent"))
|
||||
packet_loss = _as_int(success.get("packet_loss"), default=probes_sent)
|
||||
alive = bool(success.get("results")) or probes_sent > packet_loss
|
||||
@@ -165,7 +172,7 @@ def _as_int(value: Any, default: int = 0) -> int:
|
||||
return default
|
||||
|
||||
|
||||
def _as_float(value: Any) -> Optional[float]:
|
||||
def _as_float(value: Any) -> float | None:
|
||||
try:
|
||||
return float(value)
|
||||
except (TypeError, ValueError):
|
||||
|
||||
@@ -0,0 +1,89 @@
|
||||
"""
|
||||
What provisioning a VM from a cloud image needs to know about the guest.
|
||||
|
||||
The same for every hypervisor: which agent a guest runs so a QEMU-based
|
||||
hypervisor can read its IP, the network-config cloud-init and FreeBSD's
|
||||
nuageinit both read, how a packed cloud image is unpacked, and how a failure
|
||||
names the VM it left behind. How a hypervisor attaches, boots and talks to
|
||||
the VM stays in its driver.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Mapping
|
||||
from typing import Any
|
||||
|
||||
#: ``{guest_os: (packages, runcmd)}``: what cloud-init installs and runs so
|
||||
#: the hypervisor can read the guest's IP address back.
|
||||
GuestAgents = Mapping[str, tuple[tuple[str, ...], tuple[str, ...]]]
|
||||
|
||||
#: The QEMU guest agent per guest OS, for hypervisors built on QEMU. Package
|
||||
#: names and service commands were verified on FreeBSD 15.1 and OpenBSD 7.9
|
||||
#: cloud images (NetOrk/netork#793).
|
||||
QEMU_GUEST_AGENTS: GuestAgents = {
|
||||
"linux": (("qemu-guest-agent",), ("systemctl enable --now qemu-guest-agent",)),
|
||||
"freebsd": (
|
||||
("qemu-guest-agent",),
|
||||
("sysrc qemu_guest_agent_enable=YES", "service qemu-guest-agent start"),
|
||||
),
|
||||
"openbsd": (("qemu-ga",), ("rcctl enable qemu_ga", "rcctl start qemu_ga")),
|
||||
}
|
||||
|
||||
#: Suffix of a packed image -> the command that writes it unpacked to stdout.
|
||||
_DECOMPRESSORS = {
|
||||
"xz": "xz -dc",
|
||||
"gz": "gzip -dc",
|
||||
"bz2": "bzip2 -dc",
|
||||
"zst": "zstd -dc",
|
||||
}
|
||||
|
||||
|
||||
class VMProvisionIncomplete(RuntimeError):
|
||||
"""``create_vm_from_cloud_init`` failed after the VM was created.
|
||||
|
||||
The VM is still on the hypervisor, half built: ``vmid`` and ``node`` name it
|
||||
as ``VMProvisionResultDict`` would have, so the caller can remove it with
|
||||
``destroy_vm``. The failure itself is the ``__cause__`` (``raise ... from``),
|
||||
and the message is its own. A failure before the VM exists raises as before.
|
||||
"""
|
||||
|
||||
def __init__(self, message: str, *, vmid: str, node: str) -> None:
|
||||
super().__init__(message)
|
||||
self.vmid = vmid
|
||||
self.node = node
|
||||
|
||||
def __reduce__(self) -> tuple[Any, ...]:
|
||||
# The keyword-only arguments are not in ``args``, which is all that
|
||||
# pickle passes back by default.
|
||||
return (_vm_provision_incomplete, (str(self), self.vmid, self.node))
|
||||
|
||||
|
||||
def _vm_provision_incomplete(message: str, vmid: str, node: str) -> VMProvisionIncomplete:
|
||||
return VMProvisionIncomplete(message, vmid=vmid, node=node)
|
||||
|
||||
|
||||
def network_config(nics: list[tuple[str, bool]]) -> dict[str, Any] | None:
|
||||
"""Network-config v2: DHCP on every ``(mac, dhcp)`` NIC that asks for it.
|
||||
|
||||
Version 2 because FreeBSD's nuageinit reads no other: given the v1 that
|
||||
Proxmox generates, it fails and skips the rest of its first stage,
|
||||
runcmd included. ``None`` when no NIC wants DHCP, so the guest is left
|
||||
alone rather than told to configure nothing.
|
||||
"""
|
||||
ethernets = {
|
||||
f"nic{i}": {"match": {"macaddress": mac.lower()}, "dhcp4": True}
|
||||
for i, (mac, dhcp) in enumerate(nics)
|
||||
if dhcp
|
||||
}
|
||||
return {"version": 2, "ethernets": ethernets} if ethernets else None
|
||||
|
||||
|
||||
def split_compression(filename: str) -> tuple[str, str | None]:
|
||||
"""``(unpacked filename, command that unpacks to stdout)`` for *filename*.
|
||||
|
||||
The command is None for an image that is not packed, which is then
|
||||
imported as it is.
|
||||
"""
|
||||
stem, dot, suffix = filename.rpartition(".")
|
||||
command = _DECOMPRESSORS.get(suffix.lower()) if dot and stem else None
|
||||
return (stem, command) if command else (filename, None)
|
||||
Whitespace-only changes.
@@ -19,11 +19,11 @@ Usage::
|
||||
...
|
||||
"""
|
||||
|
||||
from typing import Dict, List, TYPE_CHECKING
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from napalm_device_types.base import DeviceTypeDriver
|
||||
from napalm_device_types.nat_vpn import NatVpnMixin
|
||||
from napalm_device_types.health_metrics import HealthMetricsMixin
|
||||
from napalm_device_types.dhcp import DhcpServerMixin
|
||||
from napalm_device_types.health_metrics import HealthMetricsMixin
|
||||
from napalm_device_types.models import (
|
||||
HostDict,
|
||||
RadioStatusDict,
|
||||
@@ -31,6 +31,7 @@ from napalm_device_types.models import (
|
||||
WANStatusDict,
|
||||
WirelessClientDict,
|
||||
)
|
||||
from napalm_device_types.nat_vpn import NatVpnMixin
|
||||
|
||||
|
||||
class ResidentialGatewayDriver(NatVpnMixin, HealthMetricsMixin, DhcpServerMixin, DeviceTypeDriver):
|
||||
@@ -48,8 +49,6 @@ class ResidentialGatewayDriver(NatVpnMixin, HealthMetricsMixin, DhcpServerMixin,
|
||||
ROLE: str = "residential_gateway"
|
||||
TYPE_LABEL: str = "Gateway"
|
||||
|
||||
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
||||
def get_wan_status(self) -> WANStatusDict:
|
||||
@@ -85,7 +84,7 @@ class ResidentialGatewayDriver(NatVpnMixin, HealthMetricsMixin, DhcpServerMixin,
|
||||
"""
|
||||
...
|
||||
|
||||
def get_hosts(self) -> List[HostDict]:
|
||||
def get_hosts(self) -> list[HostDict]:
|
||||
"""
|
||||
Returns the list of hosts known to the gateway (LAN clients).
|
||||
|
||||
@@ -113,9 +112,7 @@ class ResidentialGatewayDriver(NatVpnMixin, HealthMetricsMixin, DhcpServerMixin,
|
||||
"""
|
||||
...
|
||||
|
||||
|
||||
|
||||
def get_wireless_clients(self) -> List[WirelessClientDict]:
|
||||
def get_wireless_clients(self) -> list[WirelessClientDict]:
|
||||
"""
|
||||
Returns the list of wireless clients currently associated with the
|
||||
device's built-in access point(s).
|
||||
@@ -125,7 +122,7 @@ class ResidentialGatewayDriver(NatVpnMixin, HealthMetricsMixin, DhcpServerMixin,
|
||||
"""
|
||||
...
|
||||
|
||||
def get_ssids(self) -> Dict[str, SSIDDict]:
|
||||
def get_ssids(self) -> dict[str, SSIDDict]:
|
||||
"""
|
||||
Returns the configured wireless networks (SSIDs).
|
||||
|
||||
@@ -134,7 +131,7 @@ class ResidentialGatewayDriver(NatVpnMixin, HealthMetricsMixin, DhcpServerMixin,
|
||||
"""
|
||||
...
|
||||
|
||||
def get_radio_status(self) -> Dict[str, RadioStatusDict]:
|
||||
def get_radio_status(self) -> dict[str, RadioStatusDict]:
|
||||
"""
|
||||
Returns the status of the device's wireless radios.
|
||||
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Which roles a driver class fills, and which of them is primary.
|
||||
|
||||
A *role* is what a device is: a firewall, a switch, a NAS. Real devices are
|
||||
@@ -22,7 +21,7 @@ when a concrete driver provided it.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any, List, Optional, Type
|
||||
from typing import Any
|
||||
|
||||
|
||||
def _is_role_base(cls: type) -> bool:
|
||||
@@ -36,21 +35,25 @@ def _is_role_base(cls: type) -> bool:
|
||||
return isinstance(role, str) and bool(role)
|
||||
|
||||
|
||||
def roles_of(driver_cls: type) -> List[Type[Any]]:
|
||||
def roles_of(driver_cls: type) -> list[type[Any]]:
|
||||
"""Every role base in ``driver_cls``'s MRO, most significant first.
|
||||
|
||||
MRO order is inheritance order, so the list reflects exactly what the driver
|
||||
author wrote in the class definition.
|
||||
author wrote in the class definition. A class that sets ``ROLE`` itself
|
||||
replaces the role it inherits rather than adding one: a Windows driver on
|
||||
:class:`~napalm_device_types.os.OSDriver` sets ``ROLE = "windows"`` and is
|
||||
not also ``"linux"``.
|
||||
"""
|
||||
return [cls for cls in driver_cls.__mro__ if _is_role_base(cls)]
|
||||
roles = [cls for cls in driver_cls.__mro__ if _is_role_base(cls)]
|
||||
return [cls for cls in roles if not any(o is not cls and issubclass(o, cls) for o in roles)]
|
||||
|
||||
|
||||
def role_keys_of(driver_cls: type) -> List[str]:
|
||||
def role_keys_of(driver_cls: type) -> list[str]:
|
||||
"""The stable string keys of :func:`roles_of`, e.g. ``["storage", "linux"]``."""
|
||||
return [vars(cls)["ROLE"] for cls in roles_of(driver_cls)]
|
||||
|
||||
|
||||
def primary_role_of(driver_cls: type) -> Optional[str]:
|
||||
def primary_role_of(driver_cls: type) -> str | None:
|
||||
"""The driver's headline role, or ``None`` when it declares no role at all.
|
||||
|
||||
This is the value netOrk exposes as ``device_class``.
|
||||
|
||||
@@ -0,0 +1,196 @@
|
||||
"""Scheduled jobs: what a host runs on its own schedule, and switching it on, off and now.
|
||||
|
||||
A host runs work on its own: cron lines, run-parts scripts, anacron, systemd
|
||||
timers, FreeBSD's ``periodic``, a firewall's or a NAS's own scheduled jobs, the
|
||||
Windows Task Scheduler. :class:`ScheduledJobsMixin` reads all of them as one
|
||||
shape, :class:`~napalm_device_types.models.ScheduledJobDict`, and acts on them.
|
||||
Declared under ``if TYPE_CHECKING`` like a role: nothing exists at runtime until
|
||||
a driver implements it. :class:`~napalm_device_types.scheduled_jobs_posix.PosixScheduledJobsMixin`
|
||||
is the implementation for a host reached through the command channel.
|
||||
|
||||
**Who owns a definition** is on every job (``managed_by``): ``package`` (installed
|
||||
by one), ``admin`` (written on the host), ``platform`` (generated by the device's
|
||||
own configuration -- OPNsense, OpenMediaVault, QTS, Windows), ``application`` (written
|
||||
through :meth:`put_scheduled_job`, under the ``name`` the application gave it).
|
||||
What the application makes of that is the application's.
|
||||
|
||||
**What may be done is on every job too** (``actions``): every job a driver can
|
||||
switch takes ``enable``, ``disable`` and ``run``; only an ``application`` job takes
|
||||
``edit`` and ``delete``. A driver never edits or removes a job it did not write.
|
||||
|
||||
**Never over a change it did not see.** Every write names the ``digest`` the caller
|
||||
saw. The driver reads the job again right before it writes and raises
|
||||
:class:`ScheduledJobChanged` when the definition differs.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import re
|
||||
from typing import TYPE_CHECKING, Any, NotRequired, TypedDict
|
||||
|
||||
from napalm_device_types.cron import is_reboot, parse_cron
|
||||
|
||||
#: What :meth:`ScheduledJobsMixin.manage_scheduled_job` takes.
|
||||
ACTIONS = ("enable", "disable", "run")
|
||||
|
||||
#: What a job's ``actions`` may list beyond :data:`ACTIONS`.
|
||||
EDIT = "edit"
|
||||
DELETE = "delete"
|
||||
|
||||
KINDS = ("cron", "run_parts", "anacron", "systemd_timer", "periodic", "task")
|
||||
MANAGED_BY = ("package", "admin", "platform", "application")
|
||||
|
||||
#: Seconds a run now may take on the host before ``timeout`` stops it.
|
||||
RUN_TIMEOUT = 300
|
||||
|
||||
#: A job's name becomes a file and a unit name: lower case, digits, dashes.
|
||||
_NAME = re.compile(r"[a-z0-9][a-z0-9-]{0,62}")
|
||||
#: A user name as useradd and the BSDs take it.
|
||||
_USER = re.compile(r"[A-Za-z_][A-Za-z0-9_.-]{0,31}\$?")
|
||||
|
||||
#: The parts of a job that are its definition; what the host says of its runs is not.
|
||||
_DEFINITION = ("kind", "source", "run_as", "schedule", "command", "enabled")
|
||||
|
||||
|
||||
class ScheduledJobDict(TypedDict):
|
||||
"""A job the host runs on its own schedule (``get_scheduled_jobs``).
|
||||
|
||||
A cron line, a run-parts script, an anacron job, a systemd timer, a FreeBSD
|
||||
``periodic`` script, a platform's own job, a Windows task -- one shape for all
|
||||
of them (``napalm_device_types.scheduled_jobs``).
|
||||
"""
|
||||
|
||||
key: str # stable name on this host: "timer:apt-daily.timer", "cron:/etc/crontab:1a2b3c4d"
|
||||
kind: str # cron | run_parts | anacron | systemd_timer | periodic | task
|
||||
source: str # the file, unit file or configuration it is defined in
|
||||
run_as: str # the user it runs as
|
||||
schedule: str # as the host has it: "*/5 * * * *", "OnCalendar=daily", "daily"
|
||||
schedule_kind: str # cron | calendar | monotonic | period | triggers
|
||||
command: str # as the host has it
|
||||
enabled: bool
|
||||
managed_by: str # package | admin | platform | application
|
||||
digest: str # job_digest(): the definition as it is now
|
||||
actions: list[str] # what the driver can do to this job: enable, disable, run, edit, delete
|
||||
name: NotRequired[str] # the name an application gave a job it wrote (put_scheduled_job)
|
||||
description: NotRequired[str]
|
||||
last_run: NotRequired[str | None] # ISO 8601, UTC; where the host knows it
|
||||
last_result: NotRequired[str | None] # "success", or what the host says went wrong
|
||||
next_run: NotRequired[str | None] # ISO 8601, UTC; where the host knows it
|
||||
|
||||
|
||||
class ScheduledJobsReadingDict(TypedDict):
|
||||
"""Every scheduled job of a host, and the time zone its schedules are in."""
|
||||
|
||||
jobs: list[ScheduledJobDict]
|
||||
time_zone: str | None # IANA name ("Europe/Berlin"); None when the host does not say
|
||||
|
||||
|
||||
class ScheduledJobSpecDict(TypedDict):
|
||||
"""A job an application wants on the host (``put_scheduled_job``)."""
|
||||
|
||||
name: str # the job's name on the host, chosen by the application: [a-z0-9][a-z0-9-]*
|
||||
schedule: str # a cron expression (napalm_device_types.cron), "@reboot" included
|
||||
run_as: str
|
||||
command: str # one or more lines of POSIX sh; a platform may take its own (OPNsense: configd)
|
||||
enabled: bool
|
||||
description: NotRequired[str]
|
||||
|
||||
|
||||
class ScheduledJobResultDict(TypedDict):
|
||||
"""What an action on a scheduled job did."""
|
||||
|
||||
success: bool
|
||||
output: str # what the host said; for a run, what the job printed
|
||||
|
||||
|
||||
class ScheduledJobChanged(Exception): # noqa: N818 - a state, as the callers word it
|
||||
"""The job's definition differs from the digest the caller saw: nothing was written."""
|
||||
|
||||
|
||||
class ScheduledJobRefused(ValueError):
|
||||
"""The job cannot take what was asked of it, or the request itself is not valid."""
|
||||
|
||||
|
||||
def job_digest(job: dict[str, Any], *, content: str = "") -> str:
|
||||
"""The definition of *job* in 16 hex characters.
|
||||
|
||||
Over kind, source, run_as, schedule, command and enabled -- so enabling a job
|
||||
changes it -- and *content*: what the definition points at that the job's
|
||||
fields do not show, such as a run-parts script's checksum.
|
||||
"""
|
||||
parts = [job.get(field) for field in _DEFINITION] + [content]
|
||||
return hashlib.sha256(json.dumps(parts).encode()).hexdigest()[:16]
|
||||
|
||||
|
||||
def check_job_name(name: str) -> None:
|
||||
""":raises ScheduledJobRefused: for a name a file or a unit could not carry safely."""
|
||||
if not _NAME.fullmatch(name or ""):
|
||||
raise ScheduledJobRefused(f"not a job name: {name!r} (a-z, 0-9 and -, at most 63)")
|
||||
|
||||
|
||||
def check_spec(spec: ScheduledJobSpecDict) -> None:
|
||||
""":raises ScheduledJobRefused: for a spec no host could be given as it is."""
|
||||
check_job_name(spec["name"])
|
||||
if not _USER.fullmatch(spec.get("run_as") or ""):
|
||||
raise ScheduledJobRefused(f"not a user name: {spec.get('run_as')!r}")
|
||||
if not (spec.get("command") or "").strip():
|
||||
raise ScheduledJobRefused("a job needs a command")
|
||||
if "\n" in (spec.get("description") or ""):
|
||||
raise ScheduledJobRefused("a description is one line")
|
||||
if not is_reboot(spec["schedule"]):
|
||||
try:
|
||||
parse_cron(spec["schedule"])
|
||||
except ValueError as exc:
|
||||
raise ScheduledJobRefused(str(exc)) from exc
|
||||
|
||||
|
||||
class ScheduledJobsMixin:
|
||||
"""Declares the scheduled-jobs contract; a driver that can implements it.
|
||||
|
||||
A verb a driver cannot do at all is ``None`` on the driver, as ``manage_service``
|
||||
is; a verb one job cannot take is missing from that job's ``actions``.
|
||||
"""
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def get_scheduled_jobs(self) -> ScheduledJobsReadingDict:
|
||||
"""Every scheduled job of the host.
|
||||
|
||||
:raises ValueError: when the reading came back cut short -- never a
|
||||
shorter list.
|
||||
"""
|
||||
...
|
||||
|
||||
def manage_scheduled_job(
|
||||
self, key: str, action: str, *, digest: str
|
||||
) -> ScheduledJobResultDict:
|
||||
"""Enable, disable or run the job *key* now.
|
||||
|
||||
:raises ScheduledJobChanged: when the job is no longer as *digest* says.
|
||||
:raises ScheduledJobRefused: when the job does not take *action*.
|
||||
:raises LookupError: when the host has no job *key*.
|
||||
"""
|
||||
...
|
||||
|
||||
def put_scheduled_job(
|
||||
self, spec: ScheduledJobSpecDict, *, digest: str | None = None
|
||||
) -> ScheduledJobResultDict:
|
||||
"""Create the job ``spec["name"]`` (*digest* None) or change it.
|
||||
|
||||
:raises ScheduledJobChanged: when the job is no longer as *digest*
|
||||
says, or when a job of that name is there although *digest* is None.
|
||||
:raises ScheduledJobRefused: for a spec :func:`check_spec` refuses, or
|
||||
one the host cannot take.
|
||||
"""
|
||||
...
|
||||
|
||||
def delete_scheduled_job(self, key: str, *, digest: str) -> ScheduledJobResultDict:
|
||||
"""Delete the job *key*, which an application wrote.
|
||||
|
||||
:raises ScheduledJobChanged: when the job is no longer as *digest* says.
|
||||
:raises ScheduledJobRefused: when the job is not one an application wrote.
|
||||
:raises LookupError: when the host has no job *key*.
|
||||
"""
|
||||
...
|
||||
@@ -0,0 +1,493 @@
|
||||
"""Scheduled jobs on a POSIX host, over the command channel: cron and systemd timers.
|
||||
|
||||
:class:`PosixScheduledJobsMixin` implements
|
||||
:class:`~napalm_device_types.scheduled_jobs.ScheduledJobsMixin` for a driver that has
|
||||
:meth:`~napalm_device_types.channel.CommandChannelMixin.run_command`. The driver
|
||||
mixes it in and sets the paths its host keeps jobs in; the defaults are Debian's.
|
||||
|
||||
**Reading** is one framed report as root (:mod:`napalm_device_types._scheduled_reading`).
|
||||
|
||||
**Switching** a cron or anacron line comments it out behind ``#napalm-disabled: ``
|
||||
and takes exactly that marker away again; a run-parts script loses or gets its
|
||||
execute bit; a timer is ``systemctl disable --now`` / ``enable --now``. A timer that
|
||||
cannot be switched (``static``) only runs.
|
||||
|
||||
**Writing** stages the new text as the login user and installs it as root with one
|
||||
fixed command: ``crontab -u <user> <file>`` for a user's crontab, ``install`` with
|
||||
the file's own mode and owner for a system file. Root never reads stdin. Before any
|
||||
write the job is read again and compared with the digest the caller saw.
|
||||
|
||||
**An application's own job** (:meth:`put_scheduled_job`) is a timer on a host with
|
||||
systemd -- ``<name>.timer`` and ``<name>.service`` in :attr:`UNIT_DIR`, the command
|
||||
in :attr:`JOB_SCRIPT_DIR` ``/<name>`` behind a marker and its cron expression --
|
||||
else a file in :attr:`OWN_CRON_D_DIR`, else marked lines in the user's crontab.
|
||||
|
||||
**Running now** is bounded by :data:`~napalm_device_types.scheduled_jobs.RUN_TIMEOUT`:
|
||||
a timer's unit by ``systemctl start`` and its journal since then, a cron line's
|
||||
command as its user the way cron runs it (``_crontab.run_script``).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import fnmatch
|
||||
import shlex
|
||||
from collections.abc import Sequence
|
||||
from typing import TYPE_CHECKING, ClassVar
|
||||
|
||||
from napalm_device_types._crontab import (
|
||||
JOB_MARKER,
|
||||
CronLine,
|
||||
marked_lines,
|
||||
parse_anacrontab,
|
||||
parse_crontab,
|
||||
run_script,
|
||||
switched,
|
||||
with_job,
|
||||
without_job,
|
||||
)
|
||||
from napalm_device_types._scheduled_reading import (
|
||||
OWNERS_SH,
|
||||
SCHEDULE_HEADER,
|
||||
TIME_ZONE_SH,
|
||||
Paths,
|
||||
Reading,
|
||||
parse_reading,
|
||||
reading_command,
|
||||
)
|
||||
from napalm_device_types.cron import is_reboot, to_on_calendar
|
||||
from napalm_device_types.scheduled_jobs import (
|
||||
ACTIONS,
|
||||
RUN_TIMEOUT,
|
||||
ScheduledJobChanged,
|
||||
ScheduledJobRefused,
|
||||
ScheduledJobResultDict,
|
||||
ScheduledJobsMixin,
|
||||
ScheduledJobSpecDict,
|
||||
ScheduledJobsReadingDict,
|
||||
check_spec,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from napalm_device_types.channel import CommandResult
|
||||
|
||||
_STAGE = 't=$(mktemp) && chmod 644 "$t" && cat > "$t" && printf %s "$t"'
|
||||
#: What a run keeps of the job's output.
|
||||
_OUTPUT_LIMIT = 64 * 1024
|
||||
_READ_TIMEOUT = 120
|
||||
|
||||
|
||||
def _command(argv: Sequence[str]) -> str:
|
||||
return shlex.join(argv)
|
||||
|
||||
|
||||
def _said(result: CommandResult) -> str:
|
||||
return "\n".join(p for p in (result.stdout.strip(), result.stderr.strip()) if p)[
|
||||
-_OUTPUT_LIMIT:
|
||||
]
|
||||
|
||||
|
||||
def _result(result: CommandResult, done: str) -> ScheduledJobResultDict:
|
||||
if result.exit_code != 0:
|
||||
return {"success": False, "output": _said(result) or f"exit {result.exit_code}"}
|
||||
return {"success": True, "output": _said(result) or done}
|
||||
|
||||
|
||||
class PosixScheduledJobsMixin(ScheduledJobsMixin):
|
||||
"""Scheduled jobs of a host reached through ``run_command``; the paths are the driver's."""
|
||||
|
||||
CRON_SYSTEM_FILES: ClassVar[tuple[str, ...]] = ("/etc/crontab",)
|
||||
CRON_D_DIRS: ClassVar[tuple[str, ...]] = ("/etc/cron.d",)
|
||||
CRON_SPOOL_DIRS: ClassVar[tuple[str, ...]] = (
|
||||
"/var/spool/cron/crontabs",
|
||||
"/var/spool/cron",
|
||||
"/etc/crontabs",
|
||||
)
|
||||
RUN_PARTS_DIRS: ClassVar[tuple[tuple[str, str], ...]] = (
|
||||
("hourly", "/etc/cron.hourly"),
|
||||
("daily", "/etc/cron.daily"),
|
||||
("weekly", "/etc/cron.weekly"),
|
||||
("monthly", "/etc/cron.monthly"),
|
||||
)
|
||||
ANACRONTAB: ClassVar[str] = "/etc/anacrontab"
|
||||
JOB_SCRIPT_DIR: ClassVar[str] = "/usr/local/libexec/scheduled-jobs"
|
||||
UNIT_DIR: ClassVar[str] = "/etc/systemd/system"
|
||||
#: Where a host without systemd takes an application's job as a file, if the directory
|
||||
#: is there; else, or None, the user's crontab.
|
||||
OWN_CRON_D_DIR: ClassVar[str | None] = "/etc/cron.d"
|
||||
ROOT_GROUP: ClassVar[str] = "root"
|
||||
#: Prints "<octal mode> <owner> <group>" of the path appended.
|
||||
STAT_ARGV: ClassVar[tuple[str, ...]] = ("stat", "-c", "%a %U %G", "--")
|
||||
#: ``install`` and how it makes missing directories (FreeBSD's ``-D`` is a destdir).
|
||||
INSTALL: ClassVar[tuple[str, ...]] = ("install", "-D")
|
||||
#: ``sh`` that reads definition paths on stdin and prints ``<package>: <path>``.
|
||||
OWNERS_SH: ClassVar[str] = OWNERS_SH
|
||||
#: ``sh`` that prints the time zone: an IANA name, or a path ending in ``zoneinfo/<name>``.
|
||||
TIME_ZONE_SH: ClassVar[str] = TIME_ZONE_SH
|
||||
#: ``fnmatch`` patterns on a job's ``source``: what the platform generates is ``platform``.
|
||||
PLATFORM_SOURCES: ClassVar[tuple[str, ...]] = ()
|
||||
#: Run as root after a user's crontab was written; for a host that keeps it elsewhere too.
|
||||
CRONTAB_AFTER_WRITE: ClassVar[tuple[str, ...] | None] = None
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def run_command(
|
||||
self,
|
||||
command: str,
|
||||
*,
|
||||
privileged: bool = False,
|
||||
timeout: float = 60,
|
||||
stdin: bytes | None = None,
|
||||
) -> CommandResult: ...
|
||||
|
||||
# -- reading ---------------------------------------------------------------------------------
|
||||
|
||||
def _sj_paths(self) -> Paths:
|
||||
return Paths(
|
||||
system_files=self.CRON_SYSTEM_FILES,
|
||||
cron_d_dirs=self.CRON_D_DIRS,
|
||||
spool_dirs=self.CRON_SPOOL_DIRS,
|
||||
run_parts=self.RUN_PARTS_DIRS,
|
||||
anacrontab=self.ANACRONTAB,
|
||||
script_dir=self.JOB_SCRIPT_DIR,
|
||||
owners_sh=self.OWNERS_SH,
|
||||
time_zone_sh=self.TIME_ZONE_SH,
|
||||
)
|
||||
|
||||
def _sj_read(self) -> Reading:
|
||||
result = self.run_command(
|
||||
reading_command(self._sj_paths()), privileged=True, timeout=_READ_TIMEOUT
|
||||
)
|
||||
reading = parse_reading(result.stdout)
|
||||
for job in reading.jobs:
|
||||
source = str(job["source"])
|
||||
if job["managed_by"] != "application" and any(
|
||||
fnmatch.fnmatchcase(source, p) for p in self.PLATFORM_SOURCES
|
||||
):
|
||||
job["managed_by"] = "platform"
|
||||
return reading
|
||||
|
||||
def get_scheduled_jobs(self) -> ScheduledJobsReadingDict:
|
||||
"""Every scheduled job of the host, in one round trip.
|
||||
|
||||
:raises ValueError: when the report came back cut short.
|
||||
"""
|
||||
reading = self._sj_read()
|
||||
return {"jobs": reading.jobs, "time_zone": reading.time_zone} # type: ignore[typeddict-item]
|
||||
|
||||
def _sj_job(self, reading: Reading, key: str, digest: str) -> dict[str, object]:
|
||||
job = next((j for j in reading.jobs if j["key"] == key), None)
|
||||
if job is None:
|
||||
raise LookupError(f"no scheduled job {key}")
|
||||
if job["digest"] != digest:
|
||||
raise ScheduledJobChanged(f"{key} changed on the host since it was read")
|
||||
return job
|
||||
|
||||
# -- commands --------------------------------------------------------------------------------
|
||||
|
||||
def _sj_root(self, argv: Sequence[str], timeout: float = 60) -> CommandResult:
|
||||
return self.run_command(_command(argv), privileged=True, timeout=timeout)
|
||||
|
||||
def _sj_checked(self, argv: Sequence[str]) -> CommandResult:
|
||||
result = self._sj_root(argv)
|
||||
if result.exit_code != 0:
|
||||
raise RuntimeError(f"{_command(argv)}: {_said(result) or result.exit_code}")
|
||||
return result
|
||||
|
||||
def _sj_stage(self, content: str) -> str:
|
||||
result = self.run_command(_STAGE, timeout=30, stdin=content.encode())
|
||||
path = result.stdout.strip().splitlines()[-1:] or [""]
|
||||
if result.exit_code != 0 or not path[0].startswith("/"):
|
||||
raise RuntimeError(f"cannot stage a file: {_said(result)}")
|
||||
return path[0]
|
||||
|
||||
def _sj_install(self, content: str, path: str, mode: str, owner: str, group: str) -> None:
|
||||
staged = self._sj_stage(content)
|
||||
try:
|
||||
self._sj_checked(
|
||||
[*self.INSTALL, "-m", mode, "-o", owner, "-g", group, "--", staged, path]
|
||||
)
|
||||
finally:
|
||||
self.run_command(_command(["rm", "-f", "--", staged]), timeout=30)
|
||||
|
||||
# -- crontab files ---------------------------------------------------------------------------
|
||||
|
||||
def _sj_spool_user(self, path: str) -> str | None:
|
||||
"""The user whose crontab *path* is; None for a system file."""
|
||||
directory, _, user = path.rpartition("/")
|
||||
return user if directory in self.CRON_SPOOL_DIRS else None
|
||||
|
||||
def _sj_crontab(self, user: str) -> str:
|
||||
result = self._sj_root(["crontab", "-u", user, "-l"])
|
||||
if result.exit_code != 0:
|
||||
if "no crontab" in (result.stderr + result.stdout).lower():
|
||||
return ""
|
||||
raise RuntimeError(f"cannot read the crontab of {user}: {_said(result)}")
|
||||
return result.stdout
|
||||
|
||||
def _sj_text(self, path: str) -> str:
|
||||
user = self._sj_spool_user(path)
|
||||
if user is not None:
|
||||
return self._sj_crontab(user)
|
||||
return self._sj_checked(["cat", "--", path]).stdout
|
||||
|
||||
def _sj_write_crontab(self, user: str, text: str) -> None:
|
||||
staged = self._sj_stage(text)
|
||||
try:
|
||||
self._sj_checked(["crontab", "-u", user, staged])
|
||||
finally:
|
||||
self.run_command(_command(["rm", "-f", "--", staged]), timeout=30)
|
||||
if self.CRONTAB_AFTER_WRITE:
|
||||
self._sj_checked(self.CRONTAB_AFTER_WRITE)
|
||||
|
||||
def _sj_write(self, path: str, text: str) -> None:
|
||||
user = self._sj_spool_user(path)
|
||||
if user is not None:
|
||||
self._sj_write_crontab(user, text)
|
||||
return
|
||||
# Through a symbolic link to the file it names, with that file's mode: the
|
||||
# link's own mode is 777, and cron ignores a file anyone may write.
|
||||
real = self._sj_checked(["readlink", "-f", "--", path]).stdout.strip() or path
|
||||
mode, owner, group = self._sj_checked([*self.STAT_ARGV, real]).stdout.split()[:3]
|
||||
self._sj_install(text, real, mode, owner, group)
|
||||
|
||||
def _sj_line(self, job: dict[str, object], text: str) -> CronLine:
|
||||
"""The job's line in *text* as it is now, read again for the write."""
|
||||
path = str(job["source"])
|
||||
if job["kind"] == "anacron":
|
||||
found = parse_anacrontab(text)
|
||||
else:
|
||||
user = self._sj_spool_user(path)
|
||||
found = parse_crontab(path, text, system=user is None, user=user or "")
|
||||
line = next((ln for ln in found if ln.key == job["key"]), None)
|
||||
if line is None:
|
||||
raise ScheduledJobChanged(f"{job['key']} is no longer in {path}")
|
||||
return line
|
||||
|
||||
# -- switching and running -------------------------------------------------------------------
|
||||
|
||||
def _sj_timer_unit(self, job: dict[str, object]) -> str:
|
||||
key = str(job["key"])
|
||||
return f"{job['name']}.timer" if key.startswith("job:") else key.removeprefix("timer:")
|
||||
|
||||
def _sj_switch(self, job: dict[str, object], enabled: bool) -> ScheduledJobResultDict:
|
||||
word = "enabled" if enabled else "disabled"
|
||||
if job["kind"] == "systemd_timer":
|
||||
verb = "enable" if enabled else "disable"
|
||||
return _result(
|
||||
self._sj_root(["systemctl", verb, "--now", "--", self._sj_timer_unit(job)]), word
|
||||
)
|
||||
if job["kind"] == "run_parts":
|
||||
return _result(
|
||||
self._sj_root(["chmod", "+x" if enabled else "-x", "--", str(job["source"])]), word
|
||||
)
|
||||
path = str(job["source"])
|
||||
text = self._sj_text(path)
|
||||
self._sj_write(path, switched(text, self._sj_line(job, text), enabled=enabled))
|
||||
return {"success": True, "output": f"{job['key']} {word}"}
|
||||
|
||||
def _sj_as(self, user: str, script: str) -> list[str]:
|
||||
"""*script* run by ``sh`` as *user*, from root."""
|
||||
if user == "root":
|
||||
return ["sh", "-c", script]
|
||||
return ["su", "-s", "/bin/sh", user, "-c", script]
|
||||
|
||||
def _sj_run(self, job: dict[str, object], reading: Reading) -> ScheduledJobResultDict:
|
||||
bound = ["timeout", str(RUN_TIMEOUT)]
|
||||
wait = RUN_TIMEOUT + 30
|
||||
if job["kind"] == "systemd_timer":
|
||||
unit = reading.units[str(job["key"])]
|
||||
since = self.run_command("date +%s", timeout=30).stdout.strip()
|
||||
started = self._sj_root([*bound, "systemctl", "start", "--", unit], timeout=wait)
|
||||
journal = self._sj_root(
|
||||
[
|
||||
"journalctl",
|
||||
"-u",
|
||||
unit,
|
||||
"--since",
|
||||
f"@{since}",
|
||||
"--no-pager",
|
||||
"-o",
|
||||
"cat",
|
||||
"-n",
|
||||
"200",
|
||||
]
|
||||
)
|
||||
output = "\n".join(p for p in (_said(started), journal.stdout.strip()) if p)
|
||||
return {"success": started.exit_code == 0, "output": output or f"{unit} ran"}
|
||||
if job["kind"] == "run_parts":
|
||||
return _result(self._sj_root([*bound, str(job["source"])], timeout=wait), "ran")
|
||||
path = str(job["source"])
|
||||
line = self._sj_line(job, self._sj_text(path))
|
||||
script = run_script(line.command, line.env)
|
||||
return _result(
|
||||
self._sj_root([*bound, *self._sj_as(line.run_as, script)], timeout=wait), "ran"
|
||||
)
|
||||
|
||||
def manage_scheduled_job(self, key: str, action: str, *, digest: str) -> ScheduledJobResultDict:
|
||||
"""Enable, disable or run the job *key* now; see :class:`ScheduledJobsMixin`."""
|
||||
if action not in ACTIONS:
|
||||
raise ScheduledJobRefused(f"unknown action {action!r}; allowed: {list(ACTIONS)}")
|
||||
reading = self._sj_read()
|
||||
job = self._sj_job(reading, key, digest)
|
||||
if action not in job["actions"]: # type: ignore[operator]
|
||||
raise ScheduledJobRefused(f"{key} cannot be told to {action} on this host")
|
||||
if action == "run":
|
||||
return self._sj_run(job, reading)
|
||||
return self._sj_switch(job, action == "enable")
|
||||
|
||||
# -- an application's own jobs ---------------------------------------------------------------
|
||||
|
||||
def _sj_own(
|
||||
self, reading: Reading, spec: ScheduledJobSpecDict, digest: str | None
|
||||
) -> dict[str, object] | None:
|
||||
current = next((j for j in reading.jobs if j["key"] == f"job:{spec['name']}"), None)
|
||||
if digest is None:
|
||||
if current is not None:
|
||||
raise ScheduledJobChanged(f"a job {spec['name']} is there already")
|
||||
return None
|
||||
if current is None:
|
||||
raise LookupError(f"no scheduled job job:{spec['name']}")
|
||||
if current["digest"] != digest:
|
||||
raise ScheduledJobChanged(f"job:{spec['name']} changed on the host since it was read")
|
||||
return current
|
||||
|
||||
def _sj_group(self, user: str) -> str:
|
||||
result = self.run_command(_command(["id", "-gn", "--", user]), timeout=30)
|
||||
if result.exit_code != 0:
|
||||
raise ScheduledJobRefused(f"the host has no user {user}")
|
||||
return result.stdout.strip()
|
||||
|
||||
def _sj_calendar(self, schedule: str) -> list[str]:
|
||||
if is_reboot(schedule):
|
||||
return ["OnBootSec=1min"]
|
||||
lines = to_on_calendar(schedule)
|
||||
for line in lines:
|
||||
checked = self.run_command(
|
||||
_command(["systemd-analyze", "calendar", "--", line]), timeout=30
|
||||
)
|
||||
if checked.exit_code != 0:
|
||||
raise ScheduledJobRefused(f"systemd does not take {line!r}: {_said(checked)}")
|
||||
return [f"OnCalendar={line}" for line in lines]
|
||||
|
||||
def _sj_put_timer(self, spec: ScheduledJobSpecDict) -> None:
|
||||
name, description = spec["name"], spec.get("description") or spec["name"]
|
||||
script = f"{self.JOB_SCRIPT_DIR}/{name}"
|
||||
unit_description = description.replace("%", "%%")
|
||||
timer = "\n".join(
|
||||
[
|
||||
"[Unit]",
|
||||
f"Description={unit_description}",
|
||||
"",
|
||||
"[Timer]",
|
||||
*self._sj_calendar(spec["schedule"]),
|
||||
f"Unit={name}.service",
|
||||
"",
|
||||
"[Install]",
|
||||
"WantedBy=timers.target",
|
||||
"",
|
||||
]
|
||||
)
|
||||
service = "\n".join(
|
||||
[
|
||||
"[Unit]",
|
||||
f"Description={unit_description}",
|
||||
"",
|
||||
"[Service]",
|
||||
"Type=oneshot",
|
||||
f"User={spec['run_as']}",
|
||||
f"ExecStart={script}",
|
||||
"",
|
||||
]
|
||||
)
|
||||
body = f"#!/bin/sh\n{JOB_MARKER}{name} {spec.get('description') or ''}".rstrip()
|
||||
body += f"\n{SCHEDULE_HEADER}{spec['schedule']}\n{spec['command'].strip()}\n"
|
||||
self._sj_install(body, script, "750", "root", self._sj_group(spec["run_as"]))
|
||||
self._sj_install(service, f"{self.UNIT_DIR}/{name}.service", "644", "root", self.ROOT_GROUP)
|
||||
self._sj_install(timer, f"{self.UNIT_DIR}/{name}.timer", "644", "root", self.ROOT_GROUP)
|
||||
self._sj_checked(["systemctl", "daemon-reload"])
|
||||
verb = "enable" if spec["enabled"] else "disable"
|
||||
self._sj_checked(["systemctl", verb, "--now", "--", f"{name}.timer"])
|
||||
|
||||
def _sj_cron_text(self, spec: ScheduledJobSpecDict, *, system: bool) -> list[str]:
|
||||
if "\n" in spec["command"].strip():
|
||||
raise ScheduledJobRefused("a cron line holds one line of command")
|
||||
owner = f" {spec['run_as']}" if system else ""
|
||||
line = f"{spec['schedule']}{owner} {spec['command'].strip()}"
|
||||
return marked_lines(
|
||||
spec["name"], spec.get("description") or "", line, enabled=spec["enabled"]
|
||||
)
|
||||
|
||||
def _sj_cron_d(self) -> str | None:
|
||||
""":attr:`OWN_CRON_D_DIR` where the host has it; BusyBox crond reads none."""
|
||||
directory = self.OWN_CRON_D_DIR
|
||||
if directory is None:
|
||||
return None
|
||||
found = self.run_command(_command(["test", "-d", directory]), timeout=30)
|
||||
return directory if found.exit_code == 0 else None
|
||||
|
||||
def _sj_put_cron(self, spec: ScheduledJobSpecDict, current: dict[str, object] | None) -> None:
|
||||
if self._sj_cron_d() is not None:
|
||||
self._sj_group(spec["run_as"])
|
||||
text = "\n".join(self._sj_cron_text(spec, system=True)) + "\n"
|
||||
self._sj_install(
|
||||
text, f"{self.OWN_CRON_D_DIR}/{spec['name']}", "644", "root", self.ROOT_GROUP
|
||||
)
|
||||
return
|
||||
if current is not None and current["run_as"] != spec["run_as"]:
|
||||
self._sj_delete_cron(current)
|
||||
user = spec["run_as"]
|
||||
text = self._sj_crontab(user)
|
||||
jobs = parse_crontab("", text, system=False, user=user)
|
||||
self._sj_write_crontab(
|
||||
user, with_job(text, jobs, spec["name"], self._sj_cron_text(spec, system=False))
|
||||
)
|
||||
|
||||
def put_scheduled_job(
|
||||
self, spec: ScheduledJobSpecDict, *, digest: str | None = None
|
||||
) -> ScheduledJobResultDict:
|
||||
"""Create or change the application's own job; see :class:`ScheduledJobsMixin`."""
|
||||
check_spec(spec)
|
||||
reading = self._sj_read()
|
||||
current = self._sj_own(reading, spec, digest)
|
||||
if reading.systemd:
|
||||
self._sj_put_timer(spec)
|
||||
else:
|
||||
self._sj_put_cron(spec, current)
|
||||
return {
|
||||
"success": True,
|
||||
"output": f"job:{spec['name']} {'changed' if current else 'created'}",
|
||||
}
|
||||
|
||||
def _sj_delete_cron(self, job: dict[str, object]) -> None:
|
||||
path = str(job["source"])
|
||||
if self._sj_spool_user(path) is None:
|
||||
self._sj_checked(["rm", "-f", "--", path])
|
||||
return
|
||||
text = self._sj_text(path)
|
||||
jobs = parse_crontab(path, text, system=False, user=str(job["run_as"]))
|
||||
self._sj_write(path, without_job(text, jobs, str(job["name"])))
|
||||
|
||||
def delete_scheduled_job(self, key: str, *, digest: str) -> ScheduledJobResultDict:
|
||||
"""Delete the application's job *key*; see :class:`ScheduledJobsMixin`."""
|
||||
job = self._sj_job(self._sj_read(), key, digest)
|
||||
if job["managed_by"] != "application":
|
||||
raise ScheduledJobRefused(
|
||||
f"{key} was not written by an application; it is only switched"
|
||||
)
|
||||
name = str(job["name"])
|
||||
if job["kind"] != "systemd_timer":
|
||||
self._sj_delete_cron(job)
|
||||
return {"success": True, "output": f"{key} deleted"}
|
||||
self._sj_root(["systemctl", "disable", "--now", "--", f"{name}.timer"])
|
||||
self._sj_checked(
|
||||
[
|
||||
"rm",
|
||||
"-f",
|
||||
"--",
|
||||
f"{self.UNIT_DIR}/{name}.timer",
|
||||
f"{self.UNIT_DIR}/{name}.service",
|
||||
f"{self.JOB_SCRIPT_DIR}/{name}",
|
||||
]
|
||||
)
|
||||
self._sj_checked(["systemctl", "daemon-reload"])
|
||||
return {"success": True, "output": f"{key} deleted"}
|
||||
@@ -0,0 +1,538 @@
|
||||
"""Service clusters: the configuration that says which services run together.
|
||||
|
||||
Two hosts that answer at one virtual IP may run the same service by accident.
|
||||
What proves a cluster is the configuration its members share. That is read the
|
||||
same way on every Linux host, so the command and its parse live here once, and
|
||||
a driver only carries the command across (netOrk #1125).
|
||||
|
||||
**What is read.**
|
||||
|
||||
- *Patroni*: every ``patroni.service`` and ``patroni@*.service`` unit systemd
|
||||
knows. Its configuration file -- the unit's first non-flag argument, else
|
||||
``/etc/patroni/config.yml`` -- gives ``scope``, ``name`` and
|
||||
``restapi.connect_address``. The REST API there (``/cluster``, which Patroni
|
||||
answers without a login) lists the members its DCS knows, each with its role:
|
||||
``leader``, ``sync_standby``, ``replica``. When it does not answer, the host
|
||||
is a member of its scope on its own.
|
||||
- *adguardhome-sync*: a running container whose image is adguardhome-sync.
|
||||
``ORIGIN_URL`` and ``REPLICA*_URL`` name the AdGuard Home instances it keeps in
|
||||
step: the origin pushes to every replica. A sync configured by a YAML file
|
||||
instead of its environment is not read.
|
||||
- *BunkerWeb*: a running instance container (``bunkerity/bunkerweb``) admits a
|
||||
manager's configuration pushes from the addresses in ``API_WHITELIST_IP``; a
|
||||
manager (``bunkerweb-all-in-one``, ``bunkerweb-scheduler``) names its
|
||||
instances in ``BUNKERWEB_INSTANCES``. BunkerWeb's own default whitelist,
|
||||
loopback and the docker network, names no one. A manager that keeps its
|
||||
instances in its database names none here: its workers name it.
|
||||
|
||||
**Filtered on the host.** The same configuration holds secrets -- Patroni's
|
||||
superuser and REST passwords, the sync's AdGuard logins, BunkerWeb's database
|
||||
URI. The command never prints a configuration: awk programs print allowlisted
|
||||
fields only, and a URL loses its ``user:password@``. The DCS endpoints stay on
|
||||
the host too; the members come from ``/cluster``.
|
||||
|
||||
**None is "did not look".** A Patroni configuration that exists but cannot be
|
||||
read, a host without systemd, a docker the login may not ask, a sync configured
|
||||
by file: that mechanism is None, and the caller keeps what it knew. ``[]`` is a
|
||||
host that has none.
|
||||
|
||||
**Root, and without it.** Patroni's configuration is the postgres user's alone.
|
||||
The whole script goes to the host as one ``sh -c`` argument; when the privileged
|
||||
call brings no report back, it runs again without privilege.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import ipaddress
|
||||
import json
|
||||
import re
|
||||
from collections.abc import Callable
|
||||
from shlex import quote
|
||||
from typing import TYPE_CHECKING
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
from napalm_device_types.models import (
|
||||
ServiceClusterDict,
|
||||
ServiceClusterMemberDict,
|
||||
ServiceClustersDict,
|
||||
)
|
||||
from napalm_device_types.terminal import strip_terminal_codes
|
||||
|
||||
_BEGIN = "SVCC_BEGIN"
|
||||
_END = "SVCC_END"
|
||||
#: Exists only while systemd runs as PID 1; a container may carry systemctl without.
|
||||
_SYSTEMD_RUN = "/run/systemd/system"
|
||||
_PATRONI_CONFIG = "/etc/patroni/config.yml"
|
||||
#: The container environment that may leave the host.
|
||||
_ENV_KEYS = "ORIGIN_URL|REPLICA[0-9]*_URL|API_WHITELIST_IP|BUNKERWEB_INSTANCES|API_HTTP_PORT"
|
||||
#: BunkerWeb's instance API, where a manager pushes its configuration.
|
||||
_BUNKERWEB_API_PORT = 5000
|
||||
|
||||
#: Fed a unit's ``systemctl show`` output: ``active <state>`` and ``config <path>``,
|
||||
#: the first argument of the command line that is no flag.
|
||||
_UNIT_AWK = " ".join(
|
||||
(
|
||||
r'/^ActiveState=/ { print "active " substr($0, 13); next }',
|
||||
r'/^ExecStart=/ { s = $0; i = index(s, "argv[]="); if (!i) next; s = substr(s, i + 7);'
|
||||
r' j = index(s, " ;"); if (j) s = substr(s, 1, j - 1); n = split(s, t, " ");'
|
||||
r' for (k = 2; k <= n; k++) if (!c && t[k] !~ /^-/) { print "config " t[k]; c = 1 } }',
|
||||
)
|
||||
)
|
||||
|
||||
#: Fed Patroni's YAML: ``scope``, ``name`` (top level only -- ``tags`` may carry a
|
||||
#: ``name`` too), ``restapi`` (its connect_address) and ``tls yes`` when the REST
|
||||
#: API has a certificate. Nothing else leaves the file.
|
||||
_PATRONI_AWK = " ".join(
|
||||
(
|
||||
r'function v(l) { sub(/^[^:]*:[ \t]*/, "", l); return l }',
|
||||
r'/^[A-Za-z_][A-Za-z0-9_]*:/ { sec = $0; sub(/:.*/, "", sec);'
|
||||
r' if (sec == "scope" || sec == "name") print sec " " v($0); next }',
|
||||
r'sec == "restapi" && /^[ \t]+connect_address:/ { print "restapi " v($0) }',
|
||||
r'sec == "restapi" && /^[ \t]+certfile:/ { print "tls yes" }',
|
||||
)
|
||||
)
|
||||
|
||||
#: Fed a container's environment, one ``KEY=value`` a line: the allowlisted keys,
|
||||
#: each URL without its ``user:password@``.
|
||||
_ENV_AWK = " ".join(
|
||||
(
|
||||
r'{ x = index($0, "="); if (!x) next; k = substr($0, 1, x - 1); v = substr($0, x + 1);',
|
||||
rf" if (k !~ /^({_ENV_KEYS})$/) next;",
|
||||
r' gsub(/:\/\/[^@\/ ]*@/, "://", v); print "env " k " " v }',
|
||||
)
|
||||
)
|
||||
|
||||
_PATRONI_SH = (
|
||||
f"if command -v systemctl >/dev/null 2>&1 && [ -d {_SYSTEMD_RUN} ]; then echo '[systemd]'; "
|
||||
"for u in $(systemctl list-units --all --plain --no-legend --type=service "
|
||||
"patroni.service 'patroni@*' 2>/dev/null | awk '{print $1}'); do "
|
||||
'echo "[patroni $u]"; '
|
||||
'o=$(systemctl show -p ActiveState -p ExecStart "$u" 2>/dev/null | '
|
||||
f"awk {quote(_UNIT_AWK)}); "
|
||||
'echo "$o"; '
|
||||
"f=$(echo \"$o\" | sed -n 's/^config //p' | head -n 1); "
|
||||
f'[ -n "$f" ] || f={_PATRONI_CONFIG}; '
|
||||
'if [ -r "$f" ]; then '
|
||||
f'p=$(awk {quote(_PATRONI_AWK)} "$f"); echo "$p"; '
|
||||
"r=$(echo \"$p\" | sed -n 's/^restapi //p' | head -n 1 | tr -d '\\042\\047 '); "
|
||||
's=http; case "$p" in *"tls yes"*) s=https;; esac; '
|
||||
'if [ -n "$r" ]; then '
|
||||
'b=$(curl -s -k -m 3 "$s://$r/cluster" 2>/dev/null || '
|
||||
'wget -q -T 3 --no-check-certificate -O - "$s://$r/cluster" 2>/dev/null); '
|
||||
'if [ -n "$b" ]; then echo "rest $(echo "$b" | tr -d \'\\n\')"; '
|
||||
"else echo rest-unreachable; fi; fi; "
|
||||
'elif [ -e "$f" ]; then echo "unreadable $f"; '
|
||||
'else echo "missing $f"; fi; '
|
||||
"done; "
|
||||
"else echo '[no-systemd]'; fi; "
|
||||
)
|
||||
|
||||
|
||||
def _docker_sh(binary: str | None) -> str:
|
||||
"""The containers part: running containers of the clustered images, their
|
||||
allowlisted environment. ``[no-docker]`` for a host without the engine."""
|
||||
if not binary:
|
||||
return "echo '[no-docker]'; "
|
||||
return (
|
||||
f"D={quote(binary)}; "
|
||||
'if command -v "$D" >/dev/null 2>&1; then '
|
||||
"if c=$(\"$D\" ps --format '{{.Names}}' 2>/dev/null); then echo '[docker]'; "
|
||||
"for n in $c; do "
|
||||
'i=$("$D" inspect --format \'{{.Config.Image}}\' "$n" 2>/dev/null); '
|
||||
'case "$i" in *adguardhome-sync*|*bunkerweb*) '
|
||||
'echo "[container $n]"; echo "image $i"; '
|
||||
'"$D" inspect --format \'{{range .Config.Env}}{{println .}}{{end}}\' "$n" '
|
||||
f"2>/dev/null | awk {quote(_ENV_AWK)};; esac; done; "
|
||||
"else echo '[docker-unreadable]'; fi; "
|
||||
"else echo '[no-docker]'; fi; "
|
||||
)
|
||||
|
||||
|
||||
def service_clusters_command(docker: str | None = "docker") -> str:
|
||||
"""One line of POSIX ``sh``, read-only. *docker* is the engine's binary as the
|
||||
driver reaches it (QNAP's lives elsewhere), None for a host without one. The
|
||||
frame markers are printed in two halves so that a transport which echoes the
|
||||
command does not show them early."""
|
||||
return (
|
||||
"PATH=$PATH:/usr/sbin:/sbin; "
|
||||
"printf '%s%s\\n' SVCC_ BEGIN; "
|
||||
+ _PATRONI_SH
|
||||
+ _docker_sh(docker)
|
||||
+ "printf '%s%s\\n' SVCC_ END"
|
||||
)
|
||||
|
||||
|
||||
SERVICE_CLUSTERS_COMMAND = service_clusters_command()
|
||||
|
||||
_COMMENT_RE = re.compile(r"\s+#.*$")
|
||||
_Record = tuple[str, str]
|
||||
_Section = tuple[str, str, list[_Record]]
|
||||
|
||||
|
||||
def _frame(output: str) -> list[str]:
|
||||
lines = [line.strip() for line in strip_terminal_codes(output).splitlines()]
|
||||
try:
|
||||
start = lines.index(_BEGIN)
|
||||
end = lines.index(_END, start)
|
||||
except ValueError:
|
||||
raise ValueError("no intact service cluster report in the output") from None
|
||||
return lines[start + 1 : end]
|
||||
|
||||
|
||||
def _sections(lines: list[str]) -> list[_Section]:
|
||||
"""``[<kind> <name>]`` headers, each with its records as ``(kind, rest)``."""
|
||||
sections: list[_Section] = []
|
||||
for line in lines:
|
||||
if line.startswith("[") and line.endswith("]"):
|
||||
kind, _, name = line[1:-1].partition(" ")
|
||||
sections.append((kind, name, []))
|
||||
elif line and sections:
|
||||
kind, _, rest = line.partition(" ")
|
||||
sections[-1][2].append((kind, rest.strip()))
|
||||
return sections
|
||||
|
||||
|
||||
def _clean(value: str) -> str:
|
||||
"""Without a trailing ``# comment`` and the quotes around it."""
|
||||
value = _COMMENT_RE.sub("", value.strip())
|
||||
if len(value) >= 2 and value[0] == value[-1] and value[0] in "\"'":
|
||||
value = value[1:-1]
|
||||
return value.strip()
|
||||
|
||||
|
||||
def _member(
|
||||
*,
|
||||
address: str | None = None,
|
||||
port: int | None = None,
|
||||
name: str | None = None,
|
||||
role: str | None = None,
|
||||
state: str | None = None,
|
||||
this_host: bool = False,
|
||||
config_from: int | None = None,
|
||||
) -> ServiceClusterMemberDict:
|
||||
return {
|
||||
"name": name,
|
||||
"address": address,
|
||||
"port": port,
|
||||
"role": role,
|
||||
"state": state,
|
||||
"this_host": this_host,
|
||||
"config_from": config_from,
|
||||
}
|
||||
|
||||
|
||||
def _cluster(
|
||||
mechanism: str,
|
||||
service: str,
|
||||
source: str,
|
||||
cluster: str | None,
|
||||
members: list[ServiceClusterMemberDict],
|
||||
running: bool | None = True,
|
||||
) -> ServiceClusterDict:
|
||||
return {
|
||||
"mechanism": mechanism,
|
||||
"service": service,
|
||||
"source": source,
|
||||
"cluster": cluster,
|
||||
"running": running,
|
||||
"members": members,
|
||||
}
|
||||
|
||||
|
||||
def _host_port(text: str) -> tuple[str | None, int | None]:
|
||||
"""``"10.0.0.5:8008"`` or ``"[fd00::5]:8008"`` -> host and port."""
|
||||
try:
|
||||
split = urlsplit(f"//{text.strip()}")
|
||||
return split.hostname, split.port
|
||||
except ValueError:
|
||||
return None, None
|
||||
|
||||
|
||||
def _url(text: str) -> tuple[str | None, int | None]:
|
||||
"""A URL's host and port, the scheme's default where it names none."""
|
||||
value = text.strip()
|
||||
if "://" not in value:
|
||||
value = f"http://{value}"
|
||||
try:
|
||||
split = urlsplit(value)
|
||||
port = split.port or {"http": 80, "https": 443}.get(split.scheme)
|
||||
except ValueError:
|
||||
return None, None
|
||||
return split.hostname, port
|
||||
|
||||
|
||||
# --- Patroni ------------------------------------------------------------------
|
||||
|
||||
|
||||
def _patroni_members(rest: str, own: str | None) -> list[ServiceClusterMemberDict] | None:
|
||||
try:
|
||||
members = json.loads(rest).get("members")
|
||||
except (ValueError, AttributeError):
|
||||
return None
|
||||
if not isinstance(members, list):
|
||||
return None
|
||||
found = []
|
||||
for entry in members:
|
||||
if not isinstance(entry, dict) or not entry.get("name"):
|
||||
continue
|
||||
port = entry.get("port")
|
||||
found.append(
|
||||
_member(
|
||||
name=str(entry["name"]),
|
||||
address=str(entry["host"]) if entry.get("host") else None,
|
||||
port=port if isinstance(port, int) else None,
|
||||
role=str(entry["role"]) if entry.get("role") else None,
|
||||
state=str(entry["state"]) if entry.get("state") else None,
|
||||
this_host=entry["name"] == own,
|
||||
)
|
||||
)
|
||||
return sorted(found, key=lambda m: m["name"] or "")
|
||||
|
||||
|
||||
def _patroni(unit: str, records: list[_Record]) -> list[ServiceClusterDict]:
|
||||
values: dict[str, str] = {}
|
||||
rest: str | None = None
|
||||
for kind, text in records:
|
||||
if kind in ("active", "scope", "name", "restapi") and kind not in values:
|
||||
values[kind] = _clean(text)
|
||||
elif kind == "rest":
|
||||
rest = text
|
||||
scope = values.get("scope")
|
||||
if not scope:
|
||||
return []
|
||||
own = values.get("name") or None
|
||||
members = _patroni_members(rest, own) if rest else None
|
||||
if not members:
|
||||
address, _ = _host_port(values.get("restapi", ""))
|
||||
members = [_member(name=own, address=address, this_host=True)]
|
||||
running = values["active"] == "active" if "active" in values else None
|
||||
source = unit[: -len(".service")] if unit.endswith(".service") else unit
|
||||
return [_cluster("patroni", "patroni", source, scope, members, running)]
|
||||
|
||||
|
||||
# --- Containers ----------------------------------------------------------------
|
||||
|
||||
|
||||
def _env(records: list[_Record]) -> dict[str, str]:
|
||||
env: dict[str, str] = {}
|
||||
for kind, text in records:
|
||||
if kind == "env":
|
||||
key, _, value = text.partition(" ")
|
||||
env[key] = value.strip()
|
||||
return env
|
||||
|
||||
|
||||
def _image_name(records: list[_Record]) -> str:
|
||||
"""``bunkerity/bunkerweb@sha256:...`` -> ``bunkerweb``."""
|
||||
image = next((text for kind, text in records if kind == "image"), "")
|
||||
return image.split("@")[0].rsplit("/", 1)[-1].split(":")[0]
|
||||
|
||||
|
||||
def _adguardhome_sync(name: str, records: list[_Record]) -> list[ServiceClusterDict] | None:
|
||||
env = _env(records)
|
||||
origin = _url(env["ORIGIN_URL"]) if env.get("ORIGIN_URL") else (None, None)
|
||||
if origin[0] is None:
|
||||
return None # configured by its YAML file, which is not read
|
||||
replicas = sorted(
|
||||
(key for key in env if re.fullmatch(r"REPLICA[0-9]*_URL", key)),
|
||||
key=lambda key: int(key[7:-4] or 0),
|
||||
)
|
||||
members = [_member(address=origin[0], port=origin[1], role="origin")]
|
||||
for key in replicas:
|
||||
host, port = _url(env[key])
|
||||
if host:
|
||||
members.append(_member(address=host, port=port, role="replica", config_from=0))
|
||||
cluster = f"{origin[0]}:{origin[1]}" if origin[1] else origin[0]
|
||||
return [_cluster("adguardhome-sync", "adguardhome", name, cluster, members)]
|
||||
|
||||
|
||||
def _peer_address(entry: str) -> str | None:
|
||||
"""An entry of a whitelist that names one peer: a single address that is no
|
||||
loopback, link-local or unspecified one -- not a network."""
|
||||
text = entry.strip()
|
||||
if "/" in text:
|
||||
try:
|
||||
network = ipaddress.ip_network(text, strict=False)
|
||||
except ValueError:
|
||||
return None
|
||||
if network.num_addresses != 1:
|
||||
return None
|
||||
text = str(network.network_address)
|
||||
try:
|
||||
address = ipaddress.ip_address(text)
|
||||
except ValueError:
|
||||
return None
|
||||
if address.is_loopback or address.is_link_local or address.is_unspecified:
|
||||
return None
|
||||
return str(address)
|
||||
|
||||
|
||||
def _api_port(env: dict[str, str]) -> int:
|
||||
port = env.get("API_HTTP_PORT", "")
|
||||
return int(port) if port.isdigit() else _BUNKERWEB_API_PORT
|
||||
|
||||
|
||||
def _bunkerweb_worker(name: str, env: dict[str, str]) -> list[ServiceClusterDict]:
|
||||
managers = [
|
||||
address
|
||||
for entry in re.split(r"[\s,]+", env.get("API_WHITELIST_IP", ""))
|
||||
if entry and (address := _peer_address(entry))
|
||||
]
|
||||
if not managers:
|
||||
return []
|
||||
members = [_member(port=_api_port(env), role="worker", this_host=True, config_from=1)]
|
||||
members += [_member(address=address, role="manager") for address in managers]
|
||||
return [_cluster("bunkerweb", "bunkerweb", name, managers[0], members)]
|
||||
|
||||
|
||||
def _bunkerweb_manager(name: str, env: dict[str, str]) -> list[ServiceClusterDict]:
|
||||
instances = [entry for entry in env.get("BUNKERWEB_INSTANCES", "").split() if entry]
|
||||
if not instances:
|
||||
return []
|
||||
members = [_member(role="manager", this_host=True)]
|
||||
for entry in instances:
|
||||
host, port = _host_port(entry)
|
||||
if host:
|
||||
members.append(
|
||||
_member(
|
||||
address=host, port=port or _BUNKERWEB_API_PORT, role="worker", config_from=0
|
||||
)
|
||||
)
|
||||
return [_cluster("bunkerweb", "bunkerweb", name, None, members)] if len(members) > 1 else []
|
||||
|
||||
|
||||
def _bunkerweb(name: str, records: list[_Record]) -> list[ServiceClusterDict]:
|
||||
image = _image_name(records)
|
||||
if image == "bunkerweb":
|
||||
return _bunkerweb_worker(name, _env(records))
|
||||
if image in ("bunkerweb-all-in-one", "bunkerweb-scheduler"):
|
||||
return _bunkerweb_manager(name, _env(records))
|
||||
return [] # the UI, the API, autoconf: no instance and no manager
|
||||
|
||||
|
||||
_CONTAINER_READERS: dict[str, Callable[[str, list[_Record]], list[ServiceClusterDict] | None]] = {
|
||||
"adguardhome_sync": _adguardhome_sync,
|
||||
"bunkerweb": _bunkerweb,
|
||||
}
|
||||
|
||||
|
||||
def _reader_for(records: list[_Record]) -> str | None:
|
||||
image = next((text for kind, text in records if kind == "image"), "")
|
||||
if "adguardhome-sync" in image:
|
||||
return "adguardhome_sync"
|
||||
if "bunkerweb" in image:
|
||||
return "bunkerweb"
|
||||
return None
|
||||
|
||||
|
||||
def _containers(sections: list[_Section]) -> dict[str, list[ServiceClusterDict] | None]:
|
||||
kinds = {kind for kind, _, _ in sections}
|
||||
if "no-docker" in kinds:
|
||||
return {"adguardhome_sync": [], "bunkerweb": []}
|
||||
if "docker" not in kinds:
|
||||
return {"adguardhome_sync": None, "bunkerweb": None}
|
||||
found: dict[str, list[ServiceClusterDict] | None] = {"adguardhome_sync": [], "bunkerweb": []}
|
||||
for kind, name, records in sections:
|
||||
key = _reader_for(records) if kind == "container" else None
|
||||
if key is None or found[key] is None:
|
||||
continue
|
||||
clusters = _CONTAINER_READERS[key](name, records)
|
||||
if clusters is None:
|
||||
found[key] = None
|
||||
else:
|
||||
found[key].extend(clusters) # type: ignore[union-attr]
|
||||
return found
|
||||
|
||||
|
||||
def parse_service_clusters(output: str) -> ServiceClustersDict:
|
||||
"""Parse what :func:`service_clusters_command` printed.
|
||||
|
||||
:raises ValueError: when the output carries no intact report.
|
||||
"""
|
||||
sections = _sections(_frame(output))
|
||||
patroni: list[ServiceClusterDict] | None
|
||||
if any(kind == "no-systemd" for kind, _, _ in sections):
|
||||
patroni = None
|
||||
elif any(
|
||||
kind == "patroni" and any(r[0] == "unreadable" for r in records)
|
||||
for kind, _, records in sections
|
||||
):
|
||||
patroni = None
|
||||
else:
|
||||
patroni = [
|
||||
c
|
||||
for kind, unit, records in sections
|
||||
if kind == "patroni"
|
||||
for c in _patroni(unit, records)
|
||||
]
|
||||
containers = _containers(sections)
|
||||
return {
|
||||
"patroni": patroni,
|
||||
"adguardhome_sync": containers["adguardhome_sync"],
|
||||
"bunkerweb": containers["bunkerweb"],
|
||||
}
|
||||
|
||||
|
||||
class ServiceClustersMixin:
|
||||
"""Adds :meth:`get_service_clusters` to a driver that can run a command on a Linux host.
|
||||
|
||||
The template form (README, "Function classes"): the command and its parse are
|
||||
the same everywhere, so they are concrete here, and a driver supplies only
|
||||
:meth:`_run_service_clusters_command` -- how a command reaches its host, and
|
||||
how it gains root there. A driver with a container engine of its own path
|
||||
(``ContainerEngineMixin._container_engine_binary``) has it asked with that.
|
||||
Mixed in by the drivers that can, so ``hasattr(driver,
|
||||
"get_service_clusters")`` stays a truthful answer.
|
||||
"""
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def _run_service_clusters_command(self, command: str, *, privileged: bool) -> str:
|
||||
"""Run *command* on the host and return what it printed; as root when
|
||||
*privileged*. The command is a single ``sh -c`` invocation of a few kB:
|
||||
send it on an exec channel, not typed into a shell."""
|
||||
...
|
||||
|
||||
def _service_clusters_docker(self) -> str:
|
||||
binary = getattr(self, "_container_engine_binary", None)
|
||||
return binary("docker") if callable(binary) else "docker"
|
||||
|
||||
def get_service_clusters(self) -> ServiceClustersDict:
|
||||
"""
|
||||
Returns the clusters the host's configuration says it belongs to, per
|
||||
mechanism: Patroni (its scope and the members its DCS lists),
|
||||
adguardhome-sync (its origin and replicas), BunkerWeb (a worker's manager,
|
||||
a manager's instances). See :class:`~napalm_device_types.models.ServiceClusterDict`.
|
||||
|
||||
None for a mechanism means it could not be read; ``[]`` that the host has none.
|
||||
|
||||
Example::
|
||||
|
||||
{
|
||||
"patroni": [
|
||||
{"mechanism": "patroni", "service": "patroni", "source": "patroni",
|
||||
"cluster": "netork-db",
|
||||
"running": True,
|
||||
"members": [
|
||||
{"name": "db-01", "address": "10.7.234.12", "port": 5432,
|
||||
"role": "leader", "state": "running", "this_host": True,
|
||||
"config_from": None},
|
||||
{"name": "db-02", "address": "10.7.234.11", "port": 5432,
|
||||
"role": "sync_standby", "state": "streaming", "this_host": False,
|
||||
"config_from": None},
|
||||
]},
|
||||
],
|
||||
"adguardhome_sync": [],
|
||||
"bunkerweb": [],
|
||||
}
|
||||
|
||||
:raises ValueError: if neither reading carried an intact report.
|
||||
"""
|
||||
command = f"sh -c {quote(service_clusters_command(self._service_clusters_docker()))}"
|
||||
try:
|
||||
return parse_service_clusters(
|
||||
self._run_service_clusters_command(command, privileged=True)
|
||||
)
|
||||
except ValueError:
|
||||
pass
|
||||
return parse_service_clusters(self._run_service_clusters_command(command, privileged=False))
|
||||
@@ -0,0 +1,182 @@
|
||||
"""Which installed packages a systemd service is made of.
|
||||
|
||||
A vulnerability is found on a *package*; a statement about a *service* -- the
|
||||
postgresql unit, nginx -- reaches its findings only once it is known which
|
||||
packages that service is. Here that is the package shipping the unit file and
|
||||
the one shipping the binary its ``ExecStart`` runs: ``nginx-common`` and
|
||||
``nginx-core``, ``postgresql-common`` and ``postgresql-16``. Reading it is the
|
||||
same on every systemd host, so the command and its parse live here once and a
|
||||
driver only carries the command across.
|
||||
|
||||
**Only the unit's own packages.** The libraries its binary links against
|
||||
(``libssl3`` under nginx) are every service's at once; a statement about nginx
|
||||
is not one about libc, and following the dependencies would make it one.
|
||||
|
||||
The command is read-only and needs no privileges. It prints ``systemctl show``
|
||||
for every loaded service unit and then asks the package manager about every
|
||||
path at once: ``dpkg -S`` once for all of them -- one call, since each loads the
|
||||
whole file list, and per path that is minutes on an ordinary host -- or
|
||||
``rpm -qf`` per path. dpkg knows a file by the path its package shipped, so each
|
||||
path is asked across the ``/usr`` merge too. The report is framed; one whose end
|
||||
is missing raises, because a list cut short must never read as services that own
|
||||
nothing.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
_BEGIN = "SVCPKG_BEGIN"
|
||||
_END = "SVCPKG_END"
|
||||
_SUFFIX = ".service"
|
||||
|
||||
#: A Debian or RPM package name.
|
||||
_PACKAGE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9+._-]*$")
|
||||
#: The binary of an ``ExecStart`` as ``systemctl show`` prints it.
|
||||
_EXEC_PATH = re.compile(r"\bpath=(/[^ ;]+)")
|
||||
|
||||
#: Every unit file and ``ExecStart`` binary in ``$s``, each also across the
|
||||
#: ``/usr`` merge, one per line.
|
||||
_PATHS = (
|
||||
"printf '%s\\n' \"$s\" | sed -n -e 's/^FragmentPath=\\(\\/.*\\)/\\1/p' "
|
||||
"-e 's/^ExecStart=.*path=\\(\\/[^ ;]*\\).*/\\1/p' | sort -u "
|
||||
"| sed -e 'p' -e 's#^/usr/#/#;t' -e 's#^/#/usr/#'"
|
||||
)
|
||||
|
||||
#: One line, POSIX ``sh``, read-only. The frame markers are printed in two
|
||||
#: halves so that a transport which echoes the command does not show them early.
|
||||
SERVICE_PACKAGES_COMMAND = (
|
||||
"s=$(systemctl list-units --type=service --all --no-legend --no-pager --plain 2>/dev/null "
|
||||
"| awk '{print $1}' | tr '\\n' '\\0' | xargs -0 -r systemctl show --no-pager "
|
||||
"-p Id -p FragmentPath -p ExecStart -- 2>/dev/null); "
|
||||
"printf '%s%s\\n' SVCPKG_ BEGIN; echo '[units]'; printf '%s\\n\\n' \"$s\"; "
|
||||
"if command -v dpkg >/dev/null 2>&1; then echo '[dpkg]'; "
|
||||
f"{_PATHS} | tr '\\n' '\\0' | xargs -0 -r dpkg -S 2>/dev/null; "
|
||||
"elif command -v rpm >/dev/null 2>&1; then echo '[rpm]'; "
|
||||
f"{_PATHS} | while IFS= read -r f; do "
|
||||
'printf \'%s\\t%s\\n\' "$f" "$(rpm -qf --qf \'%{NAME} \' "$f" 2>/dev/null)"; done; fi; '
|
||||
"printf '%s%s\\n' SVCPKG_ END"
|
||||
)
|
||||
|
||||
|
||||
def _report(output: str) -> dict[str, list[str]]:
|
||||
lines = [line.rstrip() for line in output.splitlines()]
|
||||
stripped = [line.strip() for line in lines]
|
||||
try:
|
||||
start = stripped.index(_BEGIN)
|
||||
end = stripped.index(_END, start)
|
||||
except ValueError:
|
||||
raise ValueError("no intact service package report in the output") from None
|
||||
sections: dict[str, list[str]] = {}
|
||||
current: list[str] | None = None
|
||||
for line in lines[start + 1 : end]:
|
||||
if line in ("[units]", "[dpkg]", "[rpm]"):
|
||||
current = sections.setdefault(line[1:-1], [])
|
||||
elif current is not None:
|
||||
current.append(line)
|
||||
return sections
|
||||
|
||||
|
||||
def _units(lines: list[str]) -> dict[str, set[str]]:
|
||||
"""``systemctl show`` blocks -> ``{unit: its unit file and binary}``."""
|
||||
units: dict[str, set[str]] = {}
|
||||
unit = ""
|
||||
paths: set[str] = set()
|
||||
for line in [*lines, ""]:
|
||||
key, _, value = line.partition("=")
|
||||
if not line.strip():
|
||||
if unit:
|
||||
units[unit] = paths
|
||||
unit, paths = "", set()
|
||||
elif key == "Id" and value.endswith(_SUFFIX):
|
||||
unit = value[: -len(_SUFFIX)]
|
||||
elif key == "FragmentPath" and value.startswith("/"):
|
||||
paths.add(value)
|
||||
elif key == "ExecStart":
|
||||
match = _EXEC_PATH.search(value)
|
||||
if match:
|
||||
paths.add(match.group(1))
|
||||
return units
|
||||
|
||||
|
||||
def _names(raw: str) -> set[str]:
|
||||
"""``libfoo1:amd64, foo-bin`` -> ``{"libfoo1", "foo-bin"}``; nothing else counts."""
|
||||
found = set()
|
||||
for name in raw.replace(",", " ").split():
|
||||
name = name.split(":", 1)[0]
|
||||
if _PACKAGE.match(name):
|
||||
found.add(name)
|
||||
return found
|
||||
|
||||
|
||||
def _owners(sections: dict[str, list[str]]) -> dict[str, set[str]]:
|
||||
"""``{path: the packages owning it}`` from dpkg's or rpm's answer."""
|
||||
owners: dict[str, set[str]] = {}
|
||||
for line in sections.get("dpkg", []):
|
||||
# "pkg[, pkg2][:arch]: /path"; "diversion by x from: /path" is no owner.
|
||||
if line.startswith("diversion by") or ": /" not in line:
|
||||
continue
|
||||
names, _, path = line.partition(": /")
|
||||
owners.setdefault("/" + path.strip(), set()).update(_names(names))
|
||||
for line in sections.get("rpm", []):
|
||||
path, _, names = line.partition("\t")
|
||||
if " is not owned" not in names:
|
||||
owners.setdefault(path.strip(), set()).update(_names(names))
|
||||
return owners
|
||||
|
||||
|
||||
def _across_usr_merge(path: str) -> tuple[str, str]:
|
||||
return (path, path[4:] if path.startswith("/usr/") else "/usr" + path)
|
||||
|
||||
|
||||
def parse_service_packages(output: str) -> dict[str, list[str]]:
|
||||
"""Parse what :data:`SERVICE_PACKAGES_COMMAND` printed.
|
||||
|
||||
``{unit name without ".service": its packages, sorted}``. A unit no package
|
||||
owns -- a unit file an admin wrote -- maps to an empty list: read, and none.
|
||||
|
||||
:raises ValueError: when the output carries no intact report.
|
||||
"""
|
||||
sections = _report(output)
|
||||
owners = _owners(sections)
|
||||
return {
|
||||
unit: sorted(
|
||||
{name for p in paths for alt in _across_usr_merge(p) for name in owners.get(alt, ())}
|
||||
)
|
||||
for unit, paths in _units(sections.get("units", [])).items()
|
||||
}
|
||||
|
||||
|
||||
class ServicePackagesMixin:
|
||||
"""Adds :meth:`get_service_packages` to a driver that can run a command on a systemd host.
|
||||
|
||||
The template form, like :class:`~napalm_device_types.kernel.KernelFactsMixin`:
|
||||
the reading and its parse are concrete here, and a driver supplies only
|
||||
:meth:`_run_service_packages_command` -- how a command reaches its host.
|
||||
"""
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def _run_service_packages_command(self, command: str) -> str:
|
||||
"""Run *command* on the host with ``sh`` and return what it printed."""
|
||||
...
|
||||
|
||||
def get_service_packages(self) -> dict[str, list[str]]:
|
||||
"""
|
||||
Returns, per systemd service, the installed packages it is made of.
|
||||
|
||||
The package that ships its unit file and the one that ships the binary
|
||||
its ``ExecStart`` runs -- not the libraries that binary links against.
|
||||
|
||||
Example::
|
||||
|
||||
{
|
||||
"nginx": ["nginx-common", "nginx-core"],
|
||||
"postgresql@16-main": ["postgresql-16", "postgresql-common"],
|
||||
"local-backup": [],
|
||||
}
|
||||
|
||||
:raises ValueError: if the host's output carried no intact report.
|
||||
"""
|
||||
return parse_service_packages(self._run_service_packages_command(SERVICE_PACKAGES_COMMAND))
|
||||
@@ -1,8 +1,7 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Init-system services: listing them and starting/stopping them.
|
||||
|
||||
Deliberately *not* the same thing as a NAS's exported shares, which live on
|
||||
``StorageServiceCapability`` as ``get_storage_services``. Those two used to
|
||||
``StorageDriver`` as ``get_storage_services``. Those two used to
|
||||
share the name ``get_services`` with incompatible return types, which is why
|
||||
a QNAP -- storage and OS at once -- could not satisfy both.
|
||||
|
||||
@@ -13,7 +12,7 @@ this class in can never shadow a working implementation from a sibling base.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any, Dict, List, TYPE_CHECKING
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from napalm_device_types.models import ServiceDict
|
||||
|
||||
@@ -21,7 +20,7 @@ from napalm_device_types.models import ServiceDict
|
||||
class ServiceControlMixin:
|
||||
if TYPE_CHECKING:
|
||||
|
||||
def get_services(self) -> List[ServiceDict]:
|
||||
def get_services(self) -> list[ServiceDict]:
|
||||
"""
|
||||
Returns the list of system services known to the device's init system.
|
||||
|
||||
@@ -42,7 +41,7 @@ class ServiceControlMixin:
|
||||
"""
|
||||
...
|
||||
|
||||
def manage_service(self, name: str, action: str) -> Dict[str, Any]:
|
||||
def manage_service(self, name: str, action: str) -> dict[str, Any]:
|
||||
"""
|
||||
Execute a lifecycle action on a named service.
|
||||
|
||||
|
||||
@@ -0,0 +1,60 @@
|
||||
"""The shape a driver states its device's SNMP health OIDs in (``SNMP_HEALTH_PROFILE``).
|
||||
|
||||
Apart from ``models.py``, which is over the file size limit; ``models`` still exports them.
|
||||
"""
|
||||
|
||||
from typing import Literal
|
||||
|
||||
from typing_extensions import TypedDict
|
||||
|
||||
|
||||
class SnmpValueSpec(TypedDict, total=False):
|
||||
"""Where an SNMP agent publishes one value, inside a ``SnmpHealthProfile``.
|
||||
|
||||
Either ``oids``, scalars asked in order (the first non-empty answer wins), or
|
||||
``table``, a column walked and folded to one value by ``fold``. ``pattern`` is a
|
||||
regular expression whose first group is the number, for an agent that answers
|
||||
with text.
|
||||
|
||||
``unknown`` says the agent answers this value wrongly, or not at all, where it
|
||||
should: the application reports no value -- neither the standard reading nor a
|
||||
default. It stands alone, without ``oids`` or ``table``.
|
||||
"""
|
||||
|
||||
oids: list[str]
|
||||
table: str
|
||||
fold: Literal["first", "average", "max"]
|
||||
pattern: str
|
||||
unknown: bool
|
||||
|
||||
|
||||
class SnmpMemorySpec(TypedDict, total=False):
|
||||
"""Memory: ``total`` with ``used`` or ``free``, or ``percent`` alone."""
|
||||
|
||||
total: SnmpValueSpec
|
||||
used: SnmpValueSpec
|
||||
free: SnmpValueSpec
|
||||
percent: SnmpValueSpec
|
||||
unit: Literal["KiB", "bytes"] # of total, used and free; KiB, as UCD-MIB counts, by default
|
||||
|
||||
|
||||
class SnmpInterfacesSpec(TypedDict, total=False):
|
||||
"""How the IF-MIB counters of a device are to be read."""
|
||||
|
||||
skip: str # a pattern of interface names whose counters are noise, not signal
|
||||
tx_errors_are_drops: bool # the device counts dropped packets as TX errors (OPNsense)
|
||||
|
||||
|
||||
class SnmpHealthProfile(TypedDict, total=False):
|
||||
"""Where a device's SNMP agent publishes its health values.
|
||||
|
||||
A driver declares it as ``SNMP_HEALTH_PROFILE`` when its device does not answer
|
||||
the standard UCD-MIB and IF-MIB; the application polls and computes. What a
|
||||
profile does not name is read the standard way, and what it marks ``unknown`` is
|
||||
not read at all. Uptime is in TimeTicks, CPU in percent busy.
|
||||
"""
|
||||
|
||||
uptime: SnmpValueSpec
|
||||
cpu: SnmpValueSpec
|
||||
memory: SnmpMemorySpec
|
||||
interfaces: SnmpInterfacesSpec
|
||||
@@ -0,0 +1,354 @@
|
||||
"""SSH host keys: trust on first use, and a changed key refused before any login.
|
||||
|
||||
A driver that accepts whatever host key answers at a device's address logs in
|
||||
there with the device's credentials, whoever that is (NetOrk/netork#1167). The
|
||||
layer above keeps the store of the keys it trusts; a driver only verifies:
|
||||
|
||||
* It receives the keys to expect in ``optional_args["ssh_host_keys"]``
|
||||
(:data:`KNOWN_KEYS_ARG`), by the port they were seen on:
|
||||
``{"22": {"key_type": "ssh-ed25519", "key": "<base64>"}}``, the two fields of
|
||||
a ``known_hosts`` line. A driver takes the one for the port it connects to, so
|
||||
the caller need not know which port that is (Proxmox: its API on 8006, SSH on
|
||||
22). With a key, any other key is refused before a password or a private key
|
||||
leaves the caller. Without one, the first key is accepted: trust on first use.
|
||||
* It reports every key it sees through ``optional_args["ssh_host_key_observer"]``
|
||||
(:data:`OBSERVER_ARG`), a callable that receives one dict per connection::
|
||||
|
||||
{"host": str, "port": int,
|
||||
"seen": {"key_type", "key", "fingerprint"},
|
||||
"expected": {...} | None,
|
||||
"accepted": bool}
|
||||
|
||||
``accepted`` is False only for a refused key; the connection then raises
|
||||
:class:`SshHostKeyMismatch`. A failing observer is logged and ignored, so the
|
||||
store being away never stops a connection that verified.
|
||||
|
||||
A caller that passes neither keeps working as before (first use, every time),
|
||||
and a driver that ignores both still connects for a caller that passes them:
|
||||
old and new versions on either side do not break each other.
|
||||
|
||||
A connection sshd turns away before its banner -- OpenSSH 9.8+ does that to a
|
||||
source it has just penalised -- is tried again after a pause
|
||||
(:func:`retry_refused`, :data:`REFUSED_RETRY_WAITS`); :meth:`HostKeyCheck.connect`
|
||||
does so itself, a netmiko driver wraps its ``ConnectHandler`` in it.
|
||||
|
||||
:class:`HostKeyCheck` is the one implementation, for a paramiko ``SSHClient``
|
||||
(:meth:`~HostKeyCheck.connect`) and for netmiko (:meth:`~HostKeyCheck.netmiko`,
|
||||
then :meth:`~HostKeyCheck.observe`). Like :mod:`napalm_device_types.channel` it
|
||||
works on the objects paramiko hands it and imports no SSH library itself.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import hashlib
|
||||
import logging
|
||||
import os
|
||||
import tempfile
|
||||
import time
|
||||
from collections.abc import Callable, Iterator, Mapping
|
||||
from contextlib import contextmanager
|
||||
from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
from napalm.base.exceptions import ConnectionException
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
#: The ``optional_args`` key that carries the host keys to expect, by port.
|
||||
KNOWN_KEYS_ARG = "ssh_host_keys"
|
||||
#: The ``optional_args`` key that carries the callable every seen key goes to.
|
||||
OBSERVER_ARG = "ssh_host_key_observer"
|
||||
|
||||
#: The waits before each new attempt when sshd turns a connection away before its
|
||||
#: banner. OpenSSH 9.8+ does that to every connection from a source it has just
|
||||
#: penalised (``PerSourcePenalties``) -- one whose connections did not
|
||||
#: authenticate, a scanner's probes -- for seconds to minutes. 26 s in all
|
||||
#: outlasts the 16 s penalty seen on a new Ubuntu 26.04 host (NetOrk/netork#1800).
|
||||
REFUSED_RETRY_WAITS = (1.0, 3.0, 7.0, 15.0)
|
||||
|
||||
#: The netmiko arguments this module decides; a caller's own values give way.
|
||||
_NETMIKO_HOST_KEY_ARGS = ("ssh_strict", "system_host_keys", "alt_host_keys", "alt_key_file")
|
||||
|
||||
_DEFAULT_SSH_PORT = 22
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SshHostKey:
|
||||
"""A host's public key, as the two fields of a ``known_hosts`` line."""
|
||||
|
||||
key_type: str
|
||||
#: The public key blob, base64.
|
||||
key: str
|
||||
|
||||
@property
|
||||
def fingerprint(self) -> str:
|
||||
"""``SHA256:…``, as ``ssh-keygen -l`` and the ssh client print it."""
|
||||
digest = hashlib.sha256(base64.b64decode(self.key)).digest()
|
||||
return "SHA256:" + base64.b64encode(digest).decode("ascii").rstrip("=")
|
||||
|
||||
def as_dict(self) -> dict[str, str]:
|
||||
return {"key_type": self.key_type, "key": self.key, "fingerprint": self.fingerprint}
|
||||
|
||||
@classmethod
|
||||
def from_value(cls, value: Any) -> SshHostKey | None:
|
||||
"""The key in *value* (a mapping with ``key_type`` and ``key``), or None for none."""
|
||||
if not value:
|
||||
return None
|
||||
if not isinstance(value, Mapping) or not value.get("key_type") or not value.get("key"):
|
||||
raise ValueError(f"not an SSH host key: {value!r}")
|
||||
return cls(str(value["key_type"]), str(value["key"]))
|
||||
|
||||
@classmethod
|
||||
def of(cls, pkey: Any) -> SshHostKey:
|
||||
"""The key of a paramiko ``PKey`` (what a transport says the server presented)."""
|
||||
return cls(pkey.get_name(), pkey.get_base64())
|
||||
|
||||
def known_hosts_line(self, host: str, port: int) -> str:
|
||||
"""The line under which an ssh client looks this host up."""
|
||||
name = host if port == _DEFAULT_SSH_PORT else f"[{host}]:{port}"
|
||||
return f"{name} {self.key_type} {self.key}"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class HostKeyObservation:
|
||||
"""The key one connection saw, against the key it was to expect."""
|
||||
|
||||
host: str
|
||||
port: int
|
||||
seen: SshHostKey
|
||||
expected: SshHostKey | None
|
||||
|
||||
@property
|
||||
def accepted(self) -> bool:
|
||||
"""True on first use and for the expected key; False for a refused one."""
|
||||
return self.expected is None or self.expected == self.seen
|
||||
|
||||
def as_dict(self) -> dict[str, Any]:
|
||||
return {
|
||||
"host": self.host,
|
||||
"port": self.port,
|
||||
"seen": self.seen.as_dict(),
|
||||
"expected": self.expected.as_dict() if self.expected else None,
|
||||
"accepted": self.accepted,
|
||||
}
|
||||
|
||||
|
||||
class SshHostKeyMismatch(ConnectionException): # type: ignore[misc]
|
||||
"""The device presented another host key than the one it is known by.
|
||||
|
||||
Raised before any authentication: no password and no private key went to it.
|
||||
"""
|
||||
|
||||
def __init__(self, observation: HostKeyObservation) -> None:
|
||||
self.observation = observation
|
||||
expected = observation.expected
|
||||
trusted = f"{expected.fingerprint} ({expected.key_type})" if expected else "none"
|
||||
super().__init__(
|
||||
f"SSH host key mismatch at {observation.host}:{observation.port}: it presented "
|
||||
f"{observation.seen.fingerprint} ({observation.seen.key_type}), the trusted key is "
|
||||
f"{trusted}. Nothing was sent to it."
|
||||
)
|
||||
|
||||
|
||||
class _AcceptFirst:
|
||||
"""A paramiko missing-host-key policy for first use: accept, keep nothing.
|
||||
|
||||
The key is reported once the connection stands (:meth:`HostKeyCheck.observe`);
|
||||
nothing is written to a ``known_hosts`` file.
|
||||
"""
|
||||
|
||||
def missing_host_key(self, client: Any, hostname: str, key: Any) -> None:
|
||||
return None
|
||||
|
||||
|
||||
class _RefuseUnknown:
|
||||
"""A paramiko policy for a host with a known key: refuse anything else.
|
||||
|
||||
paramiko compares a loaded key itself and raises ``BadHostKeyException``
|
||||
for another one; this only answers if the loaded key was not found under
|
||||
the name connected to, which must not turn into an accepted key.
|
||||
"""
|
||||
|
||||
def missing_host_key(self, client: Any, hostname: str, key: Any) -> None:
|
||||
raise ConnectionException(f"SSH host key of {hostname} is not the one expected")
|
||||
|
||||
|
||||
def _expected_key(known: Any, port: int) -> SshHostKey | None:
|
||||
"""The key in *known* (``{port: key}``, the port as text or number) for *port*."""
|
||||
if not known:
|
||||
return None
|
||||
if not isinstance(known, Mapping):
|
||||
raise ValueError(f"SSH host keys must be given by port, not as {type(known).__name__}")
|
||||
return SshHostKey.from_value(known.get(str(port)) or known.get(port))
|
||||
|
||||
|
||||
def _refused_key(exc: BaseException) -> Any | None:
|
||||
"""paramiko's ``BadHostKeyException`` in *exc*'s chain (netmiko and drivers wrap it)."""
|
||||
seen: set[int] = set()
|
||||
stack: list[BaseException | None] = [exc]
|
||||
while stack:
|
||||
current = stack.pop()
|
||||
if current is None or id(current) in seen:
|
||||
continue
|
||||
seen.add(id(current))
|
||||
if hasattr(current, "expected_key") and hasattr(current, "key"):
|
||||
return current
|
||||
stack.extend([current.__cause__, current.__context__])
|
||||
return None
|
||||
|
||||
|
||||
def refused_before_banner(exc: BaseException) -> bool:
|
||||
"""Whether *exc*, or what it wraps, is sshd closing the connection before its banner.
|
||||
|
||||
paramiko says "Error reading SSH protocol banner" (netmiko and drivers wrap
|
||||
it), with the connection's EOF or reset underneath. A failed login, a
|
||||
silent host and a closed port are something else: waiting does not end them.
|
||||
"""
|
||||
seen: set[int] = set()
|
||||
stack: list[BaseException | None] = [exc]
|
||||
while stack:
|
||||
current = stack.pop()
|
||||
if current is None or id(current) in seen:
|
||||
continue
|
||||
seen.add(id(current))
|
||||
if "protocol banner" in str(current):
|
||||
return True
|
||||
stack.extend([current.__cause__, current.__context__])
|
||||
return False
|
||||
|
||||
|
||||
def retry_refused[T](
|
||||
attempt: Callable[[], T],
|
||||
*,
|
||||
waits: tuple[float, ...] = REFUSED_RETRY_WAITS,
|
||||
sleep: Callable[[float], None] | None = None,
|
||||
) -> T:
|
||||
"""*attempt*, again after each of *waits* while sshd turns it away before its banner.
|
||||
|
||||
Any other error, and the last refusal, go to the caller unchanged.
|
||||
"""
|
||||
pause = sleep or time.sleep
|
||||
for wait in waits:
|
||||
try:
|
||||
return attempt()
|
||||
except Exception as exc:
|
||||
if not refused_before_banner(exc):
|
||||
raise
|
||||
logger.info("SSH connection turned away before its banner; again in %s s", wait)
|
||||
pause(wait)
|
||||
return attempt()
|
||||
|
||||
|
||||
class HostKeyCheck:
|
||||
"""How one driver connection verifies its host's key.
|
||||
|
||||
*optional_args* are the driver's (both keys optional), *host* and *port* the
|
||||
address the connection goes to, as given to paramiko or netmiko.
|
||||
"""
|
||||
|
||||
def __init__(self, optional_args: Mapping[str, Any] | None, host: str, port: int) -> None:
|
||||
args = optional_args or {}
|
||||
self.host = host
|
||||
self.port = int(port)
|
||||
self.expected = _expected_key(args.get(KNOWN_KEYS_ARG), self.port)
|
||||
self._observer = args.get(OBSERVER_ARG)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# paramiko
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def prepare(self, client: Any) -> None:
|
||||
"""Before ``client.connect()``: the known key, or acceptance of the first one.
|
||||
|
||||
The known key goes into the client's host keys, so paramiko asks the
|
||||
server for that key type first and refuses any other key itself.
|
||||
"""
|
||||
if self.expected is None:
|
||||
client.set_missing_host_key_policy(_AcceptFirst())
|
||||
return
|
||||
with self._known_hosts_file(self.expected) as path:
|
||||
client.get_host_keys().load(path)
|
||||
client.set_missing_host_key_policy(_RefuseUnknown())
|
||||
|
||||
def connect(self, client: Any, **kwargs: Any) -> HostKeyObservation:
|
||||
"""``client.connect()`` to this host with *kwargs*, verified and reported."""
|
||||
self.prepare(client)
|
||||
with self.refusing():
|
||||
retry_refused(lambda: client.connect(hostname=self.host, port=self.port, **kwargs))
|
||||
return self.observe(client.get_transport())
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# netmiko
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
@contextmanager
|
||||
def netmiko(self, base: Mapping[str, Any]) -> Iterator[dict[str, Any]]:
|
||||
"""``ConnectHandler`` keyword arguments: *base* with the host key decided.
|
||||
|
||||
Open the connection inside the block and call :meth:`observe` on its
|
||||
transport after it. A known key goes to netmiko in a ``known_hosts``
|
||||
file that exists only for the block, with ``ssh_strict``: should the
|
||||
file not be read, netmiko refuses rather than accepts.
|
||||
"""
|
||||
kwargs = {k: v for k, v in base.items() if k not in _NETMIKO_HOST_KEY_ARGS}
|
||||
kwargs["system_host_keys"] = False
|
||||
if self.expected is None:
|
||||
kwargs.update(ssh_strict=False, alt_host_keys=False)
|
||||
with self.refusing():
|
||||
yield kwargs
|
||||
return
|
||||
with self._known_hosts_file(self.expected) as path:
|
||||
kwargs.update(ssh_strict=True, alt_host_keys=True, alt_key_file=path)
|
||||
with self.refusing():
|
||||
yield kwargs
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# both
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
@contextmanager
|
||||
def refusing(self) -> Iterator[None]:
|
||||
"""Around a connect: a refused host key becomes :class:`SshHostKeyMismatch`."""
|
||||
try:
|
||||
yield
|
||||
except Exception as exc:
|
||||
refused = _refused_key(exc)
|
||||
if refused is None:
|
||||
raise
|
||||
expected = self.expected or SshHostKey.of(refused.expected_key)
|
||||
offered = SshHostKey.of(refused.key)
|
||||
observation = HostKeyObservation(self.host, self.port, offered, expected)
|
||||
self._report(observation)
|
||||
raise SshHostKeyMismatch(observation) from exc
|
||||
|
||||
def observe(self, transport: Any) -> HostKeyObservation:
|
||||
"""After a connect: the key the server presented on *transport*, reported."""
|
||||
seen = SshHostKey.of(transport.get_remote_server_key())
|
||||
observation = HostKeyObservation(self.host, self.port, seen, self.expected)
|
||||
self._report(observation)
|
||||
return observation
|
||||
|
||||
def _report(self, observation: HostKeyObservation) -> None:
|
||||
if not callable(self._observer):
|
||||
return
|
||||
try:
|
||||
self._observer(observation.as_dict())
|
||||
except Exception: # the store is the caller's; a connection does not hang on it
|
||||
logger.warning(
|
||||
"Could not report the SSH host key of %s:%s", self.host, self.port, exc_info=True
|
||||
)
|
||||
|
||||
@contextmanager
|
||||
def _known_hosts_file(self, expected: SshHostKey) -> Iterator[str]:
|
||||
"""A private ``known_hosts`` file holding only *expected*."""
|
||||
fd, path = tempfile.mkstemp(prefix="known_hosts-")
|
||||
try:
|
||||
with os.fdopen(fd, "w") as f:
|
||||
f.write(expected.known_hosts_line(self.host, self.port) + "\n")
|
||||
yield path
|
||||
finally:
|
||||
try:
|
||||
os.unlink(path)
|
||||
except OSError:
|
||||
pass
|
||||
@@ -0,0 +1,820 @@
|
||||
"""OpenSSH server hardening: taking algorithms out of what sshd offers.
|
||||
|
||||
A scanner such as ssh-audit names algorithms a server should stop offering --
|
||||
``hmac-sha1``, the NIST curves. Taking them out is the same on every host that
|
||||
runs OpenSSH, so the reading, the plan and the write live here once, and a
|
||||
driver only mixes in :class:`OpenSshServerMixin` and supplies ``run_command``.
|
||||
|
||||
**What the server offers is read, not assumed.** ``sshd -T`` prints the lists
|
||||
sshd actually uses, after its defaults, ``sshd_config``, every ``Include`` and
|
||||
first-match-wins. The new list is that one minus what was asked, so nothing the
|
||||
host already restricted comes back, and nothing it allows is dropped by
|
||||
accident.
|
||||
|
||||
**A drop-in, not an edit.** Where ``sshd_config`` includes
|
||||
``/etc/ssh/sshd_config.d/*.conf`` -- Debian and Ubuntu do, at the top -- the
|
||||
lists go to :data:`DROPIN_PATH`, which sorts first and so wins. sshd_config
|
||||
stays as the distribution shipped it, and removing the file undoes the change.
|
||||
Only without such an ``Include`` is sshd_config itself rewritten, after a copy --
|
||||
and not at all on a platform that generates it
|
||||
(:attr:`OpenSshServerMixin.SSHD_CONFIG_GENERATED`).
|
||||
|
||||
**Nobody is locked out, by three checks.**
|
||||
|
||||
1. The plan never removes the last algorithm of a class, and -- given the
|
||||
lists of the client that manages the host -- never the last one that client
|
||||
speaks. Each one kept back is reported, with why. For host key algorithms
|
||||
only those the host has a key for count (``ssh-keygen -l`` on each
|
||||
``HostKey``); when the keys cannot be read, the host key list is not touched.
|
||||
2. ``sshd -t`` must accept the result, and ``sshd -T`` must show the new lists.
|
||||
A file read before ours can set the same directive; ours is then valid and
|
||||
ignored, and that is reported rather than called a success.
|
||||
3. After the reload, *verify* -- the caller's -- opens a new session. The
|
||||
running one survives a reload, so if the new one cannot get in, this one
|
||||
puts the old state back and reloads again.
|
||||
|
||||
**The managing client is exempt from per-source penalties** where sshd has them
|
||||
(OpenSSH 9.8+, :meth:`OpenSshServerMixin.exempt_from_ssh_penalties`). sshd
|
||||
penalises a source whose connections do not authenticate -- a scanner's probes
|
||||
-- and then drops every connection from it, the managing session's included.
|
||||
The address is the one the host sees (``$SSH_CONNECTION``); it joins the
|
||||
host's own ``PerSourcePenaltyExemptList`` in :data:`PENALTY_DROPIN_PATH`, with
|
||||
the same three checks. An older sshd is not touched: it refuses the directive.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import ipaddress
|
||||
import time
|
||||
from collections.abc import Callable, Iterable, Mapping, Sequence
|
||||
from dataclasses import dataclass, field
|
||||
from shlex import quote
|
||||
from typing import (
|
||||
TYPE_CHECKING,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - for type checkers only
|
||||
from napalm_device_types.channel import CommandResult
|
||||
|
||||
#: The algorithm classes: key -> (``sshd -T`` keyword, sshd_config directive).
|
||||
SSH_ALGORITHM_CLASSES: dict[str, tuple[str, str]] = {
|
||||
"kex": ("kexalgorithms", "KexAlgorithms"),
|
||||
"hostkey": ("hostkeyalgorithms", "HostKeyAlgorithms"),
|
||||
"cipher": ("ciphers", "Ciphers"),
|
||||
"mac": ("macs", "MACs"),
|
||||
}
|
||||
|
||||
MAIN_CONFIG = "/etc/ssh/sshd_config"
|
||||
DROPIN_DIR = "/etc/ssh/sshd_config.d"
|
||||
#: ``00-`` so it sorts before every other drop-in: sshd keeps the first value it reads.
|
||||
DROPIN_PATH = f"{DROPIN_DIR}/00-netork-ssh-hardening.conf"
|
||||
#: Where :meth:`OpenSshServerMixin.exempt_from_ssh_penalties` names the managing
|
||||
#: client; ``00-`` like the hardening drop-in, since sshd keeps the first value.
|
||||
PENALTY_DROPIN_PATH = f"{DROPIN_DIR}/00-netork-penalty-exempt.conf"
|
||||
#: What marks a line this module disabled in sshd_config.
|
||||
DISABLED_PREFIX = "# netork: "
|
||||
|
||||
_HEADER = (
|
||||
"# Written by netOrk: the SSH algorithms a security assessment flagged,\n"
|
||||
"# taken out of what this server offers. Delete this file to undo it.\n"
|
||||
)
|
||||
|
||||
_PENALTY_HEADER = (
|
||||
"# Written by netOrk: the address netOrk manages this host from, exempt from\n"
|
||||
"# sshd's per-source penalties (PerSourcePenalties). Delete this file to undo it.\n"
|
||||
)
|
||||
|
||||
#: Host key type, as ``ssh-keygen -l`` prints it -> the algorithms that key serves.
|
||||
#: Certificate algorithms are left out: they also need a HostCertificate.
|
||||
_KEY_TYPE_ALGORITHMS: dict[str, tuple[str, ...]] = {
|
||||
"ED25519": ("ssh-ed25519",),
|
||||
"RSA": ("rsa-sha2-512", "rsa-sha2-256", "ssh-rsa"),
|
||||
"DSA": ("ssh-dss",),
|
||||
"ED25519-SK": ("sk-ssh-ed25519@openssh.com",),
|
||||
"ECDSA-SK": ("sk-ecdsa-sha2-nistp256@openssh.com",),
|
||||
}
|
||||
_ECDSA_CURVES = ("256", "384", "521")
|
||||
|
||||
_RELOAD = (
|
||||
"if systemctl is-active --quiet ssh 2>/dev/null; then systemctl reload ssh;"
|
||||
" elif systemctl is-active --quiet sshd 2>/dev/null; then systemctl reload sshd;"
|
||||
" else for f in /run/sshd.pid /var/run/sshd.pid; do"
|
||||
' [ -r "$f" ] && kill -HUP "$(cat "$f")" && exit 0; done; exit 0; fi'
|
||||
)
|
||||
|
||||
|
||||
def _directive_key(line: str) -> str | None:
|
||||
"""The algorithm class a global sshd_config line sets, if it sets one."""
|
||||
parts = line.split(None, 1)
|
||||
if not parts:
|
||||
return None
|
||||
word = parts[0].lower()
|
||||
for kind, (keyword, _directive) in SSH_ALGORITHM_CLASSES.items():
|
||||
if word == keyword:
|
||||
return kind
|
||||
return None
|
||||
|
||||
|
||||
def parse_sshd_effective(text: str) -> dict[str, list[str]]:
|
||||
"""The four algorithm lists from ``sshd -T`` output, in the server's order.
|
||||
|
||||
:raises ValueError: if a list is missing -- not sshd -T output, or cut short.
|
||||
"""
|
||||
found: dict[str, list[str]] = {}
|
||||
for raw in text.splitlines():
|
||||
parts = raw.strip().split(None, 1)
|
||||
if len(parts) != 2:
|
||||
continue
|
||||
kind = _directive_key(parts[0])
|
||||
if kind and kind not in found:
|
||||
found[kind] = [a for a in parts[1].split(",") if a]
|
||||
missing = [k for k in SSH_ALGORITHM_CLASSES if k not in found]
|
||||
if missing:
|
||||
raise ValueError(f"sshd -T printed no {', '.join(missing)} list")
|
||||
return found
|
||||
|
||||
|
||||
def parse_sshd_hostkeys(text: str) -> list[str]:
|
||||
"""The ``hostkey`` paths from ``sshd -T`` output, in order."""
|
||||
paths: list[str] = []
|
||||
for raw in text.splitlines():
|
||||
parts = raw.strip().split(None, 1)
|
||||
if len(parts) == 2 and parts[0].lower() == "hostkey":
|
||||
paths.append(parts[1].strip())
|
||||
return paths
|
||||
|
||||
|
||||
def host_key_algorithms(fingerprints: str) -> set[str] | None:
|
||||
"""The host key algorithms the keys in ``ssh-keygen -l`` output can serve.
|
||||
|
||||
None when a line is not understood or there is no key: then nothing is known.
|
||||
"""
|
||||
usable: set[str] = set()
|
||||
for raw in fingerprints.splitlines():
|
||||
parts = raw.split()
|
||||
if not parts:
|
||||
continue
|
||||
kind = parts[-1].strip("()")
|
||||
if len(parts) < 3 or not parts[-1].startswith("("):
|
||||
return None
|
||||
if kind == "ECDSA" and parts[0] in _ECDSA_CURVES:
|
||||
usable.add(f"ecdsa-sha2-nistp{parts[0]}")
|
||||
elif kind in _KEY_TYPE_ALGORITHMS:
|
||||
usable.update(_KEY_TYPE_ALGORITHMS[kind])
|
||||
else:
|
||||
return None
|
||||
return usable or None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Refusal:
|
||||
"""One algorithm that was asked to go and stays, with why.
|
||||
|
||||
*reason* is ``"last"`` (the class would be empty -- for host keys: have no
|
||||
algorithm the host has a key for), ``"client"`` (the managing client would
|
||||
share none of the class with the server), ``"host_keys_unknown"`` (the host
|
||||
keys could not be read, so the host key list stays) or ``"not_offered"``
|
||||
(the server does not offer it, so there is nothing to remove).
|
||||
"""
|
||||
|
||||
kind: str
|
||||
algorithm: str
|
||||
reason: str
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SshHardeningPlan:
|
||||
keep: dict[str, list[str]]
|
||||
remove: dict[str, list[str]]
|
||||
refused: list[Refusal] = field(default_factory=list)
|
||||
not_offered: list[Refusal] = field(default_factory=list)
|
||||
|
||||
@property
|
||||
def changed_classes(self) -> list[str]:
|
||||
return [k for k in SSH_ALGORITHM_CLASSES if self.remove.get(k)]
|
||||
|
||||
|
||||
def plan_ssh_hardening(
|
||||
effective: Mapping[str, Sequence[str]],
|
||||
remove: Mapping[str, Iterable[str]],
|
||||
*,
|
||||
client: Mapping[str, Iterable[str]] | None = None,
|
||||
host_keys: Iterable[str] | None = None,
|
||||
) -> SshHardeningPlan:
|
||||
"""What the lists become when *remove* is taken out of *effective*.
|
||||
|
||||
*client* is what the client managing this host speaks, per class. A class it
|
||||
has no list for is not guarded beyond "never empty".
|
||||
|
||||
*host_keys* are the host key algorithms the host has a key for (see
|
||||
:func:`host_key_algorithms`). Only those count as a host key algorithm that
|
||||
stays: sshd offers the others but cannot use them. Without it nothing is
|
||||
known, and no host key algorithm is removed.
|
||||
|
||||
When a removal would leave a class empty, or without an algorithm *client*
|
||||
speaks, the first of the asked-for algorithms in the server's order that
|
||||
fixes it stays, and is reported in ``refused``.
|
||||
|
||||
:raises ValueError: for a class that is not one of :data:`SSH_ALGORITHM_CLASSES`.
|
||||
"""
|
||||
unknown = [k for k in remove if k not in SSH_ALGORITHM_CLASSES]
|
||||
if unknown:
|
||||
raise ValueError(f"unknown SSH algorithm class: {', '.join(unknown)}")
|
||||
keep: dict[str, list[str]] = {}
|
||||
removed: dict[str, list[str]] = {}
|
||||
refused: list[Refusal] = []
|
||||
not_offered: list[Refusal] = []
|
||||
usable_keys = set(host_keys) if host_keys is not None else None
|
||||
for kind in SSH_ALGORITHM_CLASSES:
|
||||
offered = list(effective.get(kind, []))
|
||||
asked = set(remove.get(kind, []))
|
||||
not_offered += [Refusal(kind, a, "not_offered") for a in sorted(asked - set(offered))]
|
||||
going = [a for a in offered if a in asked]
|
||||
if kind == "hostkey" and usable_keys is None:
|
||||
refused += [Refusal(kind, a, "host_keys_unknown") for a in going]
|
||||
going = []
|
||||
# A host key algorithm without a key is offered, but nobody gets in with it.
|
||||
usable = usable_keys if kind == "hostkey" and usable_keys is not None else set(offered)
|
||||
staying = [a for a in offered if a not in going and a in usable]
|
||||
speaks = set(client[kind]) if client and kind in client else None
|
||||
if going and speaks is not None and not speaks & set(staying):
|
||||
spared = next((a for a in going if a in speaks and a in usable), None)
|
||||
if spared:
|
||||
going.remove(spared)
|
||||
staying.append(spared)
|
||||
refused.append(Refusal(kind, spared, "client"))
|
||||
if going and not staying:
|
||||
spared = next((a for a in going if a in usable), None)
|
||||
if spared is None and not [a for a in offered if a not in going]:
|
||||
spared = going[0]
|
||||
if spared is not None:
|
||||
going.remove(spared)
|
||||
refused.append(Refusal(kind, spared, "last"))
|
||||
keep[kind] = [a for a in offered if a not in going]
|
||||
if going:
|
||||
removed[kind] = going
|
||||
return SshHardeningPlan(keep=keep, remove=removed, refused=refused, not_offered=not_offered)
|
||||
|
||||
|
||||
def render_dropin(keep: Mapping[str, Sequence[str]], kinds: Iterable[str]) -> str:
|
||||
"""The drop-in file stating the full list of each class in *kinds*."""
|
||||
lines = [
|
||||
f"{SSH_ALGORITHM_CLASSES[k][1]} {','.join(keep[k])}\n"
|
||||
for k in SSH_ALGORITHM_CLASSES
|
||||
if k in set(kinds)
|
||||
]
|
||||
return _HEADER + "".join(lines)
|
||||
|
||||
|
||||
def dropin_classes(text: str) -> list[str]:
|
||||
"""The classes an existing drop-in sets."""
|
||||
return [k for k in (_directive_key(line.strip()) for line in text.splitlines()) if k]
|
||||
|
||||
|
||||
def _is_match(line: str) -> bool:
|
||||
parts = line.split(None, 1)
|
||||
return bool(parts) and parts[0].lower() == "match"
|
||||
|
||||
|
||||
def has_dropin_include(text: str) -> bool:
|
||||
"""Whether sshd_config's global section includes :data:`DROPIN_DIR`."""
|
||||
for raw in text.splitlines():
|
||||
line = raw.strip()
|
||||
if _is_match(line):
|
||||
return False
|
||||
parts = line.split()
|
||||
if len(parts) >= 2 and parts[0].lower() == "include":
|
||||
if any(p.startswith(DROPIN_DIR + "/") for p in parts[1:]):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _split_our_block(text: str) -> tuple[list[str], str]:
|
||||
"""(the classes an earlier rewrite put above the file, the file without them)."""
|
||||
if not text.startswith(_HEADER):
|
||||
return [], text
|
||||
lines = text[len(_HEADER) :].splitlines(keepends=True)
|
||||
ours = 0
|
||||
while ours < len(lines) and _directive_key(lines[ours].strip()):
|
||||
ours += 1
|
||||
return dropin_classes("".join(lines[:ours])), "".join(lines[ours:])
|
||||
|
||||
|
||||
def render_main_config(text: str, keep: Mapping[str, Sequence[str]], kinds: Iterable[str]) -> str:
|
||||
"""sshd_config with the lists of *kinds* first and their old global lines disabled.
|
||||
|
||||
First because sshd keeps the first value it reads. A ``Match`` block is left
|
||||
alone -- it is not the global setting. A class an earlier rewrite set stays
|
||||
set, and rewriting a file this already rewrote gives the same file.
|
||||
"""
|
||||
earlier, body = _split_our_block(text)
|
||||
wanted = set(kinds) | set(earlier)
|
||||
out: list[str] = []
|
||||
in_match = False
|
||||
for raw in body.splitlines():
|
||||
if _is_match(raw.strip()):
|
||||
in_match = True
|
||||
if not in_match and _directive_key(raw.strip()) in wanted:
|
||||
out.append(DISABLED_PREFIX + raw)
|
||||
else:
|
||||
out.append(raw)
|
||||
return render_dropin(keep, wanted) + "\n".join(out) + ("\n" if body.endswith("\n") else "")
|
||||
|
||||
|
||||
def parse_penalty_exemptions(text: str) -> list[str] | None:
|
||||
"""The ``persourcepenaltyexemptlist`` from ``sshd -T`` output; None without penalties.
|
||||
|
||||
Per-source penalties came with OpenSSH 9.8. An older sshd prints neither
|
||||
line and refuses ``PerSourcePenaltyExemptList`` outright -- it does not
|
||||
start with it in its configuration -- so None means: write nothing.
|
||||
"""
|
||||
penalties = False
|
||||
exempt: list[str] = []
|
||||
for raw in text.splitlines():
|
||||
parts = raw.strip().split(None, 1)
|
||||
if not parts:
|
||||
continue
|
||||
word = parts[0].lower()
|
||||
if word == "persourcepenalties":
|
||||
penalties = True
|
||||
elif word == "persourcepenaltyexemptlist" and len(parts) == 2:
|
||||
value = parts[1].strip()
|
||||
exempt = [] if value.lower() == "none" else [a for a in value.split(",") if a]
|
||||
return exempt if penalties else None
|
||||
|
||||
|
||||
def client_address(ssh_connection: str) -> str | None:
|
||||
"""The client's address in ``$SSH_CONNECTION`` (``client port server port``), or None."""
|
||||
parts = ssh_connection.split()
|
||||
if len(parts) != 4:
|
||||
return None
|
||||
try:
|
||||
return str(ipaddress.ip_address(parts[0]))
|
||||
except ValueError:
|
||||
return None
|
||||
|
||||
|
||||
def is_exempt(address: str, exempt: Iterable[str]) -> bool:
|
||||
"""Whether *address* is one of *exempt*'s addresses or inside one of its networks."""
|
||||
ip = ipaddress.ip_address(address)
|
||||
for entry in exempt:
|
||||
try:
|
||||
if ip in ipaddress.ip_network(entry, strict=False):
|
||||
return True
|
||||
except ValueError:
|
||||
continue
|
||||
return False
|
||||
|
||||
|
||||
def render_penalty_dropin(exempt: Sequence[str]) -> str:
|
||||
"""The drop-in stating the full exempt list."""
|
||||
return _PENALTY_HEADER + f"PerSourcePenaltyExemptList {','.join(exempt)}\n"
|
||||
|
||||
|
||||
@dataclass
|
||||
class SshHardeningResult:
|
||||
"""What :meth:`OpenSshServerMixin.harden_ssh_server` did.
|
||||
|
||||
*applied* is True only when the new lists are live and a new session got
|
||||
in. *rolled_back* says the host was changed and then put back. *backup* is
|
||||
where sshd_config was copied to, when it was the file changed.
|
||||
"""
|
||||
|
||||
plan: SshHardeningPlan
|
||||
applied: bool = False
|
||||
rolled_back: bool = False
|
||||
path: str | None = None
|
||||
backup: str | None = None
|
||||
message: str = ""
|
||||
|
||||
|
||||
@dataclass
|
||||
class SshPenaltyExemption:
|
||||
"""What :meth:`OpenSshServerMixin.exempt_from_ssh_penalties` did.
|
||||
|
||||
*address* is the one the host sees the managing client at. *applied* is
|
||||
True only when sshd uses the exemption and a new session got in;
|
||||
*rolled_back* says the host was changed and then put back.
|
||||
"""
|
||||
|
||||
address: str | None = None
|
||||
applied: bool = False
|
||||
rolled_back: bool = False
|
||||
path: str | None = None
|
||||
message: str = ""
|
||||
|
||||
|
||||
#: Why an algorithm stays, by :class:`Refusal` reason.
|
||||
_WHYS = {
|
||||
"last": "it is the last one",
|
||||
"client": "netOrk could no longer connect",
|
||||
"host_keys_unknown": "the host keys could not be read",
|
||||
}
|
||||
|
||||
|
||||
def _unchanged(
|
||||
result: SshPenaltyExemption, message: str, say: Callable[[str], None]
|
||||
) -> SshPenaltyExemption:
|
||||
result.message = message
|
||||
say(message)
|
||||
return result
|
||||
|
||||
|
||||
class OpenSshServerMixin:
|
||||
"""Adds :meth:`harden_ssh_server` to a driver that can run commands as root on an OpenSSH host.
|
||||
|
||||
Mixed in by the drivers whose host runs a stock OpenSSH, not by a base class,
|
||||
so ``hasattr(driver, "harden_ssh_server")`` stays a truthful answer. A driver
|
||||
whose platform writes sshd_config itself (an appliance that regenerates it
|
||||
from its own database) must not inherit this, or must opt out with
|
||||
``harden_ssh_server = None``: its file would be overwritten on the next
|
||||
apply. Where that generated file still includes the drop-in directory, the
|
||||
driver sets :attr:`SSHD_CONFIG_GENERATED` instead and keeps the hardening.
|
||||
"""
|
||||
|
||||
#: True where the platform writes sshd_config from its own database
|
||||
#: (OpenMediaVault): only the drop-in directory is written, and a
|
||||
#: sshd_config that does not include it is left alone, since the platform
|
||||
#: would write over a change to it on its next apply.
|
||||
SSHD_CONFIG_GENERATED: bool = False
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def run_command(
|
||||
self,
|
||||
command: str,
|
||||
*,
|
||||
privileged: bool = False,
|
||||
timeout: float = 60,
|
||||
stdin: bytes | None = None,
|
||||
) -> CommandResult: ...
|
||||
|
||||
# ── the host ────────────────────────────────────────────────────────
|
||||
|
||||
def _ssh_run(self, command: str, stdin: bytes | None = None) -> CommandResult:
|
||||
return self.run_command(command, privileged=True, timeout=30, stdin=stdin)
|
||||
|
||||
def _ssh_read(self, path: str) -> str | None:
|
||||
result = self._ssh_run(f"cat -- {quote(path)}")
|
||||
return result.stdout if result.exit_code == 0 else None
|
||||
|
||||
def _ssh_write(self, path: str, content: str, mode_of: str | None = None) -> None:
|
||||
"""Replace *path* with *content*, as root, without root reading stdin.
|
||||
|
||||
A driver gains root with ``sudo -S`` and puts the password in front of
|
||||
stdin. A sudo that does not prompt -- NOPASSWD, a cached timestamp --
|
||||
leaves it there, and a ``cat`` under sudo writes it into the file. So the
|
||||
content goes, without privilege, to a file only the login user can read;
|
||||
root then installs that file, owned by root, and removes it.
|
||||
|
||||
The mode is that of *mode_of* (default: *path* itself), so a 0600
|
||||
sshd_config stays 0600 and so does its copy; 644 for a new file.
|
||||
"""
|
||||
staged = self.run_command(
|
||||
'umask 077 && t=$(mktemp /tmp/netork-ssh.XXXXXX) && cat > "$t" && printf %s "$t"',
|
||||
privileged=False,
|
||||
timeout=30,
|
||||
stdin=content.encode(),
|
||||
)
|
||||
tmp = staged.stdout.strip()
|
||||
# A path on the device, made by mktemp under umask 077: not a local temp file.
|
||||
if staged.exit_code != 0 or not tmp.startswith("/tmp/netork-ssh."): # nosec B108
|
||||
raise RuntimeError(
|
||||
f"could not stage {path}: {(staged.stderr or staged.stdout).strip()}"
|
||||
)
|
||||
part = path + ".netork-tmp"
|
||||
result = self._ssh_run(
|
||||
# GNU stat, then BSD's; 644 for a file that does not exist yet.
|
||||
f'r={quote(mode_of or path)}; m=$(stat -c %a -- "$r" 2>/dev/null'
|
||||
f' || stat -f %Lp -- "$r" 2>/dev/null) || m=644;'
|
||||
f' install -m "$m" -- {quote(tmp)} {quote(part)}'
|
||||
f" && mv -f -- {quote(part)} {quote(path)};"
|
||||
f" s=$?; rm -f -- {quote(tmp)} {quote(part)}; exit $s"
|
||||
)
|
||||
if result.exit_code != 0:
|
||||
raise RuntimeError(
|
||||
f"could not write {path}: {(result.stderr or result.stdout).strip()}"
|
||||
)
|
||||
|
||||
def _ssh_remove(self, path: str) -> None:
|
||||
self._ssh_run(f"rm -f -- {quote(path)}")
|
||||
|
||||
def _reload_ssh_server(self) -> None:
|
||||
"""Make the running sshd read its configuration again; sessions stay up."""
|
||||
result = self._ssh_run(_RELOAD)
|
||||
if result.exit_code != 0:
|
||||
raise RuntimeError(f"could not reload sshd: {(result.stderr or result.stdout).strip()}")
|
||||
|
||||
def get_ssh_server_algorithms(self) -> dict[str, list[str]]:
|
||||
"""The algorithm lists sshd uses now (``sshd -T``), per class.
|
||||
|
||||
:raises RuntimeError: if sshd -T fails -- no OpenSSH, or no root.
|
||||
"""
|
||||
return parse_sshd_effective(self._sshd_effective_text())
|
||||
|
||||
def _sshd_effective_text(self) -> str:
|
||||
result = self._ssh_run("PATH=$PATH:/usr/sbin:/sbin sshd -T")
|
||||
if result.exit_code != 0:
|
||||
raise RuntimeError(f"sshd -T failed: {(result.stderr or result.stdout).strip()}")
|
||||
return result.stdout
|
||||
|
||||
def _ssh_host_key_algorithms(self, sshd_t: str) -> set[str] | None:
|
||||
"""The host key algorithms the host has a key for; None when that cannot be read.
|
||||
|
||||
A ``HostKey`` whose file does not exist is skipped, as sshd skips it.
|
||||
"""
|
||||
paths = parse_sshd_hostkeys(sshd_t)
|
||||
if not paths:
|
||||
return None
|
||||
result = self._ssh_run(
|
||||
f"for k in {' '.join(quote(p) for p in paths)}; do"
|
||||
' [ -e "$k" ] || continue; ssh-keygen -l -f "$k" || exit 1; done'
|
||||
)
|
||||
if result.exit_code != 0:
|
||||
return None
|
||||
return host_key_algorithms(result.stdout)
|
||||
|
||||
def _sshd_accepts(self) -> str | None:
|
||||
"""None when ``sshd -t`` accepts the configuration, else what it said."""
|
||||
result = self._ssh_run("PATH=$PATH:/usr/sbin:/sbin sshd -t")
|
||||
return None if result.exit_code == 0 else (result.stderr or result.stdout).strip()
|
||||
|
||||
# ── the change ──────────────────────────────────────────────────────
|
||||
|
||||
def harden_ssh_server(
|
||||
self,
|
||||
remove: Mapping[str, Iterable[str]],
|
||||
*,
|
||||
client: Mapping[str, Iterable[str]] | None = None,
|
||||
verify: Callable[[], bool] | None = None,
|
||||
log: Callable[[str], None] | None = None,
|
||||
) -> SshHardeningResult:
|
||||
"""Stop the server offering the algorithms in *remove* (class -> names).
|
||||
|
||||
*client* guards the managing client's algorithms (see
|
||||
:func:`plan_ssh_hardening`); *verify* opens a new session after the
|
||||
reload and returns whether it got in; *log* receives a line per step.
|
||||
|
||||
Never raises after the host was touched: a failure from there on puts
|
||||
the previous state back and says so in the result.
|
||||
|
||||
:raises RuntimeError: if the current state cannot be read.
|
||||
"""
|
||||
say = log or (lambda _line: None)
|
||||
plan = self._plan_hardening(remove, client, say)
|
||||
kinds = plan.changed_classes
|
||||
result = SshHardeningResult(plan=plan)
|
||||
if not kinds:
|
||||
result.message = "Nothing to change: the server offers none of the selected algorithms."
|
||||
say(result.message)
|
||||
return result
|
||||
for kind in kinds:
|
||||
say(f"Removing {', '.join(plan.remove[kind])} from {SSH_ALGORITHM_CLASSES[kind][1]}")
|
||||
|
||||
main = self._ssh_read(MAIN_CONFIG)
|
||||
if main is None:
|
||||
raise RuntimeError(f"cannot read {MAIN_CONFIG}")
|
||||
if self.SSHD_CONFIG_GENERATED and not has_dropin_include(main):
|
||||
result.message = (
|
||||
f"Not changed: {MAIN_CONFIG} does not include {DROPIN_DIR}, and this"
|
||||
" platform writes it from its own settings."
|
||||
)
|
||||
say(result.message)
|
||||
return result
|
||||
restore = self._write_change(plan, main, result, say)
|
||||
return self._apply_or_restore(
|
||||
result,
|
||||
restore,
|
||||
verify,
|
||||
say,
|
||||
ignored=lambda: self._algorithms_ignored(plan, result.path),
|
||||
done="The server no longer offers the selected algorithms.",
|
||||
)
|
||||
|
||||
def _plan_hardening(
|
||||
self,
|
||||
remove: Mapping[str, Iterable[str]],
|
||||
client: Mapping[str, Iterable[str]] | None,
|
||||
say: Callable[[str], None],
|
||||
) -> SshHardeningPlan:
|
||||
"""Read what sshd offers and which host keys it has; plan, and say what stays."""
|
||||
say("Reading the algorithms sshd offers (sshd -T)")
|
||||
sshd_t = self._sshd_effective_text()
|
||||
host_keys: set[str] | None = None
|
||||
if remove.get("hostkey"):
|
||||
say("Reading which host keys the server has (ssh-keygen -l)")
|
||||
host_keys = self._ssh_host_key_algorithms(sshd_t)
|
||||
plan = plan_ssh_hardening(
|
||||
parse_sshd_effective(sshd_t), remove, client=client, host_keys=host_keys
|
||||
)
|
||||
for r in plan.refused:
|
||||
last_key = r.reason == "last" and r.kind == "hostkey"
|
||||
why = "it is the last one the host has a key for" if last_key else _WHYS[r.reason]
|
||||
say(f"Keeping {r.algorithm}: {why}")
|
||||
return plan
|
||||
|
||||
def _algorithms_ignored(self, plan: SshHardeningPlan, path: str | None) -> str | None:
|
||||
"""Why sshd does not use the new lists; None when it does."""
|
||||
now = self.get_ssh_server_algorithms()
|
||||
names = [
|
||||
SSH_ALGORITHM_CLASSES[k][1] for k in plan.changed_classes if now[k] != plan.keep[k]
|
||||
]
|
||||
if not names:
|
||||
return None
|
||||
return (
|
||||
f"sshd does not use the new {', '.join(names)}: a setting read before"
|
||||
f" {path} decides them"
|
||||
)
|
||||
|
||||
def _write_change(
|
||||
self,
|
||||
plan: SshHardeningPlan,
|
||||
main: str,
|
||||
result: SshHardeningResult,
|
||||
say: Callable[[str], None],
|
||||
) -> Callable[[], None]:
|
||||
"""Write the new lists; return what puts the previous state back."""
|
||||
if has_dropin_include(main):
|
||||
previous = self._ssh_read(DROPIN_PATH)
|
||||
kinds = set(plan.changed_classes) | set(dropin_classes(previous or ""))
|
||||
self._ssh_run(f"mkdir -p -- {quote(DROPIN_DIR)}")
|
||||
self._ssh_write(DROPIN_PATH, render_dropin(plan.keep, kinds))
|
||||
result.path = DROPIN_PATH
|
||||
say(f"Wrote {DROPIN_PATH}")
|
||||
|
||||
def restore() -> None:
|
||||
if previous is None:
|
||||
self._ssh_remove(DROPIN_PATH)
|
||||
else:
|
||||
self._ssh_write(DROPIN_PATH, previous)
|
||||
|
||||
return restore
|
||||
backup = f"{MAIN_CONFIG}.netork-{time.strftime('%Y%m%d-%H%M%S')}.bak"
|
||||
self._ssh_write(backup, main, mode_of=MAIN_CONFIG)
|
||||
self._ssh_write(MAIN_CONFIG, render_main_config(main, plan.keep, plan.changed_classes))
|
||||
result.path, result.backup = MAIN_CONFIG, backup
|
||||
say(f"{MAIN_CONFIG} has no Include of {DROPIN_DIR}; rewrote it (copy at {backup})")
|
||||
|
||||
def restore_main() -> None:
|
||||
self._ssh_write(MAIN_CONFIG, main)
|
||||
|
||||
return restore_main
|
||||
|
||||
def _apply_or_restore[R: (SshHardeningResult, SshPenaltyExemption)](
|
||||
self,
|
||||
result: R,
|
||||
restore: Callable[[], None],
|
||||
verify: Callable[[], bool] | None,
|
||||
say: Callable[[str], None],
|
||||
*,
|
||||
ignored: Callable[[], str | None],
|
||||
done: str,
|
||||
) -> R:
|
||||
"""Check what was written, reload, verify; put the old state back on any failure.
|
||||
|
||||
*ignored* reads ``sshd -T`` after ``sshd -t`` and says why sshd does not
|
||||
use the change, or None when it does; *done* is the message of success.
|
||||
"""
|
||||
|
||||
def back(message: str, reload: bool) -> R:
|
||||
return self._put_back(result, restore, say, message, reload=reload)
|
||||
|
||||
try:
|
||||
say("Checking the configuration (sshd -t)")
|
||||
complaint = self._sshd_accepts()
|
||||
if complaint:
|
||||
return back(f"sshd rejected the new configuration: {complaint}", reload=False)
|
||||
why = ignored()
|
||||
if why:
|
||||
return back(why, reload=False)
|
||||
say("Reloading sshd")
|
||||
self._reload_ssh_server()
|
||||
if verify is not None:
|
||||
say("Opening a new SSH session to check netOrk still gets in")
|
||||
if not verify():
|
||||
return back("A new SSH session could not log in after the reload", reload=True)
|
||||
except Exception as exc: # noqa: BLE001 - whatever went wrong, the host goes back
|
||||
return back(f"The change failed: {exc}", reload=True)
|
||||
result.applied = True
|
||||
result.message = done
|
||||
say(result.message)
|
||||
return result
|
||||
|
||||
def _put_back[R: (SshHardeningResult, SshPenaltyExemption)](
|
||||
self,
|
||||
result: R,
|
||||
restore: Callable[[], None],
|
||||
say: Callable[[str], None],
|
||||
message: str,
|
||||
*,
|
||||
reload: bool,
|
||||
) -> R:
|
||||
"""Say *message*, put the previous state back, and say whether that worked."""
|
||||
say(message)
|
||||
say("Putting the previous configuration back")
|
||||
try:
|
||||
restore()
|
||||
if reload:
|
||||
self._reload_ssh_server()
|
||||
except Exception as exc: # noqa: BLE001 - report it, the caller must look
|
||||
result.message = f"{message}; putting the previous configuration back failed: {exc}"
|
||||
say(result.message)
|
||||
return result
|
||||
result.rolled_back = True
|
||||
result.message = message
|
||||
return result
|
||||
|
||||
# ── the penalty exemption ───────────────────────────────────────────
|
||||
|
||||
def exempt_from_ssh_penalties(
|
||||
self,
|
||||
*,
|
||||
verify: Callable[[], bool] | None = None,
|
||||
log: Callable[[str], None] | None = None,
|
||||
) -> SshPenaltyExemption:
|
||||
"""Exempt the address this session comes from from sshd's per-source penalties.
|
||||
|
||||
OpenSSH 9.8+ penalises a source whose connections do not authenticate --
|
||||
a scanner's probes -- and then drops every connection from it, the
|
||||
managing client's included (NetOrk/netork#1800). The address is the
|
||||
one the host sees (``$SSH_CONNECTION``), so NAT and relays do not
|
||||
matter. The host's own exemptions stay: the new list is the one sshd
|
||||
uses now plus this address.
|
||||
|
||||
Only into the drop-in directory: a sshd_config that does not include it
|
||||
is left alone, and the result says so. An sshd before 9.8 is not
|
||||
touched -- it would not start with the directive. Checked like
|
||||
:meth:`harden_ssh_server`: ``sshd -t``, ``sshd -T`` shows the address,
|
||||
reload, then *verify*; on any failure the previous state goes back.
|
||||
|
||||
:raises RuntimeError: if the session names no client address, or
|
||||
``sshd -T`` or sshd_config cannot be read.
|
||||
"""
|
||||
say = log or (lambda _line: None)
|
||||
address = self._session_client_address()
|
||||
result = SshPenaltyExemption(address=address)
|
||||
say(f"netOrk connects from {address}")
|
||||
say("Reading sshd's per-source penalties (sshd -T)")
|
||||
exempt = parse_penalty_exemptions(self._sshd_effective_text())
|
||||
reason = self._penalty_exemption_unchanged(address, exempt)
|
||||
if reason is not None or exempt is None:
|
||||
return _unchanged(result, reason or "", say)
|
||||
|
||||
restore = self._write_penalty_dropin([*exempt, address])
|
||||
result.path = PENALTY_DROPIN_PATH
|
||||
say(f"Wrote {PENALTY_DROPIN_PATH}")
|
||||
return self._apply_or_restore(
|
||||
result,
|
||||
restore,
|
||||
verify,
|
||||
say,
|
||||
ignored=lambda: self._exemption_ignored(address),
|
||||
done=f"{address} is exempt from sshd's per-source penalties.",
|
||||
)
|
||||
|
||||
def _session_client_address(self) -> str:
|
||||
"""The address the host sees this session come from (``$SSH_CONNECTION``)."""
|
||||
session = self.run_command('printf %s "$SSH_CONNECTION"', privileged=False, timeout=30)
|
||||
address = client_address(session.stdout)
|
||||
if address is None:
|
||||
raise RuntimeError("the session names no client address ($SSH_CONNECTION)")
|
||||
return address
|
||||
|
||||
def _penalty_exemption_unchanged(self, address: str, exempt: list[str] | None) -> str | None:
|
||||
"""Why the exemption is left as it is; None when it is to be written."""
|
||||
if exempt is None:
|
||||
return "Nothing to change: this sshd has no per-source penalties (before OpenSSH 9.8)."
|
||||
if is_exempt(address, exempt):
|
||||
return f"Nothing to change: {address} is already exempt."
|
||||
main = self._ssh_read(MAIN_CONFIG)
|
||||
if main is None:
|
||||
raise RuntimeError(f"cannot read {MAIN_CONFIG}")
|
||||
if not has_dropin_include(main):
|
||||
return (
|
||||
f"Not changed: {MAIN_CONFIG} does not include {DROPIN_DIR},"
|
||||
" where the exemption would go."
|
||||
)
|
||||
return None
|
||||
|
||||
def _exemption_ignored(self, address: str) -> str | None:
|
||||
"""Why sshd does not use the new exemption; None when it does."""
|
||||
if is_exempt(address, parse_penalty_exemptions(self._sshd_effective_text()) or []):
|
||||
return None
|
||||
return (
|
||||
"sshd does not use the new PerSourcePenaltyExemptList: a setting read before"
|
||||
f" {PENALTY_DROPIN_PATH} decides it"
|
||||
)
|
||||
|
||||
def _write_penalty_dropin(self, exempt: Sequence[str]) -> Callable[[], None]:
|
||||
"""Write the exemption list; return what puts the previous drop-in back."""
|
||||
previous = self._ssh_read(PENALTY_DROPIN_PATH)
|
||||
self._ssh_run(f"mkdir -p -- {quote(DROPIN_DIR)}")
|
||||
self._ssh_write(PENALTY_DROPIN_PATH, render_penalty_dropin(exempt))
|
||||
|
||||
def restore() -> None:
|
||||
if previous is None:
|
||||
self._ssh_remove(PENALTY_DROPIN_PATH)
|
||||
else:
|
||||
self._ssh_write(PENALTY_DROPIN_PATH, previous)
|
||||
|
||||
return restore
|
||||
@@ -10,9 +10,9 @@ Usage::
|
||||
...
|
||||
"""
|
||||
|
||||
from typing import Any, Dict, List, TYPE_CHECKING
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from napalm_device_types.base import DeviceTypeDriver
|
||||
from napalm_device_types.packages import PackageManagementMixin
|
||||
from napalm_device_types.models import (
|
||||
DiskPoolDict,
|
||||
LogicalVolumeDict,
|
||||
@@ -23,6 +23,7 @@ from napalm_device_types.models import (
|
||||
StorageServiceDict,
|
||||
VolumeSnapshotDict,
|
||||
)
|
||||
from napalm_device_types.packages import PackageManagementMixin
|
||||
|
||||
|
||||
class StorageDriver(PackageManagementMixin, DeviceTypeDriver):
|
||||
@@ -47,7 +48,7 @@ class StorageDriver(PackageManagementMixin, DeviceTypeDriver):
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
||||
def get_disks(self) -> List[PhysicalDiskDict]:
|
||||
def get_disks(self) -> list[PhysicalDiskDict]:
|
||||
"""
|
||||
Returns all physical drives detected by the storage device.
|
||||
|
||||
@@ -62,7 +63,8 @@ class StorageDriver(PackageManagementMixin, DeviceTypeDriver):
|
||||
* rpm (int) - rotational speed; ``0`` for SSD/NVMe
|
||||
* temperature (int) - current temperature in Celsius; ``-1`` if unavailable
|
||||
* health (string) - ``"healthy"``, ``"warning"``, ``"failed"``, or ``"unknown"``
|
||||
* pool (string) - name of the pool this disk belongs to; empty string if unassigned/spare
|
||||
* pool (string) - name of the pool this disk belongs to;
|
||||
empty string if unassigned/spare
|
||||
|
||||
Example::
|
||||
|
||||
@@ -95,7 +97,7 @@ class StorageDriver(PackageManagementMixin, DeviceTypeDriver):
|
||||
"""
|
||||
...
|
||||
|
||||
def get_disk_pools(self) -> Dict[str, DiskPoolDict]:
|
||||
def get_disk_pools(self) -> dict[str, DiskPoolDict]:
|
||||
"""
|
||||
Returns the disk pools (RAID arrays, ZFS pools, LVM volume groups, etc.)
|
||||
configured on the device.
|
||||
@@ -106,12 +108,14 @@ class StorageDriver(PackageManagementMixin, DeviceTypeDriver):
|
||||
* type (string) - ``"zfs"``, ``"lvm"``, ``"md"``, ``"hardware-raid"``, ``"btrfs"``
|
||||
* level (string) - RAID/redundancy level: ``"mirror"``, ``"raidz1"``, ``"raidz2"``,
|
||||
``"raidz3"``, ``"stripe"``, ``"raid0"`` … ``"raid60"``, ``"single"``
|
||||
* status (string) - ``"online"``, ``"degraded"``, ``"faulted"``, ``"offline"``, ``"unknown"``
|
||||
* status (string) - ``"online"``, ``"degraded"``, ``"faulted"``,
|
||||
``"offline"``, ``"unknown"``
|
||||
* total (int) - usable capacity in bytes
|
||||
* used (int) - used space in bytes
|
||||
* available (int) - free space in bytes
|
||||
* disks (list of strings) - slot identifiers of member drives
|
||||
* auto_expand (bool) - whether the pool grows automatically when disks are replaced with larger ones
|
||||
* auto_expand (bool) - whether the pool grows automatically when disks
|
||||
are replaced with larger ones
|
||||
* dedup (bool) - whether deduplication is enabled
|
||||
* compression (string) - pool-level compression algorithm: ``"off"``, ``"lz4"``,
|
||||
``"gzip"``, ``"zstd"``, etc.
|
||||
@@ -140,7 +144,7 @@ class StorageDriver(PackageManagementMixin, DeviceTypeDriver):
|
||||
# Logical volumes / datasets
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def get_volumes(self) -> Dict[str, LogicalVolumeDict]:
|
||||
def get_volumes(self) -> dict[str, LogicalVolumeDict]:
|
||||
"""
|
||||
Returns all logical volumes, ZFS datasets, or LUNs on the device.
|
||||
|
||||
@@ -154,7 +158,8 @@ class StorageDriver(PackageManagementMixin, DeviceTypeDriver):
|
||||
* used (int) - space currently used in bytes
|
||||
* available (int) - space available in bytes
|
||||
* mountpoint (string) - local mount path; empty string for block volumes / LUNs
|
||||
* compression (string) - active compression algorithm (``"off"``, ``"lz4"``, ``"zstd"``, etc.)
|
||||
* compression (string) - active compression algorithm
|
||||
(``"off"``, ``"lz4"``, ``"zstd"``, etc.)
|
||||
* dedup (bool) - whether deduplication is active on this volume
|
||||
* readonly (bool) - whether the volume is mounted read-only
|
||||
* snapshots (int) - number of snapshots currently held
|
||||
@@ -196,7 +201,7 @@ class StorageDriver(PackageManagementMixin, DeviceTypeDriver):
|
||||
# Shares
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def get_shares(self) -> Dict[str, NASShareDict]:
|
||||
def get_shares(self) -> dict[str, NASShareDict]:
|
||||
"""
|
||||
Returns all network shares and iSCSI targets exported by the device.
|
||||
|
||||
@@ -244,7 +249,7 @@ class StorageDriver(PackageManagementMixin, DeviceTypeDriver):
|
||||
# Snapshots
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def get_volume_snapshots(self, volume: str = "") -> List[VolumeSnapshotDict]:
|
||||
def get_volume_snapshots(self, volume: str = "") -> list[VolumeSnapshotDict]:
|
||||
"""
|
||||
Returns volume / dataset snapshots.
|
||||
|
||||
@@ -253,7 +258,8 @@ class StorageDriver(PackageManagementMixin, DeviceTypeDriver):
|
||||
|
||||
Each entry contains:
|
||||
|
||||
* name (string) - snapshot name (e.g. ``"auto-2026-05-11"`` or full ``"tank/media@auto-2026-05-11"``)
|
||||
* name (string) - snapshot name (e.g. ``"auto-2026-05-11"``
|
||||
or full ``"tank/media@auto-2026-05-11"``)
|
||||
* volume (string) - parent volume / dataset path
|
||||
* created (float) - creation timestamp (Unix epoch)
|
||||
* size (int) - bytes of unique data referenced only by this snapshot
|
||||
@@ -339,7 +345,7 @@ class StorageDriver(PackageManagementMixin, DeviceTypeDriver):
|
||||
# Quotas
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def get_quotas(self) -> List[StorageQuotaDict]:
|
||||
def get_quotas(self) -> list[StorageQuotaDict]:
|
||||
"""
|
||||
Returns all filesystem quotas configured on the device.
|
||||
|
||||
@@ -380,7 +386,7 @@ class StorageDriver(PackageManagementMixin, DeviceTypeDriver):
|
||||
# Services
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def get_storage_services(self) -> Dict[str, StorageServiceDict]:
|
||||
def get_storage_services(self) -> dict[str, StorageServiceDict]:
|
||||
"""
|
||||
Returns the file-sharing and access services available on the device.
|
||||
|
||||
@@ -443,7 +449,7 @@ class StorageDriver(PackageManagementMixin, DeviceTypeDriver):
|
||||
# Replication
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def get_replication_jobs(self) -> List[ReplicationJobDict]:
|
||||
def get_replication_jobs(self) -> list[ReplicationJobDict]:
|
||||
"""
|
||||
Returns all replication / sync jobs configured on the device.
|
||||
|
||||
@@ -493,8 +499,3 @@ class StorageDriver(PackageManagementMixin, DeviceTypeDriver):
|
||||
# ------------------------------------------------------------------
|
||||
# Package management (plugins / extensions)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,312 @@
|
||||
"""The sudo password an earlier driver left in shell histories, and taking it back.
|
||||
|
||||
A driver that typed ``echo '<password>' | sudo -S -p "" <command>`` into an
|
||||
interactive shell left that line, password and all, in the login user's shell
|
||||
history, once per command -- and where only the part before the first ``&&``
|
||||
ran as root, it left files root should own to the login user. A driver that has
|
||||
stopped doing so still leaves both behind on every host it ever reached.
|
||||
|
||||
:class:`SudoHistoryMixin` reads the first as a fact and takes both back. The
|
||||
commands are the same on every POSIX host, so they live here once; a driver
|
||||
mixes it in, supplies ``run_command`` and says what it once left behind:
|
||||
:attr:`~SudoHistoryMixin.SUDO_HISTORY_MARKER` and
|
||||
:attr:`~SudoHistoryMixin.SUDO_ROOT_FILES`.
|
||||
|
||||
**The password never travels.** Lines are counted with ``grep -c`` and deleted
|
||||
with ``sed``, both matching the marker only, as the login user who owns the
|
||||
history. Nothing of a line's content is read back, and nothing goes to stdin.
|
||||
|
||||
**What is deleted is gone from the disk too, as far as a file can say.** ``sed
|
||||
-i`` with a backup suffix renames the old file to the backup and writes a new
|
||||
one; the backup -- the old file's blocks -- is overwritten with ``shred -u``,
|
||||
or removed with ``rm`` where there is no ``shred``. Backups, snapshots and an
|
||||
audit log of the host may still hold the password: only rotating it ends that.
|
||||
|
||||
**A file given back to root** must exist as a regular file, not be a symbolic
|
||||
link, and be owned by the login user. ``chown -h`` never follows a link that
|
||||
appears in between.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable
|
||||
from shlex import quote
|
||||
from typing import TYPE_CHECKING, ClassVar
|
||||
|
||||
from typing_extensions import TypedDict
|
||||
|
||||
from napalm_device_types.fixes import run_fix_steps
|
||||
from napalm_device_types.models import FixResultDict, FixStepDict, FixStepOutcomeDict
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - for type checkers only
|
||||
from napalm_device_types.channel import CommandResult
|
||||
|
||||
#: The shell history files looked at, in the login user's home.
|
||||
HISTORY_FILES = (".bash_history", ".zsh_history", ".ash_history", ".sh_history")
|
||||
|
||||
#: The suffix of the moment's backup ``sed -i`` makes; it carries no application's name.
|
||||
HISTORY_BACKUP_SUFFIX = ".napalm-bak"
|
||||
|
||||
_COUNT = 'LC_ALL=C grep -c -F -e "$m" -- "$p"'
|
||||
|
||||
|
||||
class HistorySudoLinesDict(TypedDict):
|
||||
"""A history file holding lines with the sudo password (``get_history_sudo_lines``)."""
|
||||
|
||||
#: The file, as ``~/<name>``.
|
||||
file: str
|
||||
lines: int
|
||||
|
||||
|
||||
class HistoryCleanupDict(TypedDict):
|
||||
"""What one history file held before the cleanup and holds after it."""
|
||||
|
||||
file: str
|
||||
before: int
|
||||
after: int
|
||||
|
||||
|
||||
class SudoHistoryCleanupDict(FixResultDict):
|
||||
"""The result of ``remove_history_sudo_lines``: the fix, and what it changed."""
|
||||
|
||||
#: Every history file there is, with its lines before and after.
|
||||
files: list[HistoryCleanupDict]
|
||||
#: How many lines were deleted, over all files.
|
||||
removed: int
|
||||
#: The files given back to root.
|
||||
owners_restored: list[str]
|
||||
|
||||
|
||||
def _bre(text: str) -> str:
|
||||
"""*text* as a POSIX basic regular expression matching it literally, for ``sed``."""
|
||||
return "".join("\\" + c if c in "\\.[]*^$/" else c for c in text)
|
||||
|
||||
|
||||
def _history_loop(body: str) -> str:
|
||||
names = " ".join(HISTORY_FILES)
|
||||
return f'for f in {names}; do p="$HOME/$f"; [ -f "$p" ] || continue; {body}; done'
|
||||
|
||||
|
||||
def count_script(marker: str) -> str:
|
||||
"""Print ``<count> <name>`` per history file there is; exit 2 when one cannot be read."""
|
||||
return f"m={quote(marker)}; " + _history_loop(
|
||||
f'n=$({_COUNT}); [ $? -le 1 ] || exit 2; printf "%s %s\\n" "$n" "$f"'
|
||||
)
|
||||
|
||||
|
||||
def cleanup_script(marker: str) -> str:
|
||||
"""Delete the lines holding *marker* from every history file there is.
|
||||
|
||||
Prints per file: ``done <before> <after> <name>``, ``link <count> <name>``
|
||||
(left alone), ``unreadable <name>`` or ``failed <before> <name>``.
|
||||
"""
|
||||
backup = f'"$p"{quote(HISTORY_BACKUP_SUFFIX)}'
|
||||
return f"m={quote(marker)}; r={quote(_bre(marker))}; " + _history_loop(
|
||||
f"b=$({_COUNT}); "
|
||||
'[ $? -le 1 ] || { printf "unreadable %s\\n" "$f"; continue; }; '
|
||||
'if [ -L "$p" ]; then printf "link %s %s\\n" "$b" "$f"; continue; fi; '
|
||||
'if [ "$b" -gt 0 ]; then '
|
||||
f'LC_ALL=C sed -i{quote(HISTORY_BACKUP_SUFFIX)} -e "/$r/d" "$p"; s=$?; '
|
||||
# A sed that renamed the file away and wrote no new one: the backup is the history.
|
||||
f'[ -e "$p" ] || mv -f {backup} "$p"; '
|
||||
f"[ -e {backup} ] && {{ shred -u {backup} 2>/dev/null || rm -f {backup}; }}; "
|
||||
'[ $s -eq 0 ] || { printf "failed %s %s\\n" "$b" "$f"; continue; }; '
|
||||
"fi; "
|
||||
f'a=$({_COUNT}); printf "done %s %s %s\\n" "$b" "$a" "$f"'
|
||||
)
|
||||
|
||||
|
||||
def owners_script(user: str, paths: tuple[str, ...]) -> str:
|
||||
"""As root: give each of *paths* back to root:root where *user* owns it.
|
||||
|
||||
Prints ``fixed <path>`` or ``failed <path>`` per file it changed or tried to.
|
||||
"""
|
||||
return (
|
||||
f"u={quote(user)}; for p in {' '.join(quote(p) for p in paths)}; do "
|
||||
'[ -f "$p" ] && [ ! -L "$p" ] || continue; '
|
||||
'[ -n "$(find "$p" -prune -user "$u" 2>/dev/null)" ] || continue; '
|
||||
'if chown -h 0:0 "$p"; then printf "fixed %s\\n" "$p"; '
|
||||
'else printf "failed %s\\n" "$p"; fi; done'
|
||||
)
|
||||
|
||||
|
||||
def _home(name: str) -> str:
|
||||
return f"~/{name}"
|
||||
|
||||
|
||||
def _lines(n: int) -> str:
|
||||
return "1 line" if n == 1 else f"{n} lines"
|
||||
|
||||
|
||||
def _done(rest: list[str]) -> tuple[bool, str, HistoryCleanupDict | None]:
|
||||
before, after, name = int(rest[0]), int(rest[1]), _home(rest[2])
|
||||
removed = before - after
|
||||
text = (
|
||||
f"{name}: {_lines(removed)} removed ({before} before, {after} after)"
|
||||
if before
|
||||
else f"{name}: no such line"
|
||||
)
|
||||
return after == 0, text, {"file": name, "before": before, "after": after}
|
||||
|
||||
|
||||
def _link(rest: list[str]) -> tuple[bool, str, HistoryCleanupDict | None]:
|
||||
count = int(rest[0])
|
||||
return count == 0, f"{_home(rest[1])}: a symbolic link, left alone ({_lines(count)})", None
|
||||
|
||||
|
||||
def _unreadable(rest: list[str]) -> tuple[bool, str, HistoryCleanupDict | None]:
|
||||
return False, f"{_home(rest[0])}: cannot be read", None
|
||||
|
||||
|
||||
def _failed(rest: list[str]) -> tuple[bool, str, HistoryCleanupDict | None]:
|
||||
kept = _lines(int(rest[0]))
|
||||
return False, f"{_home(rest[1])}: could not be rewritten ({kept} kept)", None
|
||||
|
||||
|
||||
#: Per kind of line :func:`cleanup_script` prints: how many words follow, and what it says.
|
||||
_CLEANUP_LINES: dict[
|
||||
str, tuple[int, Callable[[list[str]], tuple[bool, str, HistoryCleanupDict | None]]]
|
||||
] = {
|
||||
"done": (3, _done),
|
||||
"link": (2, _link),
|
||||
"unreadable": (1, _unreadable),
|
||||
"failed": (2, _failed),
|
||||
}
|
||||
|
||||
|
||||
class SudoHistoryMixin:
|
||||
"""Reads and removes the sudo password an earlier driver left in shell histories.
|
||||
|
||||
Mixed in by the drivers that once typed it into a shell, not by a base
|
||||
class, so ``callable(getattr(driver, "get_history_sudo_lines", None))``
|
||||
stays a truthful answer.
|
||||
"""
|
||||
|
||||
#: What every line carries that typed the sudo password into the shell, as a
|
||||
#: fixed string -- ``"| sudo -S -p "`` for ``echo '<password>' | sudo -S -p ""
|
||||
#: <command>``. None for a driver that never did: nothing is looked for.
|
||||
SUDO_HISTORY_MARKER: ClassVar[str | None] = None
|
||||
|
||||
#: Files an earlier version wrote as the login user where root should have.
|
||||
SUDO_ROOT_FILES: ClassVar[tuple[str, ...]] = ()
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def run_command(
|
||||
self,
|
||||
command: str,
|
||||
*,
|
||||
privileged: bool = False,
|
||||
timeout: float = 60,
|
||||
stdin: bytes | None = None,
|
||||
) -> CommandResult: ...
|
||||
|
||||
def get_history_sudo_lines(self) -> list[HistorySudoLinesDict]:
|
||||
"""The login user's history files that hold lines with the sudo password, and how many.
|
||||
|
||||
A fact: counted with ``grep -c`` as the login user; no line is read.
|
||||
An empty list when no file holds one, or the driver never typed it.
|
||||
|
||||
:raises RuntimeError: when a history file cannot be read.
|
||||
"""
|
||||
marker = self.SUDO_HISTORY_MARKER
|
||||
if not marker:
|
||||
return []
|
||||
result = self.run_command(count_script(marker), timeout=30)
|
||||
if result.exit_code != 0:
|
||||
raise RuntimeError(
|
||||
f"cannot count the shell history: {(result.stderr or result.stdout).strip()}"
|
||||
)
|
||||
found: list[HistorySudoLinesDict] = []
|
||||
for line in result.stdout.splitlines():
|
||||
count, _, name = line.strip().partition(" ")
|
||||
if count.isdigit() and int(count) and name:
|
||||
found.append({"file": _home(name), "lines": int(count)})
|
||||
return found
|
||||
|
||||
def remove_history_sudo_lines(self) -> SudoHistoryCleanupDict:
|
||||
"""Delete those lines, as the login user, and give :attr:`SUDO_ROOT_FILES` back to root.
|
||||
|
||||
Two steps, ``history`` and ``owners``; the second runs when the first
|
||||
failed, too. Succeeds when no history file holds such a line any more
|
||||
and every file the login user owned is root's again.
|
||||
"""
|
||||
files: list[HistoryCleanupDict] = []
|
||||
restored: list[str] = []
|
||||
parts = [
|
||||
run_fix_steps([("history", lambda: self._clean_histories(files))]),
|
||||
run_fix_steps([("owners", lambda: self._restore_owners(restored))]),
|
||||
]
|
||||
steps: list[FixStepDict] = [s for part in parts for s in part["steps"]]
|
||||
return {
|
||||
"success": all(part["success"] for part in parts),
|
||||
"steps": steps,
|
||||
"output": "\n".join(f"[{s['step']}] {s['output']}".rstrip() for s in steps),
|
||||
"files": files,
|
||||
"removed": sum(f["before"] - f["after"] for f in files),
|
||||
"owners_restored": restored,
|
||||
}
|
||||
|
||||
def _clean_histories(self, files: list[HistoryCleanupDict]) -> FixStepOutcomeDict:
|
||||
marker = self.SUDO_HISTORY_MARKER
|
||||
if not marker:
|
||||
return {"ok": True, "output": "this driver never typed the sudo password"}
|
||||
result = self.run_command(cleanup_script(marker), timeout=60)
|
||||
said: list[str] = []
|
||||
ok = result.exit_code == 0
|
||||
for line in result.stdout.splitlines():
|
||||
ok = self._history_line(line.split(), files, said) and ok
|
||||
if result.exit_code != 0:
|
||||
said.append((result.stderr or result.stdout).strip())
|
||||
return {"ok": ok, "output": "\n".join(said) or "no shell history"}
|
||||
|
||||
@staticmethod
|
||||
def _history_line(words: list[str], files: list[HistoryCleanupDict], said: list[str]) -> bool:
|
||||
"""Take one line of :func:`cleanup_script` into *files* and *said*; whether it is clean."""
|
||||
kind, rest = (words[0], words[1:]) if words else ("", [])
|
||||
read = _CLEANUP_LINES.get(kind)
|
||||
if read is None or len(rest) != read[0]:
|
||||
return True
|
||||
clean, text, entry = read[1](rest)
|
||||
said.append(text)
|
||||
if entry is not None:
|
||||
files.append(entry)
|
||||
return clean
|
||||
|
||||
def _restore_owners(self, restored: list[str]) -> FixStepOutcomeDict:
|
||||
paths = tuple(self.SUDO_ROOT_FILES)
|
||||
if not paths:
|
||||
return {"ok": True, "output": "no file to check"}
|
||||
who = self.run_command("id -u; id -un", timeout=30).stdout.split()
|
||||
if len(who) < 2:
|
||||
return {"ok": False, "output": "cannot tell who the login user is"}
|
||||
uid, user = who[0], who[1]
|
||||
if uid == "0":
|
||||
return {"ok": True, "output": "logged in as root: every file it wrote is root's"}
|
||||
result = self.run_command(owners_script(user, paths), privileged=True, timeout=30)
|
||||
said: list[str] = []
|
||||
ok = result.exit_code == 0
|
||||
for line in result.stdout.splitlines():
|
||||
kind, _, path = line.strip().partition(" ")
|
||||
if kind == "fixed":
|
||||
restored.append(path)
|
||||
said.append(f"{path}: owned by root again")
|
||||
elif kind == "failed":
|
||||
ok = False
|
||||
said.append(f"{path}: could not be given back to root")
|
||||
if result.exit_code != 0:
|
||||
said.append((result.stderr or result.stdout).strip())
|
||||
return {"ok": ok, "output": "\n".join(said) or f"no file owned by {user}"}
|
||||
|
||||
|
||||
__all__ = [
|
||||
"HISTORY_BACKUP_SUFFIX",
|
||||
"HISTORY_FILES",
|
||||
"HistoryCleanupDict",
|
||||
"HistorySudoLinesDict",
|
||||
"SudoHistoryCleanupDict",
|
||||
"SudoHistoryMixin",
|
||||
"cleanup_script",
|
||||
"count_script",
|
||||
"owners_script",
|
||||
]
|
||||
@@ -10,7 +10,9 @@ Usage::
|
||||
...
|
||||
"""
|
||||
|
||||
from typing import Any, Awaitable, Callable, Dict, List, TYPE_CHECKING
|
||||
from collections.abc import Awaitable, Callable
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from napalm_device_types.base import DeviceTypeDriver
|
||||
from napalm_device_types.mac_acl import MacAclMixin
|
||||
from napalm_device_types.models import (
|
||||
@@ -45,7 +47,7 @@ class SwitchDriver(MacAclMixin, DeviceTypeDriver):
|
||||
async def get_health_metrics(
|
||||
cls,
|
||||
snmp_get: Callable[..., Awaitable[Any]],
|
||||
snmp_walk: Callable[..., Awaitable[Dict[str, Any]]],
|
||||
snmp_walk: Callable[..., Awaitable[dict[str, Any]]],
|
||||
) -> HealthMetricsDict:
|
||||
"""Collect SNMP health metrics for this switch type.
|
||||
|
||||
@@ -54,7 +56,7 @@ class SwitchDriver(MacAclMixin, DeviceTypeDriver):
|
||||
"""
|
||||
...
|
||||
|
||||
def get_spanning_tree(self) -> Dict[str, SpanningTreeDict]:
|
||||
def get_spanning_tree(self) -> dict[str, SpanningTreeDict]:
|
||||
"""
|
||||
Returns spanning tree status for each STP instance.
|
||||
|
||||
@@ -104,7 +106,7 @@ class SwitchDriver(MacAclMixin, DeviceTypeDriver):
|
||||
"""
|
||||
...
|
||||
|
||||
def get_port_channels(self) -> Dict[str, PortChannelDict]:
|
||||
def get_port_channels(self) -> dict[str, PortChannelDict]:
|
||||
"""
|
||||
Returns port-channel (LAG) configuration and status.
|
||||
|
||||
@@ -129,8 +131,7 @@ class SwitchDriver(MacAclMixin, DeviceTypeDriver):
|
||||
"""
|
||||
...
|
||||
|
||||
|
||||
def get_dot1x_ports(self) -> Dict[str, Dot1XPortDict]:
|
||||
def get_dot1x_ports(self) -> dict[str, Dot1XPortDict]:
|
||||
"""
|
||||
Returns the 802.1X / NAC configuration per switch port.
|
||||
|
||||
@@ -350,7 +351,7 @@ class SwitchDriver(MacAclMixin, DeviceTypeDriver):
|
||||
"""
|
||||
...
|
||||
|
||||
def set_lag_members(self, lag_name: str, members: List[str]) -> None:
|
||||
def set_lag_members(self, lag_name: str, members: list[str]) -> None:
|
||||
"""
|
||||
Sets the full member-port list of a LAG/trunk interface.
|
||||
|
||||
|
||||
@@ -0,0 +1,132 @@
|
||||
"""The SMBIOS identity of a host: system UUID, product serial, board serial.
|
||||
|
||||
What ties a server to its BMC. The BMC reports the server's serial number and
|
||||
system UUID; the host reads the same values from its firmware tables. Both sides
|
||||
read them the same way on every Linux host, so the command, its parse and the
|
||||
placeholder filter live here once, and a driver only carries the command across.
|
||||
|
||||
**Placeholders are no identity.** Board vendors leave defaults in the DMI tables
|
||||
that are the same on thousands of machines: a Supermicro board's product serial
|
||||
is ``0123456789``, others read ``To be filled by O.E.M.`` or ``Default string``,
|
||||
and a common UUID on cheap boards is ``03000200-0400-0500-0006-000700080009``.
|
||||
Matching on those would join unrelated machines, so they read as empty.
|
||||
|
||||
**Root.** ``product_uuid`` and ``product_serial`` are readable by root only; without
|
||||
it they come back empty, which is "not known", not a mismatch.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from napalm_device_types.models import SystemIdentityDict
|
||||
from napalm_device_types.terminal import strip_terminal_codes
|
||||
|
||||
_BEGIN = "IDBEGIN"
|
||||
_END = "IDEND"
|
||||
_DMI = "/sys/class/dmi/id"
|
||||
|
||||
#: Reads the three files in one round trip, framed so a banner or prompt around the
|
||||
#: output cannot shift a field.
|
||||
SYSTEM_IDENTITY_COMMAND = (
|
||||
f"printf '{_BEGIN}\\n%s\\n%s\\n%s\\n{_END}\\n' "
|
||||
f'"$(cat {_DMI}/product_uuid 2>/dev/null)" '
|
||||
f'"$(cat {_DMI}/product_serial 2>/dev/null)" '
|
||||
f'"$(cat {_DMI}/board_serial 2>/dev/null)"'
|
||||
)
|
||||
|
||||
#: Lowercased defaults board vendors leave in place.
|
||||
_PLACEHOLDERS = frozenset(
|
||||
{
|
||||
"",
|
||||
"none",
|
||||
"n/a",
|
||||
"na",
|
||||
"not specified",
|
||||
"not applicable",
|
||||
"not available",
|
||||
"to be filled by o.e.m.",
|
||||
"to be filled by oem",
|
||||
"default string",
|
||||
"default",
|
||||
"system serial number",
|
||||
"base board serial number",
|
||||
"chassis serial number",
|
||||
"serial number",
|
||||
"0123456789",
|
||||
"1234567890",
|
||||
"123456789",
|
||||
"o.e.m.",
|
||||
"oem",
|
||||
"03000200-0400-0500-0006-000700080009",
|
||||
}
|
||||
)
|
||||
#: A value of one repeated character ("0", "00000000", "FFFF-...") or of zeros and Fs.
|
||||
_REPEATED = re.compile(r"^(.)\1*$")
|
||||
|
||||
|
||||
def is_placeholder_identity(value: str | None) -> bool:
|
||||
"""Whether *value* is a vendor default rather than this machine's identity."""
|
||||
text = (value or "").strip().lower()
|
||||
if text in _PLACEHOLDERS:
|
||||
return True
|
||||
compact = text.replace("-", "").replace(" ", "")
|
||||
return bool(_REPEATED.match(compact))
|
||||
|
||||
|
||||
def _clean(value: str) -> str:
|
||||
value = value.strip()
|
||||
return "" if is_placeholder_identity(value) else value
|
||||
|
||||
|
||||
def parse_system_identity(output: str) -> SystemIdentityDict:
|
||||
"""The identity from :data:`SYSTEM_IDENTITY_COMMAND`'s output, placeholders empty.
|
||||
|
||||
:raises ValueError: when the output carries no intact frame.
|
||||
"""
|
||||
lines = strip_terminal_codes(output).splitlines()
|
||||
try:
|
||||
start = lines.index(_BEGIN) + 1
|
||||
end = lines.index(_END, start)
|
||||
except ValueError:
|
||||
raise ValueError("system identity: no intact IDBEGIN/IDEND frame") from None
|
||||
fields = (lines[start:end] + ["", "", ""])[:3]
|
||||
uuid, product_serial, board_serial = (_clean(f) for f in fields)
|
||||
return {
|
||||
"system_uuid": uuid.lower(),
|
||||
"product_serial": product_serial,
|
||||
"board_serial": board_serial,
|
||||
}
|
||||
|
||||
|
||||
class SystemIdentityMixin:
|
||||
"""Adds :meth:`get_system_identity` to a driver that can run a command on a Linux host.
|
||||
|
||||
The template form (README, "Function classes"): command and parse are the same
|
||||
everywhere and concrete here; a driver supplies only
|
||||
:meth:`_run_system_identity_command` -- how a command reaches its host, and how it
|
||||
gains root there. Mixed in by the drivers that can, so
|
||||
``hasattr(driver, "get_system_identity")`` stays a truthful answer.
|
||||
"""
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def _run_system_identity_command(self, command: str, *, privileged: bool) -> str:
|
||||
"""Run *command* on the host and return what it printed; as root when
|
||||
*privileged* (``product_uuid`` and ``product_serial`` are root's)."""
|
||||
...
|
||||
|
||||
def get_system_identity(self) -> SystemIdentityDict:
|
||||
"""
|
||||
Returns the host's SMBIOS identity, each value empty when unknown or a vendor
|
||||
placeholder:
|
||||
|
||||
* system_uuid (str) - lowercase
|
||||
* product_serial (str)
|
||||
* board_serial (str)
|
||||
|
||||
:raises ValueError: if the output carried no intact reading.
|
||||
"""
|
||||
output = self._run_system_identity_command(SYSTEM_IDENTITY_COMMAND, privileged=True)
|
||||
return parse_system_identity(output)
|
||||
@@ -0,0 +1,328 @@
|
||||
"""systemd services: listing them in one round trip, and starting and stopping them.
|
||||
|
||||
What systemd reports about its services, and how one is started or stopped, is
|
||||
the same on every host that runs it. So the command, its parse, the check of a
|
||||
unit name and the reading of an action's exit status live here once, and a
|
||||
driver only carries a command across: SSH, an API's exec endpoint, whatever it
|
||||
has.
|
||||
|
||||
**Listing.** One command prints the installed unit files and, for every loaded
|
||||
service unit, what ``systemctl show`` knows about it -- state, boot state and
|
||||
main PID together, instead of asking ``systemctl is-enabled`` and ``systemctl
|
||||
show`` once per unit (two hundred round trips on an ordinary Linux host). The
|
||||
report is framed, and a report whose end is missing raises: a list cut short
|
||||
must never read as services that went away.
|
||||
|
||||
**What counts as enabled.** A unit file state of ``enabled`` or
|
||||
``enabled-runtime``. ``static`` does not: such a unit starts only when
|
||||
something else pulls it in, and calling it enabled made every one of them look
|
||||
like a service of the host. The state is read from ``UnitFileState``, never
|
||||
from a column of ``list-unit-files``, whose second column has been followed by
|
||||
a preset column since systemd 245.
|
||||
|
||||
**Starting and stopping.** ``systemctl`` runs bounded by ``timeout`` and never
|
||||
asks for a password, and its exit status is printed after it. The marker also
|
||||
keeps the output from ever being empty, which a transport that retries on an
|
||||
empty answer would otherwise take as a reason to run the action twice.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from shlex import quote
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from napalm_device_types.models import ServiceDict
|
||||
from napalm_device_types.services import ServiceControlMixin
|
||||
from napalm_device_types.terminal import strip_terminal_codes
|
||||
|
||||
_BEGIN = "SVC_BEGIN"
|
||||
_END = "SVC_END"
|
||||
_NO_SYSTEMD = "no-systemd"
|
||||
_SUFFIX = ".service"
|
||||
|
||||
#: What ``systemctl show`` prints per unit. It prints them in its own order.
|
||||
_PROPERTIES = "Id,Names,LoadState,ActiveState,SubState,UnitFileState,MainPID"
|
||||
|
||||
#: Picks the units whose file state is ``generated`` out of ``systemctl show``'s
|
||||
#: output, whatever order it prints the properties in.
|
||||
_GENERATED_AWK = (
|
||||
'awk -F= \'NF<2{id="";g=0;next} $1=="Id"{id=$2} '
|
||||
'$1=="UnitFileState"{g=($2=="generated")} id!=""&&g{print id;id="";g=0}\''
|
||||
)
|
||||
|
||||
#: One line, POSIX ``sh``, read-only. The frame markers are printed in two
|
||||
#: halves so that a transport which echoes the command does not show them early.
|
||||
#: ``xargs -0`` passes escaped names such as ``foo\x2dbar.service`` unchanged.
|
||||
#: A generated unit -- the wrapper systemd makes for a SysV script -- has no unit
|
||||
#: file whose state says whether it starts at boot; ``systemctl is-enabled``
|
||||
#: asks the script's rc links instead, for those few units only.
|
||||
SYSTEMD_SERVICES_COMMAND = (
|
||||
"printf '%s%s\\n' SVC_ BEGIN; "
|
||||
"[ -d /run/systemd/system ] || echo '[no-systemd]'; "
|
||||
"echo '[files]'; systemctl list-unit-files --type=service --no-legend --no-pager 2>/dev/null; "
|
||||
"echo '[units]'; s=$(systemctl list-units --type=service --all --no-legend --no-pager --plain "
|
||||
"2>/dev/null | awk '{print $1}' | tr '\\n' '\\0' | xargs -0 -r systemctl show --no-pager "
|
||||
f"-p {_PROPERTIES} -- 2>/dev/null); printf '%s\\n' \"$s\"; "
|
||||
f"echo '[generated]'; printf '%s\\n' \"$s\" | {_GENERATED_AWK} | while read -r u; do "
|
||||
'printf \'%s %s\\n\' "$u" "$(systemctl is-enabled -- "$u" 2>/dev/null)"; done; '
|
||||
"printf '%s%s\\n' SVC_ END"
|
||||
)
|
||||
|
||||
#: The lifecycle actions :meth:`SystemdServicesMixin.manage_service` accepts.
|
||||
SERVICE_ACTIONS = ("start", "stop", "restart", "enable", "disable")
|
||||
|
||||
#: Seconds an action may run on the host before ``timeout`` stops waiting for
|
||||
#: it. systemd itself carries on with the job.
|
||||
ACTION_TIMEOUT = 45
|
||||
|
||||
#: What a transport should allow for one command: the action's own bound plus
|
||||
#: the round trip around it.
|
||||
_TRANSPORT_TIMEOUT = ACTION_TIMEOUT + 15
|
||||
|
||||
_TIMED_OUT = 124 # timeout(1)'s exit status when the time ran out
|
||||
_RC_MARKER = "__SVC_RC="
|
||||
_RC_RE = re.compile(rf"^{_RC_MARKER}(\d+)\s*$", re.MULTILINE)
|
||||
|
||||
#: The characters systemd allows in a unit name, with ``\xHH`` for any other byte.
|
||||
_UNIT_RE = re.compile(r"(?:[A-Za-z0-9_.:@-]|\\x[0-9A-Fa-f]{2})+")
|
||||
_MAX_UNIT_LENGTH = 255
|
||||
|
||||
_ENABLED = frozenset({"enabled", "enabled-runtime"})
|
||||
#: Unit file states of a service that is installed but need not be loaded.
|
||||
_INSTALLED = frozenset({"enabled", "enabled-runtime", "disabled", "indirect"})
|
||||
|
||||
|
||||
class SystemdUnavailable(NotImplementedError):
|
||||
"""The host does not run systemd; a driver may fall back to another init system."""
|
||||
|
||||
|
||||
def unit_name(name: str) -> str:
|
||||
"""*name* as a service unit's name without ``.service``, or ``ValueError``.
|
||||
|
||||
Accepts template instances (``wg-quick@wg0``), dots (``snapd.apparmor``),
|
||||
colons and systemd's ``\\xHH`` escapes. Refuses a bare template
|
||||
(``getty@``), a leading ``-`` that a command would read as an option, and
|
||||
anything a shell would read.
|
||||
"""
|
||||
base = name[: -len(_SUFFIX)] if name.endswith(_SUFFIX) else name
|
||||
if (
|
||||
not _UNIT_RE.fullmatch(base)
|
||||
or base.startswith("-")
|
||||
or base.endswith("@")
|
||||
or len(base) + len(_SUFFIX) > _MAX_UNIT_LENGTH
|
||||
):
|
||||
raise ValueError(f"Invalid service name: {name!r}")
|
||||
return base
|
||||
|
||||
|
||||
def service_action_command(name: str, action: str) -> str:
|
||||
"""The shell command that applies *action* to the service *name*.
|
||||
|
||||
:raises ValueError: for an unknown action or an invalid name.
|
||||
"""
|
||||
if action not in SERVICE_ACTIONS:
|
||||
raise ValueError(f"Invalid action {action!r}; use one of {', '.join(SERVICE_ACTIONS)}")
|
||||
unit = quote(unit_name(name) + _SUFFIX)
|
||||
return (
|
||||
f"timeout {ACTION_TIMEOUT} systemctl --no-ask-password {action} -- {unit} 2>&1; "
|
||||
f"echo {_RC_MARKER}$?"
|
||||
)
|
||||
|
||||
|
||||
def parse_action_result(output: str) -> dict[str, Any]:
|
||||
"""``{"success", "output"}`` from what :func:`service_action_command` printed.
|
||||
|
||||
Only the exit status decides. A job still running when ``timeout`` gave up
|
||||
is not reported as done, and output without a status is no success.
|
||||
"""
|
||||
output = strip_terminal_codes(output)
|
||||
statuses = _RC_RE.findall(output)
|
||||
text = _RC_RE.sub("", output).strip()
|
||||
if not statuses:
|
||||
return {"success": False, "output": text or "No exit status came back from the host."}
|
||||
status = int(statuses[-1])
|
||||
if status == 0:
|
||||
return {"success": True, "output": text}
|
||||
if status == _TIMED_OUT:
|
||||
note = f"Still running after {ACTION_TIMEOUT} s; systemd carries on with the job."
|
||||
return {"success": False, "output": f"{text}\n{note}".strip()}
|
||||
return {"success": False, "output": text or f"systemctl exited with status {status}."}
|
||||
|
||||
|
||||
def _frame(output: str) -> list[str]:
|
||||
lines = [line.strip() for line in strip_terminal_codes(output).splitlines()]
|
||||
try:
|
||||
start = lines.index(_BEGIN)
|
||||
end = lines.index(_END, start)
|
||||
except ValueError:
|
||||
raise ValueError("no intact systemd service report in the output") from None
|
||||
return lines[start + 1 : end]
|
||||
|
||||
|
||||
def _sections(lines: list[str]) -> dict[str, list[str]]:
|
||||
sections: dict[str, list[str]] = {}
|
||||
current: list[str] = []
|
||||
for line in lines:
|
||||
if line.startswith("[") and line.endswith("]"):
|
||||
current = sections.setdefault(line[1:-1], [])
|
||||
else:
|
||||
current.append(line)
|
||||
return sections
|
||||
|
||||
|
||||
def _unit_blocks(lines: list[str]) -> list[dict[str, str]]:
|
||||
"""``systemctl show``'s output, one dict per unit.
|
||||
|
||||
Units are separated by a blank line -- except where ``xargs`` split the
|
||||
list over two runs and the blocks meet, so a key seen twice starts the next
|
||||
unit as well.
|
||||
"""
|
||||
blocks: list[dict[str, str]] = []
|
||||
current: dict[str, str] = {}
|
||||
for line in lines:
|
||||
key, sep, value = line.partition("=")
|
||||
if not sep or key in current:
|
||||
if current:
|
||||
blocks.append(current)
|
||||
current = {}
|
||||
if sep:
|
||||
current[key] = value
|
||||
if current:
|
||||
blocks.append(current)
|
||||
return blocks
|
||||
|
||||
|
||||
def _base(unit: str) -> str:
|
||||
return unit[: -len(_SUFFIX)]
|
||||
|
||||
|
||||
def _main_pid(block: dict[str, str]) -> int:
|
||||
try:
|
||||
return int(block.get("MainPID") or 0)
|
||||
except ValueError:
|
||||
return 0
|
||||
|
||||
|
||||
def _loaded(blocks: list[dict[str, str]]) -> tuple[dict[str, ServiceDict], set[str]]:
|
||||
"""The loaded services, and every name they go by (aliases included)."""
|
||||
services: dict[str, ServiceDict] = {}
|
||||
names: set[str] = set()
|
||||
for block in blocks:
|
||||
unit = block.get("Id", "")
|
||||
if not unit.endswith(_SUFFIX) or block.get("LoadState") == "not-found":
|
||||
continue
|
||||
names.update(block.get("Names", unit).split())
|
||||
running = block.get("ActiveState") == "active" and block.get("SubState") == "running"
|
||||
services[_base(unit)] = {
|
||||
"name": _base(unit),
|
||||
"running": running,
|
||||
"enabled": block.get("UnitFileState") in _ENABLED,
|
||||
"pid": _main_pid(block) if running else 0,
|
||||
}
|
||||
return services, names
|
||||
|
||||
|
||||
def _installed(lines: list[str], known: set[str]) -> dict[str, ServiceDict]:
|
||||
"""Installed services that are not loaded: neither running nor starting now.
|
||||
|
||||
Templates, static units and aliases are left out -- the last also when an
|
||||
older systemd lists an alias as ``enabled``, which is why every name a
|
||||
loaded unit goes by is skipped.
|
||||
"""
|
||||
services: dict[str, ServiceDict] = {}
|
||||
for line in lines:
|
||||
parts = line.split()
|
||||
if len(parts) < 2:
|
||||
continue
|
||||
unit, state = parts[0], parts[1]
|
||||
if (
|
||||
not unit.endswith(_SUFFIX)
|
||||
or unit.endswith("@" + _SUFFIX)
|
||||
or unit in known
|
||||
or state not in _INSTALLED
|
||||
):
|
||||
continue
|
||||
services[_base(unit)] = {
|
||||
"name": _base(unit),
|
||||
"running": False,
|
||||
"enabled": state in _ENABLED,
|
||||
"pid": 0,
|
||||
}
|
||||
return services
|
||||
|
||||
|
||||
def _apply_generated(services: dict[str, ServiceDict], lines: list[str]) -> None:
|
||||
"""Take a generated unit's boot state from ``is-enabled``'s answer."""
|
||||
for line in lines:
|
||||
parts = line.split()
|
||||
if len(parts) == 2 and parts[0].endswith(_SUFFIX) and _base(parts[0]) in services:
|
||||
services[_base(parts[0])]["enabled"] = parts[1] in _ENABLED
|
||||
|
||||
|
||||
def parse_systemd_services(output: str) -> list[ServiceDict]:
|
||||
"""Parse what :data:`SYSTEMD_SERVICES_COMMAND` printed, sorted by name.
|
||||
|
||||
Lists every loaded service unit but those that are not found, and every
|
||||
installed one that is not loaded.
|
||||
|
||||
:raises SystemdUnavailable: when the host does not run systemd.
|
||||
:raises ValueError: when the output carries no intact report.
|
||||
"""
|
||||
sections = _sections(_frame(output))
|
||||
if _NO_SYSTEMD in sections:
|
||||
raise SystemdUnavailable("the host does not run systemd")
|
||||
loaded, known = _loaded(_unit_blocks(sections.get("units", [])))
|
||||
_apply_generated(loaded, sections.get("generated", []))
|
||||
merged = {**_installed(sections.get("files", []), known), **loaded}
|
||||
return [merged[name] for name in sorted(merged)]
|
||||
|
||||
|
||||
class SystemdServicesMixin(ServiceControlMixin):
|
||||
"""Implements :class:`ServiceControlMixin` for a driver whose host runs systemd.
|
||||
|
||||
The template form (README, "Function classes"): the command, the parse,
|
||||
the check of the name and the reading of the exit status are the same
|
||||
everywhere, so they are concrete here, and a driver supplies only
|
||||
:meth:`_run_service_command` -- how a command reaches its host, and how it
|
||||
gains root there when it needs to.
|
||||
"""
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def _run_service_command(self, command: str, *, privileged: bool, timeout: int) -> str:
|
||||
"""Run *command* with ``sh`` on the host and return what it printed.
|
||||
|
||||
*privileged* commands change the system and need root; *timeout*
|
||||
is how long the transport should wait for the output, in seconds.
|
||||
"""
|
||||
...
|
||||
|
||||
def get_services(self) -> list[ServiceDict]:
|
||||
"""
|
||||
Returns the services systemd knows, in one round trip.
|
||||
|
||||
* name (string) - the unit name without ``.service``
|
||||
* running (bool) - active and running
|
||||
* enabled (bool) - the unit file is enabled
|
||||
* pid (int) - the main process; 0 when not running
|
||||
|
||||
:raises SystemdUnavailable: if the host does not run systemd.
|
||||
:raises ValueError: if the host's output carried no intact report.
|
||||
"""
|
||||
output = self._run_service_command(
|
||||
SYSTEMD_SERVICES_COMMAND, privileged=False, timeout=_TRANSPORT_TIMEOUT
|
||||
)
|
||||
return parse_systemd_services(output)
|
||||
|
||||
def manage_service(self, name: str, action: str) -> dict[str, Any]:
|
||||
"""
|
||||
Applies *action* (start, stop, restart, enable, disable) to the service *name*.
|
||||
|
||||
:returns: ``{"success": bool, "output": str}``
|
||||
:raises ValueError: for an unknown action or an invalid name, before
|
||||
anything is sent.
|
||||
"""
|
||||
command = service_action_command(name, action)
|
||||
output = self._run_service_command(command, privileged=True, timeout=_TRANSPORT_TIMEOUT)
|
||||
return parse_action_result(output)
|
||||
@@ -0,0 +1,22 @@
|
||||
"""What a pseudo-terminal adds to a command's output, taken out again.
|
||||
|
||||
A screen-scraping transport (netmiko) gives the remote command a terminal. Tools
|
||||
then colour their output and draw progress: systemctl colours its errors, apt
|
||||
switches the keypad mode with ``ESC =`` / ``ESC >``. Every parser in this package
|
||||
reads the text without them.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
|
||||
#: CSI sequences (colours, cursor), OSC sequences (window titles) and the
|
||||
#: two-character escapes (``ESC =``, ``ESC >``, ``ESC (B``).
|
||||
_TERMINAL_CODES = re.compile(
|
||||
r"\x1b(?:\[[0-?]*[ -/]*[@-~]|\][^\x07\x1b]*(?:\x07|\x1b\\)|\([0-9A-Za-z]|[=>78DEHMNOc])"
|
||||
)
|
||||
|
||||
|
||||
def strip_terminal_codes(text: str) -> str:
|
||||
"""*text* without terminal escape sequences."""
|
||||
return _TERMINAL_CODES.sub("", text)
|
||||
@@ -0,0 +1,285 @@
|
||||
"""TLS certificates: trust on first use by fingerprint, a changed one refused before any login.
|
||||
|
||||
Network devices ship self-signed certificates no CA vouches for. A driver that
|
||||
then turns verification off logs in with the device's credentials at whatever
|
||||
answers on its address (NetOrk/netork#1167, NAPALM/napalm-hpe-aruba-procurve#17).
|
||||
The same answer as for SSH host keys (:mod:`napalm_device_types.ssh_host_keys`):
|
||||
the layer above keeps the store of the certificates it trusts, a driver verifies.
|
||||
|
||||
* It receives the fingerprints to expect in ``optional_args["tls_fingerprints"]``
|
||||
(:data:`KNOWN_FINGERPRINTS_ARG`), by the port they were seen on:
|
||||
``{"443": "SHA256:AB:CD:…"}``, the SHA-256 of the DER certificate as
|
||||
``openssl x509 -fingerprint -sha256`` prints it (the ``SHA256:`` prefix,
|
||||
colons and case are optional, :func:`normalise_fingerprint`). With a
|
||||
fingerprint, any other certificate is refused before a password leaves the
|
||||
caller; no CA is consulted. Without one, the first certificate is accepted:
|
||||
trust on first use.
|
||||
* It reports every certificate it sees through
|
||||
``optional_args["tls_certificate_observer"]`` (:data:`OBSERVER_ARG`), a
|
||||
callable that receives one dict per connection::
|
||||
|
||||
{"host": str, "port": int,
|
||||
"seen": {"fingerprint"},
|
||||
"expected": {"fingerprint"} | None,
|
||||
"accepted": bool}
|
||||
|
||||
``accepted`` is False only for a refused certificate; the connection then
|
||||
raises :class:`TlsCertificateMismatch`. A failing observer is logged and
|
||||
ignored.
|
||||
|
||||
A caller that passes neither keeps working as before (first use, every time),
|
||||
and a driver that ignores both still connects for a caller that passes them.
|
||||
|
||||
:class:`CertificateCheck` is the one implementation. :meth:`~CertificateCheck.verify_server`
|
||||
takes the certificate from a handshake of its own, before any request;
|
||||
:meth:`~CertificateCheck.pin` makes every later connection of a ``requests``
|
||||
session present that same certificate (urllib3's ``assert_fingerprint``), so
|
||||
another host cannot step in between the check and the login;
|
||||
:meth:`~CertificateCheck.refusing` turns such a refusal into
|
||||
:class:`TlsCertificateMismatch`; :meth:`~CertificateCheck.observe` reports the
|
||||
certificate once the login stands.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import logging
|
||||
import re
|
||||
import socket
|
||||
import ssl
|
||||
from collections.abc import Callable, Iterator, Mapping
|
||||
from contextlib import contextmanager
|
||||
from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
from napalm.base.exceptions import ConnectionException
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
#: The ``optional_args`` key that carries the fingerprints to expect, by port.
|
||||
KNOWN_FINGERPRINTS_ARG = "tls_fingerprints"
|
||||
#: The ``optional_args`` key that carries the callable every seen certificate goes to.
|
||||
OBSERVER_ARG = "tls_certificate_observer"
|
||||
|
||||
_PREFIX = "SHA256:"
|
||||
_SHA256_HEX = re.compile(r"^[0-9A-F]{64}$")
|
||||
#: What urllib3 says when a connection presents another certificate than the pinned one
|
||||
#: (``got "…"``; 1.26 writes the digest as ``b'…'``, escaped once requests wraps it).
|
||||
_URLLIB3_MISMATCH = re.compile(r"Fingerprints did not match.*?got \"(?:b\\?')?([0-9a-fA-F]{64})")
|
||||
|
||||
|
||||
def fingerprint(der: bytes) -> str:
|
||||
"""``SHA256:AB:CD:…``: the SHA-256 of a DER certificate, as openssl prints it."""
|
||||
return _format(hashlib.sha256(der).hexdigest())
|
||||
|
||||
|
||||
def _format(hex_digest: str) -> str:
|
||||
upper = hex_digest.upper()
|
||||
return _PREFIX + ":".join(upper[i : i + 2] for i in range(0, len(upper), 2))
|
||||
|
||||
|
||||
def normalise_fingerprint(value: Any) -> str:
|
||||
"""*value* as :func:`fingerprint` writes it; ``SHA256:``, colons, spaces and case optional."""
|
||||
if not isinstance(value, str):
|
||||
raise ValueError(f"not a SHA-256 certificate fingerprint: {value!r}")
|
||||
text = value.strip()
|
||||
if text[: len(_PREFIX)].upper() == _PREFIX:
|
||||
text = text[len(_PREFIX) :]
|
||||
digits = re.sub(r"[\s:]", "", text).upper()
|
||||
if not _SHA256_HEX.match(digits):
|
||||
raise ValueError(f"not a SHA-256 certificate fingerprint: {value!r}")
|
||||
return _format(digits)
|
||||
|
||||
|
||||
def _hex(fingerprint_: str) -> str:
|
||||
"""The bare lower-case hex urllib3's ``assert_fingerprint`` takes."""
|
||||
return fingerprint_.removeprefix(_PREFIX).replace(":", "").lower()
|
||||
|
||||
|
||||
def fetch_certificate(
|
||||
host: str,
|
||||
port: int,
|
||||
timeout: float,
|
||||
*,
|
||||
connect: Callable[..., socket.socket] = socket.create_connection,
|
||||
) -> bytes:
|
||||
"""The DER certificate *host* presents on *port*, from a handshake that sends nothing else.
|
||||
|
||||
No CA and no host name are checked: the certificate is judged by its fingerprint.
|
||||
"""
|
||||
context = ssl.create_default_context()
|
||||
context.check_hostname = False
|
||||
context.verify_mode = ssl.CERT_NONE
|
||||
with connect((host, port), timeout=timeout) as sock:
|
||||
with context.wrap_socket(sock, server_hostname=host) as tls:
|
||||
der = tls.getpeercert(binary_form=True)
|
||||
if not der: # pragma: no cover - a TLS server always presents one
|
||||
raise ConnectionException(f"{host}:{port} presented no TLS certificate")
|
||||
return der
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CertificateObservation:
|
||||
"""The certificate one connection saw, against the one it was to expect."""
|
||||
|
||||
host: str
|
||||
port: int
|
||||
#: The fingerprint of the certificate presented.
|
||||
seen: str
|
||||
#: The fingerprint it was to present; None on first use.
|
||||
expected: str | None
|
||||
|
||||
@property
|
||||
def accepted(self) -> bool:
|
||||
"""True on first use and for the expected certificate; False for a refused one."""
|
||||
return self.expected is None or self.expected == self.seen
|
||||
|
||||
def as_dict(self) -> dict[str, Any]:
|
||||
return {
|
||||
"host": self.host,
|
||||
"port": self.port,
|
||||
"seen": {"fingerprint": self.seen},
|
||||
"expected": {"fingerprint": self.expected} if self.expected else None,
|
||||
"accepted": self.accepted,
|
||||
}
|
||||
|
||||
|
||||
class TlsCertificateMismatch(ConnectionException): # type: ignore[misc]
|
||||
"""The device presented another TLS certificate than the one it is known by.
|
||||
|
||||
Raised before any request that carries credentials: nothing was sent to it.
|
||||
"""
|
||||
|
||||
def __init__(self, observation: CertificateObservation) -> None:
|
||||
self.observation = observation
|
||||
super().__init__(
|
||||
f"TLS certificate mismatch at {observation.host}:{observation.port}: it presented "
|
||||
f"{observation.seen}, the trusted certificate is {observation.expected or 'none'}. "
|
||||
"Nothing was sent to it."
|
||||
)
|
||||
|
||||
|
||||
def _expected(known: Any, port: int) -> str | None:
|
||||
"""The fingerprint in *known* (``{port: fingerprint}``, port as text or number) for *port*."""
|
||||
if not known:
|
||||
return None
|
||||
if not isinstance(known, Mapping):
|
||||
raise ValueError(f"TLS fingerprints must be given by port, not as {type(known).__name__}")
|
||||
value = known.get(str(port)) or known.get(port)
|
||||
return normalise_fingerprint(value) if value else None
|
||||
|
||||
|
||||
def _refused_fingerprint(exc: BaseException) -> str | None:
|
||||
"""The fingerprint urllib3 saw when it refused a pinned connection, in *exc*'s chain."""
|
||||
seen: set[int] = set()
|
||||
stack: list[BaseException | None] = [exc]
|
||||
while stack:
|
||||
current = stack.pop()
|
||||
if current is None or id(current) in seen:
|
||||
continue
|
||||
seen.add(id(current))
|
||||
match = _URLLIB3_MISMATCH.search(str(current))
|
||||
if match:
|
||||
return _format(match.group(1))
|
||||
stack.extend([current.__cause__, current.__context__])
|
||||
return None
|
||||
|
||||
|
||||
class CertificateCheck:
|
||||
"""How one driver connection verifies its host's TLS certificate.
|
||||
|
||||
*optional_args* are the driver's (both keys optional), *host* and *port* the
|
||||
address the connection goes to.
|
||||
"""
|
||||
|
||||
def __init__(self, optional_args: Mapping[str, Any] | None, host: str, port: int) -> None:
|
||||
args = optional_args or {}
|
||||
self.host = host
|
||||
self.port = int(port)
|
||||
self.expected = _expected(args.get(KNOWN_FINGERPRINTS_ARG), self.port)
|
||||
self._observer = args.get(OBSERVER_ARG)
|
||||
self._verified: CertificateObservation | None = None
|
||||
|
||||
def verify_server(
|
||||
self,
|
||||
timeout: float,
|
||||
*,
|
||||
connect: Callable[..., socket.socket] = socket.create_connection,
|
||||
) -> CertificateObservation:
|
||||
"""Take the certificate from a handshake of its own and :meth:`verify` it."""
|
||||
return self.verify(fetch_certificate(self.host, self.port, timeout, connect=connect))
|
||||
|
||||
def verify(self, der: bytes) -> CertificateObservation:
|
||||
"""Judge the DER certificate *der*: a refused one is reported and raises.
|
||||
|
||||
An accepted one is reported by :meth:`observe`, once the login stands.
|
||||
"""
|
||||
observation = CertificateObservation(self.host, self.port, fingerprint(der), self.expected)
|
||||
if not observation.accepted:
|
||||
self._report(observation)
|
||||
raise TlsCertificateMismatch(observation)
|
||||
self._verified = observation
|
||||
return observation
|
||||
|
||||
def pin(self, session: Any) -> None:
|
||||
"""Every HTTPS connection of the ``requests`` *session* must present the verified one."""
|
||||
verified = self._require_verified()
|
||||
session.mount("https://", _pinned_adapter(_hex(verified.seen)))
|
||||
|
||||
@contextmanager
|
||||
def refusing(self) -> Iterator[None]:
|
||||
"""Around requests on a pinned session: a refused certificate becomes a mismatch."""
|
||||
try:
|
||||
yield
|
||||
except Exception as exc:
|
||||
offered = _refused_fingerprint(exc)
|
||||
if offered is None:
|
||||
raise
|
||||
trusted = self.expected or self._require_verified().seen
|
||||
observation = CertificateObservation(self.host, self.port, offered, trusted)
|
||||
self._report(observation)
|
||||
raise TlsCertificateMismatch(observation) from exc
|
||||
|
||||
def observe(self) -> CertificateObservation:
|
||||
"""After the login: the verified certificate, reported."""
|
||||
verified = self._require_verified()
|
||||
self._report(verified)
|
||||
return verified
|
||||
|
||||
def _require_verified(self) -> CertificateObservation:
|
||||
if self._verified is None:
|
||||
raise RuntimeError("verify the certificate first (verify_server or verify)")
|
||||
return self._verified
|
||||
|
||||
def _report(self, observation: CertificateObservation) -> None:
|
||||
if not callable(self._observer):
|
||||
return
|
||||
try:
|
||||
self._observer(observation.as_dict())
|
||||
except Exception: # the store is the caller's; a connection does not hang on it
|
||||
logger.warning(
|
||||
"Could not report the TLS certificate of %s:%s",
|
||||
self.host,
|
||||
self.port,
|
||||
exc_info=True,
|
||||
)
|
||||
|
||||
|
||||
def _pinned_adapter(hex_digest: str) -> Any:
|
||||
"""A ``requests`` adapter whose connections, direct or through a proxy, must match *hex_digest*.
|
||||
|
||||
``requests`` is imported here: it comes with napalm, and a module that only
|
||||
verifies a fingerprint does not need it loaded.
|
||||
"""
|
||||
from requests.adapters import HTTPAdapter
|
||||
|
||||
class _PinnedAdapter(HTTPAdapter): # type: ignore[misc,unused-ignore]
|
||||
def init_poolmanager(self, *args: Any, **kwargs: Any) -> None:
|
||||
kwargs["assert_fingerprint"] = hex_digest
|
||||
super().init_poolmanager(*args, **kwargs)
|
||||
|
||||
def proxy_manager_for(self, proxy: str, **kwargs: Any) -> Any:
|
||||
kwargs["assert_fingerprint"] = hex_digest
|
||||
return super().proxy_manager_for(proxy, **kwargs)
|
||||
|
||||
return _PinnedAdapter()
|
||||
@@ -1,4 +1,3 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Pending package updates and applying them.
|
||||
|
||||
The two device types that offer this used to declare it under two different
|
||||
@@ -13,15 +12,15 @@ this class in can never shadow a working implementation from a sibling base.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import List, TYPE_CHECKING
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from napalm_device_types.models import ApplyUpdatesResultDict, UpdateDict
|
||||
from napalm_device_types.models import ApplyUpdatesResultDict, UpdateDict, UpdateProgressDict
|
||||
|
||||
|
||||
class UpdateMixin:
|
||||
if TYPE_CHECKING:
|
||||
|
||||
def get_available_updates(self) -> List[UpdateDict]:
|
||||
def get_available_updates(self) -> list[UpdateDict]:
|
||||
"""
|
||||
Returns a list of packages that have a newer version available.
|
||||
|
||||
@@ -30,6 +29,14 @@ class UpdateMixin:
|
||||
* name (string) - package name
|
||||
* current_version (string) - currently installed version
|
||||
* new_version (string) - version available in the repository
|
||||
* origin (string, optional) - where it comes from, e.g. apt's suites
|
||||
* security (bool or None, optional) - a security update; leave it
|
||||
out or None when the source does not say
|
||||
|
||||
**An empty list means nothing is pending.** A reader that cannot
|
||||
read -- no package index yet, an API that did not answer -- raises
|
||||
instead: netOrk keeps "pending since" per package, and an empty
|
||||
list for "don't know" would reset every one of those clocks.
|
||||
|
||||
Example::
|
||||
|
||||
@@ -43,7 +50,29 @@ class UpdateMixin:
|
||||
"""
|
||||
...
|
||||
|
||||
def apply_updates(self, packages: List[str]) -> ApplyUpdatesResultDict:
|
||||
def refresh_available_updates(self) -> dict[str, Any]:
|
||||
"""
|
||||
Refreshes the host's package index, so that :meth:`get_available_updates`
|
||||
reports what the repositories offer now (``apt-get update``,
|
||||
``dnf makecache``, ``opkg update``, a firmware check). Installs nothing.
|
||||
|
||||
:returns: ``{"success": bool, "output": str}``
|
||||
"""
|
||||
...
|
||||
|
||||
def get_update_progress(self) -> UpdateProgressDict:
|
||||
"""
|
||||
Where an update the device runs by itself stands, for a caller that
|
||||
follows it: a firewall's firmware upgrade, which :meth:`apply_updates`
|
||||
only starts. ``state`` is ``"running"`` until it ends in ``"done"``
|
||||
or ``"failed"``; ``log`` is the device's whole log of it.
|
||||
|
||||
Declared only: a driver whose :meth:`apply_updates` returns when the
|
||||
update is over has nothing to report here.
|
||||
"""
|
||||
...
|
||||
|
||||
def apply_updates(self, packages: list[str]) -> ApplyUpdatesResultDict:
|
||||
"""
|
||||
Upgrades the given packages to the newest available version.
|
||||
|
||||
|
||||
@@ -0,0 +1,413 @@
|
||||
"""Virtual IPs: the addresses a host's HA configuration lets float between machines.
|
||||
|
||||
vip-manager and keepalived put a service address on whichever member is master
|
||||
*right now*. The address list of a standby shows no trace of it; what every
|
||||
member has is the configuration that declares it. That is read the same way on
|
||||
every Linux host, so the command and its parse live here once, and a driver
|
||||
only carries the command across.
|
||||
|
||||
**Declared, not held.** The reading says which addresses the configuration
|
||||
lets onto this host. Whether the host holds one at the moment is in its
|
||||
interface addresses, which the caller already has.
|
||||
|
||||
**Filtered on the host.** The same files hold secrets -- vip-manager's etcd
|
||||
password, keepalived's ``auth_pass``, notify scripts with their arguments. The
|
||||
command therefore never prints a configuration: an awk program reads the unit's
|
||||
properties and files and prints allowlisted fields only. A secret cannot reach
|
||||
the caller's memory, its log or an exception text, because it never leaves the
|
||||
host.
|
||||
|
||||
**What is read.**
|
||||
|
||||
- *vip-manager*: every ``vip-manager.service`` and ``vip-manager@*.service`` unit
|
||||
systemd knows. The address comes from the first of: a command-line flag
|
||||
(``--ip``; 1.x's ``-ip=${VIP_IP}`` resolved through the unit's environment), the
|
||||
environment (``VIP_IP``, an ``EnvironmentFile`` over ``Environment``), the
|
||||
``--config`` YAML (5.x's ``ip``). Mask, interface and trigger key alike.
|
||||
- *keepalived*: ``keepalived.service``, its ``-f``/``--use-file`` or
|
||||
``/etc/keepalived/keepalived.conf``. The ``virtual_ipaddress`` and
|
||||
``virtual_ipaddress_excluded`` entries of every ``vrrp_instance``, with the
|
||||
instance's ``interface`` and ``virtual_router_id``. ``include`` is followed
|
||||
four levels deep, a relative pattern from the including file's directory. A
|
||||
pattern that is not a plain path glob is not followed: it would otherwise be
|
||||
handed to a root shell.
|
||||
|
||||
**None is "did not look".** A file that exists but cannot be read, an include
|
||||
that cannot be followed, a host without systemd (no ``systemctl``, or systemd not
|
||||
running as PID 1, as in a container): that mechanism is None, and the
|
||||
caller keeps what it knew. ``[]`` is a host that has none.
|
||||
|
||||
**Root, and without it.** The configuration may be root's alone. The whole
|
||||
script goes to the host as one ``sh -c`` argument; when the privileged call
|
||||
brings no report back, it runs again without privilege and reads what it can.
|
||||
|
||||
**An exec channel, not a terminal.** With its awk program the command is about
|
||||
4 kB, past the line an interactive shell reliably takes. A driver sends it the
|
||||
way it runs any command that needs no PTY -- napalm-linux's ``run_command``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import ipaddress
|
||||
import re
|
||||
from collections.abc import Callable
|
||||
from dataclasses import dataclass, field
|
||||
from shlex import quote
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from napalm_device_types.models import VirtualIpDict, VirtualIpsDict
|
||||
from napalm_device_types.terminal import strip_terminal_codes
|
||||
|
||||
_BEGIN = "VIPS_BEGIN"
|
||||
_END = "VIPS_END"
|
||||
_NO_SYSTEMD = "no-systemd"
|
||||
#: Exists only while systemd runs as PID 1; a container may carry systemctl without.
|
||||
_SYSTEMD_RUN = "/run/systemd/system"
|
||||
#: A record that says a mechanism was not fully read.
|
||||
_BLIND = frozenset({"unreadable", "too-deep"})
|
||||
_KEEPALIVED_CONF = "/etc/keepalived/keepalived.conf"
|
||||
_ENV_KEYS = "VIP_(IP|MASK|NETMASK|IFACE|INTERFACE|KEY|TRIGGER_KEY)"
|
||||
_CFG_KEYS = "(ip|netmask|mask|interface|iface|trigger-key|key)"
|
||||
|
||||
#: One line of awk, fed a unit's ``systemctl show`` output with ``-v m=<mechanism>``.
|
||||
#: It prints records, never a line of configuration: ``active <state>``,
|
||||
#: ``arg <flag> <value>``, ``env <VIP_*> <value>``, ``cfg <key> <value>``,
|
||||
#: ``instance <name>`` / ``interface`` / ``virtual_router_id`` / ``vip <addr> [dev <if>]``
|
||||
#: / ``end``, and ``unreadable``/``too-deep``/``missing <path>``. No ``>`` but in
|
||||
#: ``2>/dev/null``: comparisons are written the other way round. ``\047`` is a
|
||||
#: single quote, so the program itself needs none. A statement ends with its line
|
||||
#: unless the next one opens with ``{``: keepalived takes ``vrrp_instance VI_2``
|
||||
#: and its brace on two lines.
|
||||
_AWK = " ".join(
|
||||
(
|
||||
r'function q(p) { return "\047" p "\047" }',
|
||||
r'function ok(f) { if (f == "" || f ~ /\047/) { print "unreadable " f; return 0 }'
|
||||
r' if (system("test -r " q(f)) == 0) return 1;'
|
||||
r' if (system("test -e " q(f)) == 0) print "unreadable " f; else print "missing " f;'
|
||||
r" return 0 }",
|
||||
r"function rf(f, c, l, r, k) { if (!ok(f)) return;"
|
||||
r" while (0 < (r = (getline l < f))) {"
|
||||
rf' if (c == "cfg" && l ~ /^[ \t]*{_CFG_KEYS}[ \t]*:/) {{ sub(/^[ \t]*/, "", l); k = l;'
|
||||
r' sub(/[ \t]*:.*/, "", k); sub(/^[^:]*:[ \t]*/, "", l); print "cfg " k " " l }'
|
||||
rf' else if (c == "env" && l ~ /^[ \t]*(export[ \t]+)?{_ENV_KEYS}[ \t]*=/) {{'
|
||||
r' sub(/^[ \t]*(export[ \t]+)?/, "", l); k = l; sub(/[ \t]*=.*/, "", k);'
|
||||
r' sub(/^[^=]*=[ \t]*/, "", l); print "env " k " " l } }'
|
||||
r' if (r < 0) print "unreadable " f; close(f) }',
|
||||
r'function push(n, a) { st[++sd] = n; if (n == "vrrp_instance") print "instance " a }',
|
||||
r'function pop() { if (0 < sd) { if (st[sd] == "vrrp_instance") print "end"; sd-- } }',
|
||||
r"function stmt(w, nw, f, d, x, i) { if (!nw) return;"
|
||||
r' if (w[1] == "include") { if (1 < nw) inc(w[2], f, d); return }'
|
||||
r' if (sd == 1 && st[1] == "vrrp_instance") {'
|
||||
r' if (1 < nw && (w[1] == "interface" || w[1] == "virtual_router_id")) print w[1] " " w[2];'
|
||||
r" return }"
|
||||
r' if (sd == 2 && st[1] == "vrrp_instance" && st[2] ~ /^virtual_ipaddress(_excluded)?$/) {'
|
||||
r' x = "vip " w[1]; for (i = 2; i < nw; i++) if (w[i] == "dev") x = x " dev " w[i + 1];'
|
||||
r" print x } }",
|
||||
r"function rk(f, d, l, r, n, t, k, w, nw) {"
|
||||
r' if (4 < d || (f in seen)) { print "too-deep " f; return }'
|
||||
r" if (!ok(f)) return; seen[f] = 1;"
|
||||
r' while (0 < (r = (getline l < f))) { sub(/[#!].*/, "", l); gsub(/[{}]/, " & ", l);'
|
||||
r' n = split(l, t, " "); if (n && nw && t[1] != "{") { stmt(w, nw, f, d); nw = 0 }'
|
||||
r" for (k = 1; k <= n; k++) {"
|
||||
r' if (t[k] == "{") { push(nw ? w[1] : "", 1 < nw ? w[2] : ""); nw = 0 }'
|
||||
r' else if (t[k] == "}") { stmt(w, nw, f, d); nw = 0; pop() } else w[++nw] = t[k] } }'
|
||||
r" stmt(w, nw, f, d);"
|
||||
r' if (r < 0) print "unreadable " f; close(f); delete seen[f] }',
|
||||
r"function inc(p, f, d, c, g, dir, fs, n, i) {"
|
||||
r' if (p !~ /^\//) { dir = f; sub(/[^\/]*$/, "", dir); p = dir p }'
|
||||
r' if (p !~ /^[A-Za-z0-9_.\/*?-]+$/) { print "unreadable " p; return }'
|
||||
r' c = "for g in " p "; do [ -e \"$g\" ] && echo \"$g\"; done"; n = 0;'
|
||||
r" while (0 < (c | getline g)) fs[++n] = g; close(c);"
|
||||
r" for (i = 1; i <= n; i++) rk(fs[i], d + 1) }",
|
||||
r'/^ActiveState=/ { print "active " substr($0, 13); next }',
|
||||
r'/^ExecStart=/ { s = $0; i = index(s, "argv[]="); if (!i) next; s = substr(s, i + 7);'
|
||||
r' j = index(s, " ;"); if (j) s = substr(s, 1, j - 1); n = split(s, t, " ");'
|
||||
r' for (k = 2; k <= n; k++) { f = t[k]; if (f !~ /^-/) continue; v = ""; x = index(f, "=");'
|
||||
r" if (x) { v = substr(f, x + 1); f = substr(f, 1, x - 1) }"
|
||||
r' else if (k < n && t[k + 1] !~ /^-/) v = t[++k]; sub(/^--?/, "", f);'
|
||||
r" if (f ~ /^(config|ip|netmask|mask|interface|iface|trigger-key|key|f|use-file)$/) {"
|
||||
r' print "arg " f " " v; a[f] = v } } next }',
|
||||
r'/^EnvironmentFiles=/ { s = substr($0, 18); sub(/ \(ignore_errors=[a-z]*\)$/, "", s);'
|
||||
r' if (s != "") ef[++ne] = s; next }',
|
||||
r'/^Environment=/ { n = split(substr($0, 13), t, " "); for (k = 1; k <= n; k++) {'
|
||||
rf' x = index(t[k], "="); if (x && substr(t[k], 1, x - 1) ~ /^{_ENV_KEYS}$/)'
|
||||
r' print "env " substr(t[k], 1, x - 1) " " substr(t[k], x + 1) } next }',
|
||||
rf'END {{ if (m == "keepalived") {{ p = "{_KEEPALIVED_CONF}";'
|
||||
r' if ("use-file" in a) p = a["use-file"]; if ("f" in a) p = a["f"]; rk(p, 0) }'
|
||||
r' else { for (k = 1; k <= ne; k++) rf(ef[k], "env");'
|
||||
r' if ("config" in a) rf(a["config"], "cfg") } }',
|
||||
)
|
||||
)
|
||||
|
||||
#: One line, POSIX ``sh``, read-only. The frame markers are printed in two halves
|
||||
#: so that a transport which echoes the command does not show them early.
|
||||
VIRTUAL_IPS_COMMAND = (
|
||||
"PATH=$PATH:/usr/sbin:/sbin; "
|
||||
"printf '%s%s\\n' VIPS_ BEGIN; "
|
||||
f"if command -v systemctl >/dev/null 2>&1 && [ -d {_SYSTEMD_RUN} ]; then echo '[systemd]'; "
|
||||
"for u in $(systemctl list-units --all --plain --no-legend --type=service "
|
||||
"vip-manager.service 'vip-manager@*' keepalived.service 2>/dev/null | awk '{print $1}'); do "
|
||||
"case $u in keepalived*) m=keepalived;; *) m=vip-manager;; esac; "
|
||||
'echo "[$m $u]"; '
|
||||
'systemctl show -p ActiveState -p ExecStart -p EnvironmentFiles -p Environment "$u" '
|
||||
f"2>/dev/null | awk -v m=$m {quote(_AWK)}; done; "
|
||||
"else echo '[no-systemd]'; fi; "
|
||||
"printf '%s%s\\n' VIPS_ END"
|
||||
)
|
||||
|
||||
_COMMENT_RE = re.compile(r"\s+#.*$")
|
||||
_VARIABLE_RE = re.compile(r"^\$\{?(\w+)\}?$")
|
||||
_Record = tuple[str, str]
|
||||
|
||||
|
||||
def _frame(output: str) -> list[str]:
|
||||
lines = [line.strip() for line in strip_terminal_codes(output).splitlines()]
|
||||
try:
|
||||
start = lines.index(_BEGIN)
|
||||
end = lines.index(_END, start)
|
||||
except ValueError:
|
||||
raise ValueError("no intact virtual IP report in the output") from None
|
||||
return lines[start + 1 : end]
|
||||
|
||||
|
||||
def _sections(lines: list[str]) -> list[tuple[str, str, list[_Record]]]:
|
||||
"""``[<mechanism> <unit>]`` headers, each with its records as ``(kind, rest)``."""
|
||||
sections: list[tuple[str, str, list[_Record]]] = []
|
||||
for line in lines:
|
||||
if line.startswith("[") and line.endswith("]"):
|
||||
mechanism, _, unit = line[1:-1].partition(" ")
|
||||
sections.append((mechanism, unit, []))
|
||||
elif line and sections:
|
||||
kind, _, rest = line.partition(" ")
|
||||
sections[-1][2].append((kind, rest.strip()))
|
||||
return sections
|
||||
|
||||
|
||||
def _clean(value: str) -> str:
|
||||
"""Without a trailing ``# comment`` and the quotes around it."""
|
||||
value = _COMMENT_RE.sub("", value.strip())
|
||||
if len(value) >= 2 and value[0] == value[-1] and value[0] in "\"'":
|
||||
value = value[1:-1]
|
||||
return value.strip()
|
||||
|
||||
|
||||
def _address(text: str) -> tuple[str, int | None] | None:
|
||||
"""``"10.0.0.10/24"`` -> ``("10.0.0.10", 24)``; None for what is no address."""
|
||||
host, slash, prefix = text.partition("/")
|
||||
try:
|
||||
address = ipaddress.ip_address(host)
|
||||
except ValueError:
|
||||
return None
|
||||
if not slash:
|
||||
return str(address), None
|
||||
if not prefix.isdigit() or int(prefix) > address.max_prefixlen:
|
||||
return None
|
||||
return str(address), int(prefix)
|
||||
|
||||
|
||||
def _prefix(mask: str | None) -> int | None:
|
||||
"""``"24"`` or ``"255.255.255.0"`` -> 24."""
|
||||
if not mask:
|
||||
return None
|
||||
if mask.isdigit():
|
||||
return int(mask) if int(mask) <= 128 else None
|
||||
try:
|
||||
return ipaddress.IPv4Network(f"0.0.0.0/{mask}").prefixlen
|
||||
except ValueError:
|
||||
return None
|
||||
|
||||
|
||||
def _running(records: list[_Record]) -> bool | None:
|
||||
states = [rest for kind, rest in records if kind == "active"]
|
||||
return states[-1] == "active" if states else None
|
||||
|
||||
|
||||
def _vip_manager(unit: str, records: list[_Record]) -> list[VirtualIpDict]:
|
||||
values: dict[str, dict[str, str]] = {"arg": {}, "env": {}, "cfg": {}}
|
||||
for kind, rest in records:
|
||||
if kind in values:
|
||||
key, _, value = rest.partition(" ")
|
||||
values[kind][key] = value
|
||||
env = values["env"]
|
||||
|
||||
def resolve(value: str) -> str | None:
|
||||
value = _clean(value)
|
||||
variable = _VARIABLE_RE.match(value)
|
||||
if variable:
|
||||
value = _clean(env.get(variable.group(1), ""))
|
||||
return value if value and "$" not in value else None
|
||||
|
||||
def pick(args: tuple[str, ...], envs: tuple[str, ...], cfgs: tuple[str, ...]) -> str | None:
|
||||
for kind, names in (("arg", args), ("env", envs), ("cfg", cfgs)):
|
||||
for name in names:
|
||||
if name in values[kind]:
|
||||
found = resolve(values[kind][name])
|
||||
if found:
|
||||
return found
|
||||
return None
|
||||
|
||||
ip = pick(("ip",), ("VIP_IP",), ("ip",))
|
||||
parsed = _address(ip) if ip else None
|
||||
if parsed is None:
|
||||
return []
|
||||
address, prefix = parsed
|
||||
mask = pick(("netmask", "mask"), ("VIP_NETMASK", "VIP_MASK"), ("netmask", "mask"))
|
||||
return [
|
||||
{
|
||||
"mechanism": "vip-manager",
|
||||
"instance": unit[: -len(".service")] if unit.endswith(".service") else unit,
|
||||
"address": address,
|
||||
"prefix_length": _prefix(mask) if mask else prefix,
|
||||
"interface": pick(
|
||||
("interface", "iface"), ("VIP_INTERFACE", "VIP_IFACE"), ("interface", "iface")
|
||||
),
|
||||
"group_key": pick(
|
||||
("trigger-key", "key"), ("VIP_TRIGGER_KEY", "VIP_KEY"), ("trigger-key", "key")
|
||||
),
|
||||
"running": _running(records),
|
||||
}
|
||||
]
|
||||
|
||||
|
||||
@dataclass
|
||||
class _Instance:
|
||||
name: str
|
||||
interface: str | None = None
|
||||
vrid: str = ""
|
||||
vips: list[str] = field(default_factory=list)
|
||||
|
||||
|
||||
def _keepalived(unit: str, records: list[_Record]) -> list[VirtualIpDict]:
|
||||
running = _running(records)
|
||||
instances: list[_Instance] = []
|
||||
current: _Instance | None = None
|
||||
for kind, rest in records:
|
||||
if kind == "instance":
|
||||
current = _Instance(rest or unit)
|
||||
instances.append(current)
|
||||
elif current is None:
|
||||
continue
|
||||
elif kind == "interface":
|
||||
current.interface = rest or None
|
||||
elif kind == "virtual_router_id":
|
||||
current.vrid = rest
|
||||
elif kind == "vip":
|
||||
current.vips.append(rest)
|
||||
elif kind == "end":
|
||||
current = None
|
||||
return [entry for instance in instances for entry in _instance_entries(instance, running)]
|
||||
|
||||
|
||||
def _instance_entries(instance: _Instance, running: bool | None) -> list[VirtualIpDict]:
|
||||
entries: list[VirtualIpDict] = []
|
||||
for vip in instance.vips:
|
||||
tokens = vip.split()
|
||||
parsed = _address(tokens[0]) if tokens else None
|
||||
if parsed is None:
|
||||
continue
|
||||
address, prefix = parsed
|
||||
dev = tokens[tokens.index("dev") + 1] if "dev" in tokens[1:-1] else None
|
||||
entries.append(
|
||||
{
|
||||
"mechanism": "keepalived",
|
||||
"instance": instance.name,
|
||||
"address": address,
|
||||
"prefix_length": prefix if prefix is not None else _host_prefix(address),
|
||||
"interface": dev or instance.interface,
|
||||
"group_key": f"vrid:{instance.vrid}" if instance.vrid.isdigit() else None,
|
||||
"running": running,
|
||||
}
|
||||
)
|
||||
return entries
|
||||
|
||||
|
||||
def _host_prefix(address: str) -> int:
|
||||
"""keepalived's default for an entry without one: the single address."""
|
||||
return ipaddress.ip_address(address).max_prefixlen
|
||||
|
||||
|
||||
_READERS: dict[str, tuple[str, Callable[[str, list[_Record]], list[VirtualIpDict]]]] = {
|
||||
"vip-manager": ("vip_manager", _vip_manager),
|
||||
"keepalived": ("keepalived", _keepalived),
|
||||
}
|
||||
|
||||
|
||||
def _order(entry: VirtualIpDict) -> tuple[int, int, str]:
|
||||
address = ipaddress.ip_address(entry["address"])
|
||||
return address.version, int(address), entry["instance"]
|
||||
|
||||
|
||||
def parse_virtual_ips(output: str) -> VirtualIpsDict:
|
||||
"""Parse what :data:`VIRTUAL_IPS_COMMAND` printed, each mechanism's entries
|
||||
sorted by address.
|
||||
|
||||
:raises ValueError: when the output carries no intact report.
|
||||
"""
|
||||
sections = _sections(_frame(output))
|
||||
if any(mechanism == _NO_SYSTEMD for mechanism, _, _ in sections):
|
||||
return {"vip_manager": None, "keepalived": None}
|
||||
found: dict[str, list[VirtualIpDict]] = {"vip_manager": [], "keepalived": []}
|
||||
blind: set[str] = set()
|
||||
for mechanism, unit, records in sections:
|
||||
if mechanism not in _READERS:
|
||||
continue
|
||||
key, read = _READERS[mechanism]
|
||||
if any(kind in _BLIND for kind, _ in records):
|
||||
blind.add(key)
|
||||
else:
|
||||
found[key].extend(read(unit, records))
|
||||
|
||||
def result(key: str) -> list[VirtualIpDict] | None:
|
||||
return None if key in blind else sorted(found[key], key=_order)
|
||||
|
||||
return {"vip_manager": result("vip_manager"), "keepalived": result("keepalived")}
|
||||
|
||||
|
||||
class VirtualIpsMixin:
|
||||
"""Adds :meth:`get_virtual_ips` to a driver that can run a command on a Linux host.
|
||||
|
||||
The template form (README, "Function classes"): the command and its parse are
|
||||
the same everywhere, so they are concrete here, and a driver supplies only
|
||||
:meth:`_run_virtual_ips_command` -- how a command reaches its host, and how it
|
||||
gains root there. Mixed in by the drivers that can, not by
|
||||
:class:`~napalm_device_types.os.OSDriver`, so ``hasattr(driver,
|
||||
"get_virtual_ips")`` stays a truthful answer.
|
||||
"""
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
|
||||
|
||||
def _run_virtual_ips_command(self, command: str, *, privileged: bool) -> str:
|
||||
"""Run *command* on the host and return what it printed; as root
|
||||
when *privileged*. The command is a single ``sh -c`` invocation of
|
||||
about 4 kB: send it on an exec channel, not typed into a shell."""
|
||||
...
|
||||
|
||||
def get_virtual_ips(self) -> VirtualIpsDict:
|
||||
"""
|
||||
Returns the virtual IPs the host's vip-manager and keepalived configuration
|
||||
declares -- whether or not the host holds them at the moment.
|
||||
|
||||
* vip_manager (list or None) - see :class:`~napalm_device_types.models.VirtualIpDict`
|
||||
* keepalived (list or None)
|
||||
|
||||
None for a mechanism means it could not be read; ``[]`` that the host has none.
|
||||
|
||||
Example::
|
||||
|
||||
{
|
||||
"vip_manager": [
|
||||
{"mechanism": "vip-manager", "instance": "vip-manager",
|
||||
"address": "10.7.224.10", "prefix_length": 24, "interface": "ens7",
|
||||
"group_key": "/service/netork-db/leader", "running": True},
|
||||
],
|
||||
"keepalived": [],
|
||||
}
|
||||
|
||||
:raises ValueError: if neither reading carried an intact report.
|
||||
"""
|
||||
command = f"sh -c {quote(VIRTUAL_IPS_COMMAND)}"
|
||||
try:
|
||||
return parse_virtual_ips(self._run_virtual_ips_command(command, privileged=True))
|
||||
except ValueError:
|
||||
pass
|
||||
return parse_virtual_ips(self._run_virtual_ips_command(command, privileged=False))
|
||||
@@ -0,0 +1,26 @@
|
||||
"""Moving a guest to another node of its cluster (``HypervisorDriver.migrate_vm``).
|
||||
|
||||
``get_vm_migration_preflight`` reports what the platform says before a move:
|
||||
which nodes would take the guest, which refuse it and why, and what the guest
|
||||
holds on its current node. ``migrate_vm`` then moves it and blocks until the
|
||||
platform's task ended; a failure raises with the platform's own cause.
|
||||
``set_node_maintenance`` switches a node's HA maintenance mode, which makes the
|
||||
HA manager move its HA guests away while it is on.
|
||||
|
||||
Facts only: which node to pick, and whether a local disk is worth copying, is
|
||||
the caller's to decide.
|
||||
"""
|
||||
|
||||
from typing import TypedDict
|
||||
|
||||
|
||||
class MigrationPreflightDict(TypedDict):
|
||||
"""Where a guest could move to, and what stands in the way."""
|
||||
|
||||
running: bool
|
||||
allowed_nodes: list[str] # nodes the platform would move it to
|
||||
refused_nodes: dict[str, str] # node -> the platform's reason, in plain words
|
||||
# volume ids on storage the target does not share; they block nothing by
|
||||
# themselves, the migration copies them
|
||||
local_disks: list[str]
|
||||
local_resources: list[str] # e.g. "hostpci0", "usb0": they block the migration
|
||||
@@ -0,0 +1,29 @@
|
||||
"""Asking for a VM's clean shutdown without waiting for it.
|
||||
|
||||
``HypervisorDriver.stop_vm`` blocks until the VM is stopped, within a limit
|
||||
of the driver's own. A caller that keeps its waits as timers asks with
|
||||
``begin_stop_vm`` and reads ``get_vm_status`` until the VM says it stopped.
|
||||
|
||||
Declared under ``if TYPE_CHECKING``: a driver mixes this in when its
|
||||
hypervisor can take the request on its own; ``hasattr`` tells a caller
|
||||
whether it did. No role base carries it.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
|
||||
class VmStopRequestMixin:
|
||||
if TYPE_CHECKING:
|
||||
|
||||
def begin_stop_vm(self, name: str) -> None:
|
||||
"""
|
||||
Asks the hypervisor to shut a virtual machine down cleanly (ACPI)
|
||||
and returns as soon as the request was accepted.
|
||||
|
||||
:param name: VM name or its ``vmid``.
|
||||
:raises ValueError: If no VM with the given name/ID exists.
|
||||
:raises RuntimeError: If the hypervisor refuses the request.
|
||||
"""
|
||||
...
|
||||
+39
-16
@@ -1,14 +1,14 @@
|
||||
[build-system]
|
||||
requires = ["setuptools>=68", "wheel"]
|
||||
requires = ["setuptools>=77"]
|
||||
build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "napalm-device-types"
|
||||
version = "2.0.0"
|
||||
version = "4.29.0"
|
||||
description = "Abstract device-type base classes for NAPALM drivers"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.10"
|
||||
license = { text = "Apache-2.0" }
|
||||
requires-python = ">=3.12"
|
||||
license = "Apache-2.0"
|
||||
authors = [
|
||||
{ name = "Christian Manivong", email = "christian@manivong.de" },
|
||||
]
|
||||
@@ -28,38 +28,61 @@ classifiers = [
|
||||
"Development Status :: 3 - Alpha",
|
||||
"Intended Audience :: Developers",
|
||||
"Intended Audience :: System Administrators",
|
||||
"License :: OSI Approved :: Apache Software License",
|
||||
"Operating System :: OS Independent",
|
||||
"Programming Language :: Python :: 3",
|
||||
"Programming Language :: Python :: 3.10",
|
||||
"Programming Language :: Python :: 3.11",
|
||||
"Programming Language :: Python :: 3.12",
|
||||
"Programming Language :: Python :: 3.13",
|
||||
"Topic :: System :: Networking",
|
||||
"Topic :: System :: Systems Administration",
|
||||
"Typing :: Typed",
|
||||
]
|
||||
dependencies = [
|
||||
"napalm>=4.0",
|
||||
"napalm>=5.1", # older releases import pkg_resources, which a 3.12 venv lacks
|
||||
"requests>=2.25.0",
|
||||
"typing-extensions>=4.0",
|
||||
]
|
||||
|
||||
[project.urls]
|
||||
Homepage = "https://github.com/chrismanivong/napalm-device-types"
|
||||
Repository = "https://github.com/chrismanivong/napalm-device-types"
|
||||
Issues = "https://github.com/chrismanivong/napalm-device-types/issues"
|
||||
Repository = "https://git.netork.io/NAPALM/napalm-device-types"
|
||||
Issues = "https://git.netork.io/NAPALM/napalm-device-types/issues"
|
||||
Changelog = "https://git.netork.io/NAPALM/napalm-device-types/src/branch/main/CHANGELOG.md"
|
||||
|
||||
[project.optional-dependencies]
|
||||
dev = [
|
||||
"pytest",
|
||||
"mypy",
|
||||
"build",
|
||||
"twine",
|
||||
"pytest>=8",
|
||||
"pytest-cov>=5",
|
||||
"mypy>=1.11",
|
||||
"ruff>=0.6",
|
||||
]
|
||||
|
||||
[tool.setuptools.packages.find]
|
||||
where = ["."]
|
||||
include = ["napalm_device_types*"]
|
||||
|
||||
[tool.setuptools.package-data]
|
||||
napalm_device_types = ["py.typed"]
|
||||
|
||||
[tool.ruff]
|
||||
line-length = 100
|
||||
target-version = "py312"
|
||||
|
||||
[tool.ruff.lint]
|
||||
select = ["E", "F", "I", "UP", "B"]
|
||||
|
||||
[tool.ruff.lint.per-file-ignores]
|
||||
# Recorded device output keeps its lines.
|
||||
"tests/**" = ["E501"]
|
||||
|
||||
[tool.mypy]
|
||||
python_version = "3.10"
|
||||
python_version = "3.12"
|
||||
strict = true
|
||||
ignore_missing_imports = true
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
testpaths = ["tests"]
|
||||
|
||||
[tool.coverage.run]
|
||||
source = ["napalm_device_types"]
|
||||
|
||||
[tool.coverage.report]
|
||||
fail_under = 95
|
||||
@@ -0,0 +1,38 @@
|
||||
{
|
||||
"coverage_floor": 95.0,
|
||||
"function_complexity": {
|
||||
"napalm_device_types/_ucd_metrics.py::build_if_metrics": 20,
|
||||
"napalm_device_types/_ucd_metrics.py::collect_ucd_metrics": 17,
|
||||
"napalm_device_types/host_firewall.py::_iptables_rule": 11,
|
||||
"napalm_device_types/host_firewall.py::_module_match": 13,
|
||||
"napalm_device_types/host_firewall.py::_module_options": 14,
|
||||
"napalm_device_types/host_firewall.py::_nft_field": 15,
|
||||
"napalm_device_types/host_firewall.py::_nft_match": 12,
|
||||
"napalm_device_types/host_firewall.py::_nft_rule": 16,
|
||||
"napalm_device_types/host_firewall.py::_parse_nft": 14,
|
||||
"napalm_device_types/host_firewall.py::rule_text": 14,
|
||||
"napalm_device_types/host_status.py::_reboot": 12,
|
||||
"napalm_device_types/lag.py::add_lag_interfaces": 11,
|
||||
"napalm_device_types/listening.py::parse_listening_sockets": 13,
|
||||
"napalm_device_types/service_clusters.py::_patroni": 11,
|
||||
"napalm_device_types/ssh_server.py::plan_ssh_hardening": 38,
|
||||
"napalm_device_types/virtual_ips.py::_keepalived": 12
|
||||
},
|
||||
"function_lines": {
|
||||
"napalm_device_types/_ucd_metrics.py::collect_ucd_metrics": 119,
|
||||
"napalm_device_types/dhcp.py::DhcpServerMixin.diff_dhcp_reservations": 54,
|
||||
"napalm_device_types/dhcp.py::DhcpServerMixin.diff_dhcp_subnets": 60,
|
||||
"napalm_device_types/firewall_rules.py::FirewallRuleMixin.diff_firewall_rules": 56,
|
||||
"napalm_device_types/hypervisor.py::HypervisorDriver.create_vm_from_cloud_init": 84,
|
||||
"napalm_device_types/hypervisor.py::HypervisorDriver.get_vm_config": 77,
|
||||
"napalm_device_types/ssh_server.py::plan_ssh_hardening": 60,
|
||||
"napalm_device_types/storage.py::StorageDriver.get_volumes": 52,
|
||||
"napalm_device_types/switch.py::SwitchDriver.get_dot1x_ports": 51,
|
||||
"napalm_device_types/switch.py::SwitchDriver.get_poe_status": 63
|
||||
},
|
||||
"mypy_relaxed": [],
|
||||
"python_file_lines": {
|
||||
"napalm_device_types/models.py": 1454
|
||||
},
|
||||
"untyped_public": []
|
||||
}
|
||||
@@ -0,0 +1,826 @@
|
||||
"""apt's proxy: what apt fetches through, whether a proxy answers, pointing apt at one.
|
||||
|
||||
apt reads every file in ``apt.conf.d`` in order and ``apt.conf`` last; the last
|
||||
value wins. The Debian installer writes its proxy into ``apt.conf``, Ubuntu's
|
||||
into ``apt.conf.d/90curtin-aptproxy``: reading one file was not enough
|
||||
(NetOrk/netork#1397). The fetch outputs are apt-helper's own, from a Mint 22.3
|
||||
host against netOrk's test server.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types.apt_proxy import (
|
||||
APT_PROXY_COMMAND,
|
||||
AptProxyMixin,
|
||||
apt_fetch_command,
|
||||
comment_out_command,
|
||||
https_proxy_for,
|
||||
parse_apt_fetch,
|
||||
parse_apt_proxy,
|
||||
same_proxy_url,
|
||||
)
|
||||
from napalm_device_types.fixes import PackageProxyMixin
|
||||
|
||||
|
||||
def _report(dump=(), parts=(), main=(), sources=()):
|
||||
lines = ["APROXY_BEGIN", "[dump]", *dump, "[parts]", *parts, "[main]", *main]
|
||||
return "\n".join([*lines, "[sources]", *sources, "APROXY_END"])
|
||||
|
||||
|
||||
_D = "/etc/apt/apt.conf.d/"
|
||||
|
||||
|
||||
class TestEffectiveProxy:
|
||||
def test_the_proxy_apt_fetches_through(self):
|
||||
report = parse_apt_proxy(
|
||||
_report(
|
||||
dump=['Acquire::http::Proxy "http://c:3142";'],
|
||||
parts=[f'{_D}00proxy:1:Acquire::http::Proxy "http://c:3142";'],
|
||||
)
|
||||
)
|
||||
|
||||
assert report["proxy"] == "http://c:3142"
|
||||
assert report["settings"] == [
|
||||
{"file": f"{_D}00proxy", "line": 1, "value": "http://c:3142", "editable": True}
|
||||
]
|
||||
|
||||
def test_none_without_a_proxy(self):
|
||||
report = parse_apt_proxy(_report())
|
||||
|
||||
assert report["proxy"] is None
|
||||
assert report["settings"] == []
|
||||
assert report["per_host"] == {}
|
||||
assert report["auto_detect"] is None
|
||||
|
||||
@pytest.mark.parametrize("value", ["", "DIRECT", "direct"])
|
||||
def test_an_empty_or_direct_proxy_is_none(self, value):
|
||||
assert parse_apt_proxy(_report(dump=[f'Acquire::http::Proxy "{value}";']))["proxy"] is None
|
||||
|
||||
def test_keys_are_case_insensitive(self):
|
||||
report = parse_apt_proxy(_report(dump=['acquire::HTTP::proxy "http://c:3142";']))
|
||||
assert report["proxy"] == "http://c:3142"
|
||||
|
||||
def test_per_host_overrides_and_auto_detect(self):
|
||||
report = parse_apt_proxy(
|
||||
_report(
|
||||
dump=[
|
||||
'Acquire::http::Proxy "http://c:3142";',
|
||||
'Acquire::http::Proxy::deb.debian.org "DIRECT";',
|
||||
'Acquire::http::Proxy::ppa.launchpad.net "http://other:3128";',
|
||||
'Acquire::http::Proxy-Auto-Detect "/usr/bin/auto-apt-proxy";',
|
||||
]
|
||||
)
|
||||
)
|
||||
|
||||
assert report["proxy"] == "http://c:3142"
|
||||
assert report["per_host"] == {
|
||||
"deb.debian.org": "DIRECT",
|
||||
"ppa.launchpad.net": "http://other:3128",
|
||||
}
|
||||
assert report["auto_detect"] == "/usr/bin/auto-apt-proxy"
|
||||
|
||||
def test_no_intact_report_raises(self):
|
||||
with pytest.raises(ValueError):
|
||||
parse_apt_proxy("APROXY_BEGIN\n[dump]\n")
|
||||
|
||||
|
||||
class TestWhereItIsSet:
|
||||
def test_every_setting_in_apt_s_reading_order(self):
|
||||
report = parse_apt_proxy(
|
||||
_report(
|
||||
dump=['Acquire::http::Proxy "http://old:3128";'],
|
||||
parts=[
|
||||
f'{_D}00proxy:1:Acquire::http::Proxy "http://c:3142";',
|
||||
f'{_D}90curtin-aptproxy:1:Acquire::http::Proxy "http://old:3128";',
|
||||
],
|
||||
main=['/etc/apt/apt.conf:4: Acquire::http::Proxy "http://older:8080";'],
|
||||
)
|
||||
)
|
||||
|
||||
assert [(s["file"], s["line"], s["value"]) for s in report["settings"]] == [
|
||||
(f"{_D}00proxy", 1, "http://c:3142"),
|
||||
(f"{_D}90curtin-aptproxy", 1, "http://old:3128"),
|
||||
("/etc/apt/apt.conf", 4, "http://older:8080"),
|
||||
]
|
||||
|
||||
def test_a_nested_setting_is_named_but_not_editable(self):
|
||||
report = parse_apt_proxy(_report(parts=[f'{_D}01proxy:2: Proxy "http://nested:3142";']))
|
||||
|
||||
assert report["settings"] == [
|
||||
{"file": f"{_D}01proxy", "line": 2, "value": "http://nested:3142", "editable": False}
|
||||
]
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"line",
|
||||
[
|
||||
'// Acquire::http::Proxy "http://c:3142";',
|
||||
'# Acquire::http::Proxy "http://c:3142";',
|
||||
'Acquire::https::Proxy "http://c:3142";',
|
||||
'Acquire::ftp::Proxy "http://c:3142";',
|
||||
'Acquire::http::Proxy::deb.debian.org "DIRECT";',
|
||||
'Acquire::http::Proxy-Auto-Detect "/usr/bin/x";',
|
||||
"Acquire::http {",
|
||||
],
|
||||
)
|
||||
def test_what_does_not_set_the_http_proxy_is_left_out(self, line):
|
||||
assert parse_apt_proxy(_report(parts=[f"{_D}00x:1:{line}"]))["settings"] == []
|
||||
|
||||
@pytest.mark.parametrize("name", ["00proxy.bak", "00proxy.dpkg-old", "00proxy.save", "a b"])
|
||||
def test_files_apt_does_not_read_are_left_out(self, name):
|
||||
line = f'{_D}{name}:1:Acquire::http::Proxy "http://c:3142";'
|
||||
assert parse_apt_proxy(_report(parts=[line]))["settings"] == []
|
||||
|
||||
def test_a_conf_extension_is_read(self):
|
||||
line = f'{_D}01proxy.conf:1:Acquire::http::Proxy "http://c:3142";'
|
||||
assert len(parse_apt_proxy(_report(parts=[line]))["settings"]) == 1
|
||||
|
||||
|
||||
class TestSources:
|
||||
def test_sources_by_scheme(self):
|
||||
report = parse_apt_proxy(
|
||||
_report(
|
||||
sources=[
|
||||
"http://archive.ubuntu.com/ubuntu/dists/noble/",
|
||||
"https://download.docker.com/linux/ubuntu/dists/noble/",
|
||||
]
|
||||
)
|
||||
)
|
||||
|
||||
assert report["http_sources"] == ["http://archive.ubuntu.com/ubuntu/dists/noble/"]
|
||||
assert report["https_sources"] == ["https://download.docker.com/linux/ubuntu/dists/noble/"]
|
||||
|
||||
|
||||
class TestHttpsProxy:
|
||||
"""apt's https method asks ``Acquire::https::Proxy`` and then ``Acquire::http::Proxy``:
|
||||
an http proxy alone takes the https sources too (NetOrk/netork#1811). Measured with
|
||||
apt 2.8.3's apt-helper against a dead port."""
|
||||
|
||||
def test_the_http_proxy_alone_takes_https_too(self):
|
||||
report = parse_apt_proxy(_report(dump=['Acquire::http::Proxy "http://c:3142";']))
|
||||
assert report["https_proxy"] == "http://c:3142"
|
||||
|
||||
def test_an_https_proxy_of_its_own_wins(self):
|
||||
report = parse_apt_proxy(
|
||||
_report(
|
||||
dump=[
|
||||
'Acquire::http::Proxy "http://c:3142";',
|
||||
'Acquire::https::Proxy "http://other:3128";',
|
||||
]
|
||||
)
|
||||
)
|
||||
|
||||
assert (report["proxy"], report["https_proxy"]) == ("http://c:3142", "http://other:3128")
|
||||
|
||||
@pytest.mark.parametrize("value", ["DIRECT", "direct"])
|
||||
def test_direct_https_is_none(self, value):
|
||||
report = parse_apt_proxy(
|
||||
_report(
|
||||
dump=['Acquire::http::Proxy "http://c:3142";', f'Acquire::https::Proxy "{value}";']
|
||||
)
|
||||
)
|
||||
assert (report["proxy"], report["https_proxy"]) == ("http://c:3142", None)
|
||||
|
||||
def test_an_empty_https_proxy_is_not_set(self):
|
||||
"""apt takes an empty value for unset and goes on to the http one."""
|
||||
report = parse_apt_proxy(
|
||||
_report(dump=['Acquire::http::Proxy "http://c:3142";', 'Acquire::https::Proxy "";'])
|
||||
)
|
||||
assert report["https_proxy"] == "http://c:3142"
|
||||
|
||||
def test_per_host_https_then_http(self):
|
||||
report = parse_apt_proxy(
|
||||
_report(
|
||||
dump=[
|
||||
'Acquire::http::Proxy::a.example "DIRECT";',
|
||||
'Acquire::http::Proxy::b.example "DIRECT";',
|
||||
'Acquire::https::Proxy::b.example "http://c:3142";',
|
||||
]
|
||||
)
|
||||
)
|
||||
|
||||
assert report["https_per_host"] == {"a.example": "DIRECT", "b.example": "http://c:3142"}
|
||||
assert report["per_host"] == {"a.example": "DIRECT", "b.example": "DIRECT"}
|
||||
|
||||
def test_where_the_https_proxy_is_set(self):
|
||||
report = parse_apt_proxy(
|
||||
_report(
|
||||
parts=[
|
||||
f'{_D}00proxy:1:Acquire::http::Proxy "http://c:3142";',
|
||||
f'{_D}90curtin-aptproxy:2:Acquire::https::Proxy "http://c:3142";',
|
||||
f'{_D}91x:1:Acquire::https {{ Proxy "http://n:1"; }};',
|
||||
f'{_D}92x:1:// Acquire::https::Proxy "http://c:3142";',
|
||||
f'{_D}93x:1:Acquire::https::Proxy::h.example "DIRECT";',
|
||||
]
|
||||
)
|
||||
)
|
||||
|
||||
assert report["https_settings"] == [
|
||||
{
|
||||
"file": f"{_D}90curtin-aptproxy",
|
||||
"line": 2,
|
||||
"value": "http://c:3142",
|
||||
"editable": True,
|
||||
},
|
||||
{"file": f"{_D}91x", "line": 1, "value": "http://n:1", "editable": False},
|
||||
]
|
||||
assert [s["file"] for s in report["settings"]] == [f"{_D}00proxy"]
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"host, proxy",
|
||||
[
|
||||
("deb.example", "http://c:3142"),
|
||||
("direct.example", None),
|
||||
("Other.Example", "http://o:1"),
|
||||
],
|
||||
)
|
||||
def test_the_proxy_for_one_host(self, host, proxy):
|
||||
report = parse_apt_proxy(
|
||||
_report(
|
||||
dump=[
|
||||
'Acquire::http::Proxy "http://c:3142";',
|
||||
'Acquire::https::Proxy::direct.example "DIRECT";',
|
||||
'Acquire::http::Proxy::other.example "http://o:1";',
|
||||
]
|
||||
)
|
||||
)
|
||||
assert https_proxy_for(report, host) == proxy
|
||||
|
||||
|
||||
class TestSameProxyUrl:
|
||||
@pytest.mark.parametrize(
|
||||
"a, b",
|
||||
[
|
||||
("http://c:3142", "http://c:3142/"),
|
||||
("http://C:3142", "http://c:3142"),
|
||||
("http://c", "http://c:80"),
|
||||
],
|
||||
)
|
||||
def test_the_same(self, a, b):
|
||||
assert same_proxy_url(a, b)
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"a, b",
|
||||
[
|
||||
("http://c:3142", "http://c:31420"),
|
||||
("http://c:3142", "http://c:3142.evil"),
|
||||
("http://c:3142", "https://c:3142"),
|
||||
("http://c:3142", ""),
|
||||
("http://c:3142", None),
|
||||
],
|
||||
)
|
||||
def test_not_the_same(self, a, b):
|
||||
assert not same_proxy_url(a, b)
|
||||
|
||||
|
||||
_REFUSED = """Err:1 http://172.22.8.50:3143/acng-report.html
|
||||
Could not connect to 172.22.8.50:3143 (172.22.8.50). - connect (111: Connection refused)
|
||||
E: Failed to fetch http://172.22.8.50:3143/acng-report.html Could not connect to 172.22.8.50:3143 (172.22.8.50). - connect (111: Connection refused)
|
||||
E: Download Failed"""
|
||||
|
||||
_HTTP_ERROR = """Err:1 http://172.22.8.50:3142/nothing-here.html
|
||||
503 Host not found [IP: 172.22.8.50 3142]
|
||||
E: Failed to fetch http://172.22.8.50:3142/nothing-here.html 503 Host not found [IP: 172.22.8.50 3142]
|
||||
E: Download Failed"""
|
||||
|
||||
_UNRESOLVED = """Err:1 http://no-such-host.invalid:3142/acng-report.html
|
||||
Could not resolve 'no-such-host.invalid'
|
||||
E: Failed to fetch http://no-such-host.invalid:3142/acng-report.html Could not resolve 'no-such-host.invalid'
|
||||
E: Download Failed"""
|
||||
|
||||
_FETCHED = """Get:1 http://172.22.8.50:3142/acng-report.html
|
||||
Fetched 14.0 kB in 0s (0 B/s)"""
|
||||
|
||||
#: apt-cacher-ng without a PassThroughPattern, as apt 2.8.3 tells it (NetOrk/netork#1811).
|
||||
_CONNECT_DENIED = """Err:1 https://download.docker.com/linux/ubuntu/dists/noble/InRelease
|
||||
Invalid response from proxy: HTTP/1.0 403 CONNECT denied (ask the admin to allow HTTPS tunnels) [IP: 10.40.100.10 3142]
|
||||
E: Failed to fetch https://download.docker.com/linux/ubuntu/dists/noble/InRelease Invalid response from proxy: HTTP/1.0 403 CONNECT denied (ask the admin to allow HTTPS tunnels) [IP: 10.40.100.10 3142]
|
||||
E: Download Failed"""
|
||||
|
||||
|
||||
def _fetch(output, rc, match=None):
|
||||
lines = ["APROXY_BEGIN", "[fetch]", *output.splitlines(), "[rc]", str(rc)]
|
||||
if match is not None:
|
||||
lines += ["[match]", str(match)]
|
||||
return "\n".join([*lines, "APROXY_END"])
|
||||
|
||||
|
||||
class TestFetch:
|
||||
def test_fetched_and_expected(self):
|
||||
assert parse_apt_fetch(_fetch(_FETCHED, 0, match=4)) == {
|
||||
"reachable": True,
|
||||
"expected": True,
|
||||
"error": "",
|
||||
}
|
||||
|
||||
def test_fetched_but_not_the_expected_page(self):
|
||||
assert parse_apt_fetch(_fetch(_FETCHED, 0, match=0))["expected"] is False
|
||||
|
||||
def test_without_an_expectation_expected_is_none(self):
|
||||
assert parse_apt_fetch(_fetch(_FETCHED, 0))["expected"] is None
|
||||
|
||||
def test_refused(self):
|
||||
result = parse_apt_fetch(_fetch(_REFUSED, 100))
|
||||
|
||||
assert result["reachable"] is False
|
||||
assert result["error"] == (
|
||||
"Could not connect to 172.22.8.50:3143 (172.22.8.50). - connect (111: Connection refused)"
|
||||
)
|
||||
|
||||
def test_unresolved(self):
|
||||
result = parse_apt_fetch(_fetch(_UNRESOLVED, 100))
|
||||
assert (result["reachable"], result["error"]) == (
|
||||
False,
|
||||
"Could not resolve 'no-such-host.invalid'",
|
||||
)
|
||||
|
||||
def test_an_http_error_was_an_answer(self):
|
||||
result = parse_apt_fetch(_fetch(_HTTP_ERROR, 100))
|
||||
|
||||
assert result["reachable"] is True
|
||||
assert result["expected"] is False
|
||||
assert result["error"] == "503 Host not found [IP: 172.22.8.50 3142]"
|
||||
|
||||
def test_a_refused_tunnel(self):
|
||||
result = parse_apt_fetch(_fetch(_CONNECT_DENIED, 100))
|
||||
|
||||
assert result["reachable"] is False
|
||||
assert "403 CONNECT denied" in result["error"]
|
||||
|
||||
def test_without_apt_helper_nothing_could_be_tried(self):
|
||||
with pytest.raises(ValueError, match="apt-helper"):
|
||||
parse_apt_fetch(_fetch("sh: 1: /usr/lib/apt/apt-helper: not found", 127))
|
||||
|
||||
def test_no_intact_report_raises(self):
|
||||
with pytest.raises(ValueError):
|
||||
parse_apt_fetch("APROXY_BEGIN\n[fetch]\n")
|
||||
|
||||
|
||||
class TestFetchCommand:
|
||||
def test_past_the_proxy_to_the_url_itself(self):
|
||||
command = apt_fetch_command(
|
||||
"http://c:3142/acng-report.html", direct=True, expect="Cache NG"
|
||||
)
|
||||
|
||||
assert "'Acquire::http::Proxy::c=DIRECT'" in command
|
||||
assert "grep -c -F -- 'Cache NG'" in command
|
||||
|
||||
def test_through_apt_s_own_proxy(self):
|
||||
command = apt_fetch_command("http://archive.ubuntu.com/ubuntu/dists/noble/InRelease")
|
||||
assert "DIRECT" not in command
|
||||
|
||||
def test_an_https_source_through_apt_s_own_proxy(self):
|
||||
command = apt_fetch_command(
|
||||
"https://download.docker.com/linux/ubuntu/dists/noble/InRelease"
|
||||
)
|
||||
assert "DIRECT" not in command
|
||||
assert "download-file https://download.docker.com/" in command
|
||||
|
||||
def test_past_the_proxy_to_an_https_host(self):
|
||||
"""For https the host's own https setting comes first; the http one alone
|
||||
would lose to an https proxy for that host."""
|
||||
command = apt_fetch_command("https://h.example/x", direct=True)
|
||||
|
||||
assert "'Acquire::https::Proxy::h.example=DIRECT'" in command
|
||||
assert "'Acquire::http::Proxy::h.example=DIRECT'" in command
|
||||
|
||||
@pytest.mark.parametrize("url", ["http://c:3142/x'; rm -rf /", "ftp://c/x", "c:3142"])
|
||||
def test_a_url_that_would_break_the_command_is_refused(self, url):
|
||||
with pytest.raises(ValueError):
|
||||
apt_fetch_command(url)
|
||||
|
||||
|
||||
def test_the_commands_are_valid_posix_sh():
|
||||
for command in (
|
||||
APT_PROXY_COMMAND,
|
||||
apt_fetch_command("http://c:3142/x", direct=True, expect="y"),
|
||||
apt_fetch_command("https://h.example/dists/x/InRelease"),
|
||||
):
|
||||
subprocess.run(["sh", "-n", "-c", command], check=True)
|
||||
|
||||
|
||||
@pytest.mark.skipif(not os.path.exists("/usr/bin/apt-config"), reason="needs an apt host")
|
||||
def test_the_report_command_on_this_host():
|
||||
output = subprocess.run(["sh", "-c", APT_PROXY_COMMAND], capture_output=True, text=True).stdout
|
||||
report = parse_apt_proxy(output)
|
||||
assert set(report) >= {"proxy", "https_proxy", "settings", "https_settings", "https_sources"}
|
||||
|
||||
|
||||
_DOCKER = "https://download.docker.com/linux/ubuntu/dists/noble/"
|
||||
|
||||
|
||||
class _Host(AptProxyMixin):
|
||||
"""A Debian host: its apt files, and what apt-config would make of them.
|
||||
|
||||
*https_sources* are its sources over https. *tunnel* is whether the proxy lets
|
||||
them through; *direct_rc* is how a fetch of one goes that takes no proxy.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self, files=None, fetch_rc=0, apt=True, https_sources=(), tunnel=True, direct_rc=0
|
||||
):
|
||||
self.files = dict(files or {})
|
||||
self.fetch_rc = fetch_rc
|
||||
self.apt = apt
|
||||
self.https_sources = list(https_sources)
|
||||
self.tunnel = tunnel
|
||||
self.direct_rc = direct_rc
|
||||
self.root = []
|
||||
self.fetched = []
|
||||
|
||||
def _is_apt_host(self):
|
||||
return self.apt
|
||||
|
||||
def _settings(self):
|
||||
out = []
|
||||
for path in sorted(p for p in self.files if p.startswith(_D)) + [
|
||||
p for p in self.files if p == "/etc/apt/apt.conf"
|
||||
]:
|
||||
for n, text in enumerate(self.files[path].splitlines(), 1):
|
||||
if "proxy" in text.lower():
|
||||
out.append((path, n, text))
|
||||
return out
|
||||
|
||||
def _effective(self, scheme="http"):
|
||||
value = None
|
||||
for _path, _n, text in self._settings():
|
||||
m = re.match(rf'\s*Acquire::{scheme}::Proxy\s+"([^"]*)"', text, re.I)
|
||||
if m:
|
||||
value = m.group(1)
|
||||
return value
|
||||
|
||||
def _run_apt_command(self, command):
|
||||
if command == APT_PROXY_COMMAND:
|
||||
value, https = self._effective(), self._effective("https")
|
||||
dump = [f'Acquire::http::Proxy "{value}";'] if value else []
|
||||
dump += [f'Acquire::https::Proxy "{https}";'] if https else []
|
||||
return _report(
|
||||
dump=dump,
|
||||
parts=[f"{p}:{n}:{t}" for p, n, t in self._settings() if p.startswith(_D)],
|
||||
main=[f"{p}:{n}:{t}" for p, n, t in self._settings() if not p.startswith(_D)],
|
||||
sources=["http://deb.debian.org/debian/dists/trixie/", *self.https_sources],
|
||||
)
|
||||
self.fetched.append(command)
|
||||
if re.search(r"download-file '?https://", command):
|
||||
return self._fetch_https(command)
|
||||
return _fetch(_FETCHED if self.fetch_rc == 0 else _REFUSED, self.fetch_rc, match=1)
|
||||
|
||||
def _fetch_https(self, command):
|
||||
https = self._effective("https") or self._effective()
|
||||
if "DIRECT" in command or not https or https.upper() == "DIRECT":
|
||||
return _fetch(_FETCHED if self.direct_rc == 0 else _UNRESOLVED, self.direct_rc)
|
||||
return _fetch(_FETCHED, 0) if self.tunnel else _fetch(_CONNECT_DENIED, 100)
|
||||
|
||||
def _write_apt_file(self, path, content):
|
||||
self.files[path] = content
|
||||
return {"ok": True, "output": f"wrote {path}"}
|
||||
|
||||
def _run_apt_root_command(self, command, done):
|
||||
self.root.append(command)
|
||||
path = command.rsplit(" ", 1)[-1].strip("'")
|
||||
for m in re.finditer(r"(\d+)\{/[^}]*\}", command):
|
||||
n = int(m.group(1))
|
||||
lines = self.files[path].splitlines()
|
||||
lines[n - 1] = "// " + lines[n - 1]
|
||||
self.files[path] = "\n".join(lines) + "\n"
|
||||
if command.startswith("rm -f "):
|
||||
self.files.pop(command.split()[-1].strip("'"), None)
|
||||
return {"ok": True, "output": done}
|
||||
|
||||
|
||||
class TestCheckHttpsSource:
|
||||
def test_one_source_the_way_apt_fetches_it(self):
|
||||
host = _Host(
|
||||
{f"{_D}00proxy": 'Acquire::http::Proxy "http://c:3142";\n'},
|
||||
https_sources=[_DOCKER],
|
||||
tunnel=False,
|
||||
)
|
||||
|
||||
result = host.check_https_source()
|
||||
|
||||
assert result["url"] == f"{_DOCKER}InRelease"
|
||||
assert result["proxy"] == "http://c:3142"
|
||||
assert result["reachable"] is False and "403 CONNECT denied" in result["error"]
|
||||
assert "DIRECT" not in host.fetched[0]
|
||||
|
||||
def test_none_without_an_https_source(self):
|
||||
host = _Host({f"{_D}00proxy": 'Acquire::http::Proxy "http://c:3142";\n'})
|
||||
assert host.check_https_source() is None
|
||||
|
||||
def test_none_where_https_goes_direct(self):
|
||||
host = _Host(
|
||||
{
|
||||
f"{_D}00proxy": (
|
||||
'Acquire::http::Proxy "http://c:3142";\nAcquire::https::Proxy "DIRECT";\n'
|
||||
)
|
||||
},
|
||||
https_sources=[_DOCKER],
|
||||
)
|
||||
assert host.check_https_source() is None
|
||||
|
||||
def test_not_on_a_host_without_apt(self):
|
||||
with pytest.raises(NotImplementedError):
|
||||
_Host(apt=False).check_https_source()
|
||||
|
||||
|
||||
class TestTheMixin:
|
||||
def test_it_is_a_package_proxy(self):
|
||||
assert issubclass(AptProxyMixin, PackageProxyMixin)
|
||||
|
||||
def test_the_fact(self):
|
||||
host = _Host({f"{_D}00proxy": 'Acquire::http::Proxy "http://c:3142";\n'})
|
||||
|
||||
assert host.get_package_proxy() == "http://c:3142"
|
||||
assert host.get_package_proxy_report()["settings"][0]["file"] == f"{_D}00proxy"
|
||||
|
||||
def test_not_an_apt_host(self):
|
||||
host = _Host(apt=False)
|
||||
|
||||
assert host.get_package_proxy() is None
|
||||
assert host.get_package_proxy_report() is None
|
||||
|
||||
def test_check_fetches_the_page_past_any_proxy(self):
|
||||
host = _Host()
|
||||
|
||||
result = host.check_package_proxy("http://c:3142/acng-report.html", expect="Cache")
|
||||
|
||||
assert result == {"reachable": True, "expected": True, "error": ""}
|
||||
assert "Proxy::c=DIRECT" in host.fetched[0]
|
||||
|
||||
|
||||
def _step(result, name):
|
||||
return next(s for s in result["steps"] if s["step"] == name)
|
||||
|
||||
|
||||
class TestConfigurePackageProxy:
|
||||
def test_a_fresh_host(self):
|
||||
host = _Host()
|
||||
|
||||
result = host.configure_package_proxy("http://c:3142")
|
||||
|
||||
assert result["success"] is True
|
||||
assert [s["step"] for s in result["steps"]] == [
|
||||
"configure",
|
||||
"conflicts",
|
||||
"verify",
|
||||
"fetch",
|
||||
"https",
|
||||
]
|
||||
assert "no https source" in result["steps"][-1]["output"]
|
||||
assert host.files[f"{_D}00proxy"] == 'Acquire::http::Proxy "http://c:3142";\n'
|
||||
assert "InRelease" in host.fetched[0] and "DIRECT" not in host.fetched[0]
|
||||
|
||||
def test_a_later_file_is_commented_out_and_named(self):
|
||||
host = _Host(
|
||||
{
|
||||
f"{_D}90curtin-aptproxy": 'Acquire::http::Proxy "http://old:3128";\n',
|
||||
"/etc/apt/apt.conf": 'APT::Foo "1";\nAcquire::http::Proxy "http://older:8080";\n',
|
||||
}
|
||||
)
|
||||
|
||||
result = host.configure_package_proxy("http://c:3142")
|
||||
|
||||
assert result["success"] is True
|
||||
assert host.files[f"{_D}90curtin-aptproxy"].startswith("// Acquire::http::Proxy")
|
||||
assert host.files["/etc/apt/apt.conf"] == (
|
||||
'APT::Foo "1";\n// Acquire::http::Proxy "http://older:8080";\n'
|
||||
)
|
||||
conflicts = result["steps"][1]["output"]
|
||||
assert f"{_D}90curtin-aptproxy:1 (http://old:3128)" in conflicts
|
||||
assert "/etc/apt/apt.conf:2 (http://older:8080)" in conflicts
|
||||
|
||||
def test_the_same_proxy_elsewhere_stays(self):
|
||||
host = _Host({f"{_D}01proxy": 'Acquire::http::Proxy "http://c:3142/";\n'})
|
||||
|
||||
host.configure_package_proxy("http://c:3142")
|
||||
|
||||
assert host.root == []
|
||||
|
||||
def test_a_setting_it_cannot_comment_out_fails_verification(self):
|
||||
host = _Host({"/etc/apt/apt.conf": 'Acquire::http { Proxy "http://older:8080"; };\n'})
|
||||
host._effective = lambda scheme="http": "http://older:8080"
|
||||
|
||||
result = host.configure_package_proxy("http://c:3142")
|
||||
|
||||
assert result["success"] is False
|
||||
assert result["steps"][-1]["step"] == "verify"
|
||||
assert "http://older:8080" in result["steps"][-1]["output"]
|
||||
assert "/etc/apt/apt.conf:1" in result["steps"][-1]["output"]
|
||||
|
||||
def test_a_dead_proxy_fails_the_fetch(self):
|
||||
host = _Host(fetch_rc=100)
|
||||
|
||||
result = host.configure_package_proxy("http://c:3142")
|
||||
|
||||
assert result["success"] is False
|
||||
assert result["steps"][-1]["step"] == "fetch"
|
||||
assert "Connection refused" in result["steps"][-1]["output"]
|
||||
|
||||
def test_an_https_source_the_cache_tunnels_stays_with_it(self):
|
||||
host = _Host(https_sources=[_DOCKER])
|
||||
|
||||
result = host.configure_package_proxy("http://c:3142")
|
||||
|
||||
assert result["success"] is True
|
||||
assert host.files[f"{_D}00proxy"] == 'Acquire::http::Proxy "http://c:3142";\n'
|
||||
assert result["steps"][-1]["output"] == (
|
||||
f"fetched {_DOCKER}InRelease through http://c:3142"
|
||||
)
|
||||
|
||||
def test_an_https_source_the_cache_refuses_goes_direct(self):
|
||||
"""NetOrk/netork#1811: apt-cacher-ng without a PassThroughPattern answers
|
||||
403 to every https source apt sends it."""
|
||||
host = _Host(https_sources=[_DOCKER], tunnel=False)
|
||||
|
||||
result = host.configure_package_proxy("http://c:3142")
|
||||
|
||||
assert result["success"] is True
|
||||
assert host.files[f"{_D}00proxy"] == (
|
||||
'Acquire::http::Proxy "http://c:3142";\nAcquire::https::Proxy "DIRECT";\n'
|
||||
)
|
||||
output = result["steps"][-1]["output"]
|
||||
assert "http://c:3142 refused https" in output and "403 CONNECT denied" in output
|
||||
assert f"https sources go direct now: fetched {_DOCKER}InRelease" in output
|
||||
|
||||
def test_a_later_https_line_to_the_cache_is_commented_out(self):
|
||||
"""Ubuntu's installer writes the https proxy too; it would win over 00proxy."""
|
||||
curtin = f"{_D}90curtin-aptproxy"
|
||||
host = _Host(
|
||||
{
|
||||
curtin: (
|
||||
'Acquire::http::Proxy "http://c:3142";\n'
|
||||
'Acquire::https::Proxy "http://c:3142";\n'
|
||||
)
|
||||
},
|
||||
https_sources=[_DOCKER],
|
||||
tunnel=False,
|
||||
)
|
||||
|
||||
result = host.configure_package_proxy("http://c:3142")
|
||||
|
||||
assert result["success"] is True
|
||||
assert host.files[curtin] == (
|
||||
'Acquire::http::Proxy "http://c:3142";\n// Acquire::https::Proxy "http://c:3142";\n'
|
||||
)
|
||||
assert f"{curtin}:2 (http://c:3142) commented out" in result["steps"][-1]["output"]
|
||||
|
||||
def test_an_https_proxy_of_its_own_is_not_the_fix_s(self):
|
||||
host = _Host(
|
||||
{f"{_D}01https": 'Acquire::https::Proxy "http://corp:3128";\n'},
|
||||
https_sources=[_DOCKER],
|
||||
tunnel=False,
|
||||
)
|
||||
|
||||
result = host.configure_package_proxy("http://c:3142")
|
||||
|
||||
assert result["success"] is True
|
||||
assert "no https source goes through http://c:3142" in result["steps"][-1]["output"]
|
||||
assert f"{_D}00proxy" in host.files and "https" not in host.files[f"{_D}00proxy"]
|
||||
|
||||
def test_an_http_error_through_the_tunnel_is_a_tunnel(self):
|
||||
"""The mirror answered, so the cache let the source through."""
|
||||
host = _Host(https_sources=[_DOCKER])
|
||||
host._fetch_https = lambda command: _fetch(_HTTP_ERROR, 100)
|
||||
|
||||
result = host.configure_package_proxy("http://c:3142")
|
||||
|
||||
assert result["success"] is True
|
||||
assert "https" not in host.files[f"{_D}00proxy"]
|
||||
|
||||
def test_refused_and_no_way_direct_fails(self):
|
||||
host = _Host(https_sources=[_DOCKER], tunnel=False, direct_rc=100)
|
||||
|
||||
result = host.configure_package_proxy("http://c:3142")
|
||||
|
||||
assert result["success"] is False
|
||||
assert result["steps"][-1]["step"] == "https"
|
||||
assert "Could not resolve" in result["steps"][-1]["output"]
|
||||
|
||||
def test_a_later_https_line_it_cannot_comment_out_fails(self):
|
||||
host = _Host(
|
||||
{"/etc/apt/apt.conf": 'Acquire::https { Proxy "http://c:3142"; };\n'},
|
||||
https_sources=[_DOCKER],
|
||||
tunnel=False,
|
||||
)
|
||||
host._effective = lambda scheme="http": "http://c:3142"
|
||||
|
||||
result = host.configure_package_proxy("http://c:3142")
|
||||
|
||||
assert result["success"] is False
|
||||
assert result["steps"][-1]["step"] == "https"
|
||||
assert "/etc/apt/apt.conf:1" in result["steps"][-1]["output"]
|
||||
|
||||
def test_without_an_http_source_the_fetch_says_so(self):
|
||||
host = _Host()
|
||||
real = host._run_apt_command
|
||||
|
||||
def no_sources(command):
|
||||
return real(command).replace("http://deb.debian.org/debian/dists/trixie/\n", "")
|
||||
|
||||
host._run_apt_command = no_sources
|
||||
|
||||
result = host.configure_package_proxy("http://c:3142")
|
||||
|
||||
assert result["success"] is True
|
||||
assert "no http source" in _step(result, "fetch")["output"]
|
||||
|
||||
def test_a_direct_source_is_not_proof(self):
|
||||
host = _Host()
|
||||
real = host._run_apt_command
|
||||
host._run_apt_command = lambda command: real(command).replace(
|
||||
"[parts]", 'Acquire::http::Proxy::deb.debian.org "DIRECT";\n[parts]'
|
||||
)
|
||||
|
||||
result = host.configure_package_proxy("http://c:3142")
|
||||
|
||||
assert "no http source" in _step(result, "fetch")["output"]
|
||||
|
||||
@pytest.mark.parametrize("url", ['http://x"; Acquire::x "y', "ftp://c", "c:3142"])
|
||||
def test_a_url_that_would_break_the_file_is_refused(self, url):
|
||||
host = _Host()
|
||||
|
||||
result = host.configure_package_proxy(url)
|
||||
|
||||
assert result["success"] is False
|
||||
assert host.files == {}
|
||||
|
||||
def test_an_empty_url_removes_the_file(self):
|
||||
host = _Host({f"{_D}00proxy": 'Acquire::http::Proxy "http://c:3142";\n'})
|
||||
|
||||
result = host.configure_package_proxy("")
|
||||
|
||||
assert result["success"] is True
|
||||
assert host.files == {}
|
||||
|
||||
def test_only_an_apt_host_takes_one(self):
|
||||
host = _Host(apt=False)
|
||||
|
||||
result = host.configure_package_proxy("http://c:3142")
|
||||
|
||||
assert result["success"] is False
|
||||
assert host.files == {}
|
||||
|
||||
|
||||
class TestWhatTheParserSkips:
|
||||
def test_a_line_that_is_no_grep_output(self):
|
||||
assert (
|
||||
parse_apt_proxy(_report(parts=["grep: /etc/apt/apt.conf.d/x: Permission denied"]))[
|
||||
"settings"
|
||||
]
|
||||
== []
|
||||
)
|
||||
|
||||
def test_a_dump_line_of_another_key(self):
|
||||
report = parse_apt_proxy(_report(dump=['Acquire::http::Timeout "10";']))
|
||||
assert report["proxy"] is None
|
||||
|
||||
def test_a_fetch_report_without_an_exit_status_raises(self):
|
||||
with pytest.raises(ValueError, match="exit status"):
|
||||
parse_apt_fetch("APROXY_BEGIN\n[fetch]\nGet:1 x\nAPROXY_END\n")
|
||||
|
||||
|
||||
class _Plain(AptProxyMixin):
|
||||
"""A driver that says nothing about its package manager: apt is assumed."""
|
||||
|
||||
|
||||
def test_without_its_own_answer_a_host_is_an_apt_host():
|
||||
assert _Plain()._is_apt_host() is True
|
||||
|
||||
|
||||
class TestNotAnAptHost:
|
||||
def test_no_check(self):
|
||||
with pytest.raises(NotImplementedError):
|
||||
_Host(apt=False).check_package_proxy("http://c:3142/x")
|
||||
|
||||
def test_no_removal(self):
|
||||
result = _Host(apt=False).configure_package_proxy("")
|
||||
assert result["success"] is False
|
||||
assert "not an apt host" in result["output"]
|
||||
|
||||
|
||||
def test_a_comment_out_that_fails_stops_the_fix():
|
||||
host = _Host({f"{_D}90curtin-aptproxy": 'Acquire::http::Proxy "http://old:3128";\n'})
|
||||
host._run_apt_root_command = lambda command, done: {"ok": False, "output": "sed: denied"}
|
||||
|
||||
result = host.configure_package_proxy("http://c:3142")
|
||||
|
||||
assert result["success"] is False
|
||||
assert result["steps"][-1] == {"step": "conflicts", "ok": False, "output": "sed: denied"}
|
||||
|
||||
|
||||
def test_the_comment_out_command():
|
||||
assert comment_out_command("/etc/apt/apt.conf", [2, 4]) == (
|
||||
"sed -i -e '2{/^[[:space:]]*Acquire::http::Proxy[[:space:]]/Is|^|// |}' "
|
||||
"-e '4{/^[[:space:]]*Acquire::http::Proxy[[:space:]]/Is|^|// |}' /etc/apt/apt.conf"
|
||||
)
|
||||
|
||||
|
||||
def test_the_comment_out_command_for_https():
|
||||
assert comment_out_command("/etc/apt/apt.conf", [3], scheme="https") == (
|
||||
"sed -i -e '3{/^[[:space:]]*Acquire::https::Proxy[[:space:]]/Is|^|// |}' /etc/apt/apt.conf"
|
||||
)
|
||||
@@ -0,0 +1,27 @@
|
||||
"""An ARP or neighbour entry whose age the device does not report says so.
|
||||
|
||||
NAPALM's ``get_arp_table`` and ``get_ipv6_neighbors_table`` require an ``age``
|
||||
on every entry, and a driver that could not read one used to fill in ``0.0``.
|
||||
To a caller that is the freshest sighting there is: netOrk took a neighbour the
|
||||
device had not confirmed for hours as seen this second (NetOrk/netork#1327).
|
||||
Unknown is ``-1.0``, as in NAPALM's own IOS driver; ``0.0`` means "just now".
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from napalm_device_types import ARP_AGE_UNKNOWN
|
||||
from napalm_device_types.arp import ARP_AGE_UNKNOWN as FROM_MODULE
|
||||
|
||||
|
||||
def test_an_unknown_age_is_minus_one():
|
||||
assert ARP_AGE_UNKNOWN == -1.0
|
||||
assert isinstance(ARP_AGE_UNKNOWN, float)
|
||||
|
||||
|
||||
def test_the_package_exports_the_modules_constant():
|
||||
assert ARP_AGE_UNKNOWN is FROM_MODULE
|
||||
|
||||
|
||||
def test_no_real_age_can_be_mistaken_for_unknown():
|
||||
# An age is seconds since the entry was last confirmed: never negative.
|
||||
assert ARP_AGE_UNKNOWN < 0
|
||||
@@ -0,0 +1,234 @@
|
||||
"""A backup server is a role of its own (NAPALM/napalm-device-types#82).
|
||||
|
||||
What makes a device a backup server -- datastores, the backup groups in them,
|
||||
the jobs that keep them (garbage collection, verify, prune, sync, tape) and the
|
||||
remotes it syncs with -- has no counterpart in ``StorageDriver``. A backup
|
||||
server that also manages its own disks lists ``StorageDriver`` as a second role.
|
||||
|
||||
The declarations sit under ``TYPE_CHECKING`` (see test_role_contracts), so the
|
||||
signatures are read from the source rather than from the class.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import ast
|
||||
import inspect
|
||||
from typing import NotRequired, get_args, get_type_hints
|
||||
|
||||
import pytest
|
||||
|
||||
import napalm_device_types.backup_server as backup_server_module
|
||||
from napalm_device_types import (
|
||||
BACKUP_JOB_KINDS,
|
||||
BackupDatastoreDict,
|
||||
BackupGroupDict,
|
||||
BackupRemoteDict,
|
||||
BackupServerDriver,
|
||||
BackupServerJobDict,
|
||||
BackupServerRefDict,
|
||||
BackupSnapshotDict,
|
||||
BackupSnapshotRefDict,
|
||||
BackupTaskDict,
|
||||
BackupTaskStatusDict,
|
||||
ClusterStorageDict,
|
||||
OSDriver,
|
||||
StorageDriver,
|
||||
)
|
||||
from napalm_device_types.backups import BackupJobKind, BackupRunState, BackupVerifyState
|
||||
from napalm_device_types.roles import primary_role_of, role_keys_of
|
||||
|
||||
|
||||
def _declared(name: str) -> ast.FunctionDef:
|
||||
tree = ast.parse(inspect.getsource(backup_server_module))
|
||||
for node in ast.walk(tree):
|
||||
if isinstance(node, ast.FunctionDef) and node.name == name:
|
||||
return node
|
||||
raise AssertionError(f"BackupServerDriver does not declare {name}()")
|
||||
|
||||
|
||||
def _signature(name: str) -> tuple[list[str], str]:
|
||||
fn = _declared(name)
|
||||
params = ast.unparse(fn.args).split(", ")[1:]
|
||||
return params, ast.unparse(fn.returns) if fn.returns else ""
|
||||
|
||||
|
||||
class TestRole:
|
||||
def test_role_key_and_label(self):
|
||||
assert vars(BackupServerDriver)["ROLE"] == "backup_server"
|
||||
assert BackupServerDriver.TYPE_LABEL == "Backup Server"
|
||||
|
||||
def test_backup_server_first_storage_second(self):
|
||||
"""A PBS manages its disks and ZFS pools too; that part is storage."""
|
||||
|
||||
class Pbs(BackupServerDriver, StorageDriver, OSDriver):
|
||||
pass
|
||||
|
||||
assert role_keys_of(Pbs) == ["backup_server", "storage", "linux"]
|
||||
assert primary_role_of(Pbs) == "backup_server"
|
||||
|
||||
|
||||
class TestDeclarations:
|
||||
@pytest.mark.parametrize(
|
||||
("name", "params", "returns"),
|
||||
[
|
||||
("get_backup_datastores", [], "list[BackupDatastoreDict]"),
|
||||
("get_backup_groups", ["datastore: str=''"], "list[BackupGroupDict]"),
|
||||
(
|
||||
"get_backup_snapshots",
|
||||
["datastore: str", "namespace: str", "group_type: str", "group_id: str"],
|
||||
"list[BackupSnapshotDict]",
|
||||
),
|
||||
("get_backup_jobs", [], "list[BackupServerJobDict]"),
|
||||
("get_backup_remotes", [], "list[BackupRemoteDict]"),
|
||||
("start_garbage_collection", ["datastore: str"], "BackupTaskDict"),
|
||||
("run_backup_job", ["kind: BackupJobKind", "job_id: str"], "BackupTaskDict"),
|
||||
("verify_backup_snapshot", ["snapshot: BackupSnapshotRefDict"], "BackupTaskDict"),
|
||||
(
|
||||
"set_backup_snapshot_protected",
|
||||
["snapshot: BackupSnapshotRefDict", "protected: bool"],
|
||||
"None",
|
||||
),
|
||||
("delete_backup_snapshot", ["snapshot: BackupSnapshotRefDict"], "None"),
|
||||
("get_backup_task", ["task: str"], "BackupTaskStatusDict"),
|
||||
],
|
||||
)
|
||||
def test_declared_with_its_signature(self, name, params, returns):
|
||||
got_params, got_returns = _signature(name)
|
||||
assert [p.replace(" ", "") for p in got_params] == [p.replace(" ", "") for p in params]
|
||||
assert got_returns == returns
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"name",
|
||||
[
|
||||
"get_backup_datastores",
|
||||
"get_backup_groups",
|
||||
"get_backup_snapshots",
|
||||
"get_backup_jobs",
|
||||
"get_backup_remotes",
|
||||
"start_garbage_collection",
|
||||
"run_backup_job",
|
||||
"verify_backup_snapshot",
|
||||
"set_backup_snapshot_protected",
|
||||
"delete_backup_snapshot",
|
||||
"get_backup_task",
|
||||
],
|
||||
)
|
||||
def test_absent_until_a_driver_implements_it(self, name):
|
||||
assert not hasattr(BackupServerDriver, name)
|
||||
|
||||
def test_every_declaration_has_a_docstring(self):
|
||||
tree = ast.parse(inspect.getsource(backup_server_module))
|
||||
missing = [
|
||||
n.name
|
||||
for n in ast.walk(tree)
|
||||
if isinstance(n, ast.FunctionDef) and not ast.get_docstring(n)
|
||||
]
|
||||
assert missing == []
|
||||
|
||||
|
||||
class TestShapes:
|
||||
"""Neutral names: nothing here is Proxmox's. A value the platform does not
|
||||
report is None; it is never guessed."""
|
||||
|
||||
def test_datastore(self):
|
||||
assert set(get_type_hints(BackupDatastoreDict)) == {
|
||||
"name",
|
||||
"path",
|
||||
"backend",
|
||||
"total",
|
||||
"used",
|
||||
"available",
|
||||
"estimated_full",
|
||||
"maintenance",
|
||||
"comment",
|
||||
}
|
||||
|
||||
def test_group_carries_its_newest_snapshot_not_all_of_them(self):
|
||||
assert set(get_type_hints(BackupGroupDict)) == {
|
||||
"datastore",
|
||||
"namespace",
|
||||
"type",
|
||||
"id",
|
||||
"owner",
|
||||
"count",
|
||||
"newest",
|
||||
"oldest_time",
|
||||
"comment",
|
||||
}
|
||||
assert get_type_hints(BackupGroupDict)["newest"] == BackupSnapshotDict | None
|
||||
|
||||
def test_snapshot(self):
|
||||
assert set(get_type_hints(BackupSnapshotDict)) == {
|
||||
"time",
|
||||
"size",
|
||||
"verify",
|
||||
"protected",
|
||||
"comment",
|
||||
}
|
||||
assert set(get_args(BackupVerifyState)) == {"ok", "failed"}
|
||||
|
||||
def test_snapshot_reference_names_one_snapshot(self):
|
||||
assert set(get_type_hints(BackupSnapshotRefDict)) == {
|
||||
"datastore",
|
||||
"namespace",
|
||||
"group_type",
|
||||
"group_id",
|
||||
"time",
|
||||
}
|
||||
|
||||
def test_job_kinds_include_garbage_collection(self):
|
||||
"""GC is a job with a schedule and a last run like the others, so it is
|
||||
listed with them rather than restated on the datastore."""
|
||||
assert set(get_args(BackupJobKind)) == {"gc", "verify", "prune", "sync", "tape"}
|
||||
assert set(BACKUP_JOB_KINDS) == set(get_args(BackupJobKind))
|
||||
|
||||
def test_job(self):
|
||||
assert set(get_type_hints(BackupServerJobDict)) == {
|
||||
"id",
|
||||
"kind",
|
||||
"datastore",
|
||||
"namespace",
|
||||
"schedule",
|
||||
"enabled",
|
||||
"last_run",
|
||||
"last_state",
|
||||
"last_error",
|
||||
"next_run",
|
||||
"remote",
|
||||
"direction",
|
||||
"comment",
|
||||
}
|
||||
assert set(get_args(BackupRunState)) == {"ok", "warning", "error", "running"}
|
||||
|
||||
def test_remote_never_carries_its_secret(self):
|
||||
fields = set(get_type_hints(BackupRemoteDict))
|
||||
assert fields == {"name", "host", "port", "fingerprint", "auth_id", "comment"}
|
||||
assert not fields & {"password", "secret", "token"}
|
||||
|
||||
def test_task(self):
|
||||
assert set(get_type_hints(BackupTaskDict)) == {"task"}
|
||||
assert set(get_type_hints(BackupTaskStatusDict)) == {
|
||||
"task",
|
||||
"running",
|
||||
"state",
|
||||
"error",
|
||||
"log",
|
||||
}
|
||||
|
||||
|
||||
class TestClusterStorageNamesItsBackupServer:
|
||||
"""A hypervisor's backup storage says which server and datastore it points
|
||||
at, so an application can tie the server's groups to the cluster's guests."""
|
||||
|
||||
def test_backup_server_reference_is_optional_on_a_storage(self):
|
||||
hints = get_type_hints(ClusterStorageDict, include_extras=True)
|
||||
assert hints["backup_server"] == NotRequired[BackupServerRefDict | None]
|
||||
|
||||
def test_reference(self):
|
||||
assert set(get_type_hints(BackupServerRefDict)) == {
|
||||
"server",
|
||||
"port",
|
||||
"datastore",
|
||||
"namespace",
|
||||
"fingerprint",
|
||||
}
|
||||
@@ -0,0 +1,184 @@
|
||||
"""The command channel: how a driver runs a command and opens a byte stream.
|
||||
|
||||
netOrk's container runtime driver speaks the Docker Engine API over a stream
|
||||
from ``docker system dial-stdio`` (NetOrk/netork#765). The driver only provides
|
||||
the way there; these are the SSH pieces every SSH driver can reuse. The
|
||||
paramiko channel is faked, so the tests pin behaviour, not paramiko.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types.channel import (
|
||||
CommandResult,
|
||||
ParamikoExecStream,
|
||||
open_stream_on_transport,
|
||||
run_on_transport,
|
||||
)
|
||||
|
||||
|
||||
class FakeChannel:
|
||||
"""Just enough of paramiko.Channel: scripted stdout/stderr chunks, an exit code."""
|
||||
|
||||
def __init__(self, stdout=(), stderr=(), exit_code=0, hang=False):
|
||||
self.out = list(stdout)
|
||||
self.err = list(stderr)
|
||||
self.exit_code = exit_code
|
||||
self.hang = hang
|
||||
self.sent = b""
|
||||
self.command = None
|
||||
self.write_closed = False
|
||||
self.closed = False
|
||||
self.timeout = None
|
||||
|
||||
def exec_command(self, command):
|
||||
self.command = command
|
||||
|
||||
def settimeout(self, timeout):
|
||||
self.timeout = timeout
|
||||
|
||||
def sendall(self, data):
|
||||
self.sent += data
|
||||
|
||||
def shutdown_write(self):
|
||||
self.write_closed = True
|
||||
|
||||
def close(self):
|
||||
self.closed = True
|
||||
|
||||
def recv_ready(self):
|
||||
return bool(self.out)
|
||||
|
||||
def recv(self, n):
|
||||
if self.out:
|
||||
return self.out.pop(0)
|
||||
if self.hang:
|
||||
raise TimeoutError()
|
||||
return b""
|
||||
|
||||
def recv_stderr_ready(self):
|
||||
return bool(self.err)
|
||||
|
||||
def recv_stderr(self, n):
|
||||
return self.err.pop(0) if self.err else b""
|
||||
|
||||
def exit_status_ready(self):
|
||||
return not self.hang and not self.out and not self.err
|
||||
|
||||
def recv_exit_status(self):
|
||||
return self.exit_code
|
||||
|
||||
|
||||
class FakeTransport:
|
||||
def __init__(self, channel):
|
||||
self.channel = channel
|
||||
|
||||
def open_session(self):
|
||||
return self.channel
|
||||
|
||||
|
||||
def test_run_collects_stdout_stderr_and_the_exit_code():
|
||||
ch = FakeChannel(stdout=[b"hel", b"lo\n"], stderr=[b"warn\n"], exit_code=3)
|
||||
|
||||
result = run_on_transport(FakeTransport(ch), "echo hello", timeout=5)
|
||||
|
||||
assert result == CommandResult(stdout="hello\n", stderr="warn\n", exit_code=3)
|
||||
assert ch.command == "echo hello"
|
||||
assert ch.closed
|
||||
|
||||
|
||||
def test_run_sends_stdin_and_then_closes_the_write_side():
|
||||
ch = FakeChannel(stdout=[b"ok"])
|
||||
|
||||
run_on_transport(FakeTransport(ch), "sudo -S true", stdin=b"pw\n", timeout=5)
|
||||
|
||||
assert ch.sent == b"pw\n"
|
||||
assert ch.write_closed
|
||||
|
||||
|
||||
def test_run_raises_when_the_command_does_not_finish_in_time():
|
||||
ch = FakeChannel(hang=True)
|
||||
|
||||
with pytest.raises(TimeoutError, match="did not finish"):
|
||||
run_on_transport(FakeTransport(ch), "sleep 999", timeout=0.05)
|
||||
assert ch.closed
|
||||
|
||||
|
||||
def test_a_stream_reads_until_eof_and_drains_stderr_on_the_way():
|
||||
ch = FakeChannel(stdout=[b"HTTP/1.1 200 OK\r\n"], stderr=[b"note\n"])
|
||||
stream = open_stream_on_transport(FakeTransport(ch), "docker system dial-stdio")
|
||||
|
||||
assert stream.read(4096, timeout=5) == b"HTTP/1.1 200 OK\r\n"
|
||||
assert stream.read(4096, timeout=5) == b""
|
||||
assert stream.stderr == "note\n"
|
||||
assert ch.command == "docker system dial-stdio"
|
||||
|
||||
|
||||
def test_a_stream_writes_and_closes_its_write_side_only_when_asked():
|
||||
"""dial-stdio answers HTTP 499 to a request whose writer closed early
|
||||
(netork#771), so write() never implies close_write()."""
|
||||
ch = FakeChannel()
|
||||
stream = open_stream_on_transport(FakeTransport(ch), "docker system dial-stdio")
|
||||
|
||||
stream.write(b"GET /_ping HTTP/1.1\r\n\r\n")
|
||||
assert not ch.write_closed
|
||||
|
||||
stream.close_write()
|
||||
assert ch.write_closed and ch.sent.startswith(b"GET /_ping")
|
||||
|
||||
|
||||
def test_a_stream_sends_its_stdin_prefix_first():
|
||||
"""How a sudo password reaches `sudo -S` before the stream's own bytes."""
|
||||
ch = FakeChannel()
|
||||
|
||||
open_stream_on_transport(FakeTransport(ch), "sudo -S -p '' cmd", stdin_prefix=b"pw\n")
|
||||
|
||||
assert ch.sent == b"pw\n"
|
||||
|
||||
|
||||
def test_a_stream_times_out_as_timeout_error():
|
||||
stream = ParamikoExecStream(FakeChannel(hang=True))
|
||||
|
||||
with pytest.raises(TimeoutError):
|
||||
stream.read(10, timeout=0.01)
|
||||
|
||||
|
||||
def test_the_stream_keeps_only_a_bounded_tail_of_stderr():
|
||||
ch = FakeChannel(stderr=[b"x" * 100, b"y" * 100])
|
||||
stream = ParamikoExecStream(ch, stderr_limit=50)
|
||||
|
||||
assert stream.stderr == "y" * 50
|
||||
|
||||
|
||||
def test_exit_status_is_none_while_running_and_the_code_after():
|
||||
running = ParamikoExecStream(FakeChannel(hang=True))
|
||||
finished = ParamikoExecStream(FakeChannel(exit_code=1))
|
||||
|
||||
assert running.exit_status is None
|
||||
assert finished.exit_status == 1
|
||||
|
||||
|
||||
def test_close_closes_the_channel():
|
||||
ch = FakeChannel()
|
||||
|
||||
ParamikoExecStream(ch).close()
|
||||
|
||||
assert ch.closed
|
||||
|
||||
|
||||
def test_run_keeps_output_that_arrives_after_the_exit_status():
|
||||
"""sshd may send exit-status before the last stdout bytes."""
|
||||
|
||||
class EarlyExit(FakeChannel):
|
||||
def exit_status_ready(self):
|
||||
return True
|
||||
|
||||
def recv_ready(self):
|
||||
return False
|
||||
|
||||
channel = EarlyExit(stdout=[b"head ", b"tail"], stderr=[b"warn"])
|
||||
|
||||
result = run_on_transport(FakeTransport(channel), "cmd")
|
||||
|
||||
assert result == CommandResult("head tail", "warn", 0)
|
||||
@@ -0,0 +1,173 @@
|
||||
"""A hypervisor says which cluster it belongs to, and what that cluster holds.
|
||||
|
||||
A Proxmox node is one member of a cluster and can read the whole of it; a vCenter
|
||||
manages several clusters and standalone hosts at once; a standalone host is a
|
||||
cluster of one. One getter serves all three: ``get_clusters()`` lists every
|
||||
cluster the connection can see, each with its nodes, its storages and its
|
||||
networks, and which nodes see which. What a caller does with that (placement,
|
||||
headroom, identity) is the caller's.
|
||||
|
||||
A new VM can be placed on a named node (``create_vm_from_cloud_init(node=…)``),
|
||||
so a manager that runs several hosts takes the caller's choice instead of making
|
||||
its own.
|
||||
|
||||
The declarations sit under ``TYPE_CHECKING`` (see test_role_contracts), so the
|
||||
signatures are read from the source rather than from the class.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import ast
|
||||
import inspect
|
||||
from typing import NotRequired, get_type_hints
|
||||
|
||||
import napalm_device_types.hypervisor as hypervisor_module
|
||||
from napalm_device_types import (
|
||||
BackupDict,
|
||||
BackupJobDict,
|
||||
BackupServerRefDict,
|
||||
ClusterDict,
|
||||
ClusterNetworkDict,
|
||||
ClusterNodeDict,
|
||||
ClusterStorageDict,
|
||||
HAResourceDict,
|
||||
HypervisorDriver,
|
||||
)
|
||||
|
||||
|
||||
def _declared(name: str) -> ast.FunctionDef:
|
||||
tree = ast.parse(inspect.getsource(hypervisor_module))
|
||||
for node in ast.walk(tree):
|
||||
if isinstance(node, ast.FunctionDef) and node.name == name:
|
||||
return node
|
||||
raise AssertionError(f"HypervisorDriver does not declare {name}()")
|
||||
|
||||
|
||||
class TestClusterListing:
|
||||
def test_is_declared(self):
|
||||
fn = _declared("get_clusters")
|
||||
assert ast.unparse(fn.returns) == "list[ClusterDict]"
|
||||
|
||||
def test_absent_until_a_driver_implements_it(self):
|
||||
"""netOrk probes capabilities with hasattr; a hypervisor that cannot
|
||||
describe its cluster must not seem to."""
|
||||
assert not hasattr(HypervisorDriver, "get_clusters")
|
||||
|
||||
|
||||
class TestClusterShape:
|
||||
def test_cluster(self):
|
||||
assert get_type_hints(ClusterDict, include_extras=True) == {
|
||||
"id": str | None,
|
||||
"name": str | None,
|
||||
"quorate": bool | None,
|
||||
"nodes": list[ClusterNodeDict],
|
||||
"storages": list[ClusterStorageDict],
|
||||
"networks": list[ClusterNetworkDict],
|
||||
"ha_resources": NotRequired[list[HAResourceDict] | None],
|
||||
"backup_jobs": NotRequired[list[BackupJobDict] | None],
|
||||
"backups": NotRequired[list[BackupDict] | None],
|
||||
}
|
||||
|
||||
def test_cluster_keys_added_later_stay_optional(self):
|
||||
"""A driver written before HA and backups were part of the report stays valid."""
|
||||
assert ClusterDict.__required_keys__ == frozenset(
|
||||
{"id", "name", "quorate", "nodes", "storages", "networks"}
|
||||
)
|
||||
|
||||
def test_node(self):
|
||||
hints = get_type_hints(ClusterNodeDict, include_extras=True)
|
||||
assert hints == {
|
||||
"name": str,
|
||||
"status": str,
|
||||
"local": bool,
|
||||
"maintenance": bool | None,
|
||||
"cpu_threads": int | None,
|
||||
"cpu_cores": int | None,
|
||||
"cpu_model": str | None,
|
||||
"cpu_flags": list[str] | None,
|
||||
"cpu_usage": float | None,
|
||||
"memory_total": int | None,
|
||||
"memory_used": int | None,
|
||||
"votes": NotRequired[int | None],
|
||||
}
|
||||
assert "votes" not in ClusterNodeDict.__required_keys__
|
||||
|
||||
def test_storage(self):
|
||||
hints = get_type_hints(ClusterStorageDict, include_extras=True)
|
||||
assert hints == {
|
||||
"name": str,
|
||||
"type": str,
|
||||
"shared": bool,
|
||||
"content": list[str],
|
||||
"nodes": list[str],
|
||||
"total": int | None,
|
||||
"used": int | None,
|
||||
"backup_server": NotRequired[BackupServerRefDict | None],
|
||||
}
|
||||
# A driver written before backup servers were part of the report stays valid.
|
||||
assert "backup_server" not in ClusterStorageDict.__required_keys__
|
||||
|
||||
def test_network(self):
|
||||
hints = get_type_hints(ClusterNetworkDict, include_extras=True)
|
||||
assert hints == {
|
||||
"name": str,
|
||||
"kind": str,
|
||||
"vlan_aware": bool,
|
||||
"fixed_vlan_tag": NotRequired[int | None],
|
||||
"nodes": list[str],
|
||||
}
|
||||
assert ClusterNetworkDict.__required_keys__ == frozenset(
|
||||
{"name", "kind", "vlan_aware", "nodes"}
|
||||
)
|
||||
|
||||
|
||||
class TestHAAndBackupShapes:
|
||||
def test_ha_resource(self):
|
||||
hints = get_type_hints(HAResourceDict, include_extras=True)
|
||||
assert hints == {
|
||||
"vmid": int,
|
||||
"kind": str,
|
||||
"requested": str,
|
||||
"status": str | None,
|
||||
"node": str | None,
|
||||
"group": NotRequired[str | None],
|
||||
"rules": list[str],
|
||||
}
|
||||
assert HAResourceDict.__required_keys__ == frozenset(
|
||||
{"vmid", "kind", "requested", "status", "node", "rules"}
|
||||
)
|
||||
|
||||
def test_backup_job(self):
|
||||
assert get_type_hints(BackupJobDict) == {
|
||||
"id": str,
|
||||
"enabled": bool,
|
||||
"schedule": str | None,
|
||||
"storage": str | None,
|
||||
"mode": str | None,
|
||||
"all": bool,
|
||||
"vmids": list[int],
|
||||
"exclude": list[int],
|
||||
"pool": str | None,
|
||||
"node": str | None,
|
||||
}
|
||||
|
||||
def test_backup(self):
|
||||
assert get_type_hints(BackupDict) == {
|
||||
"vmid": int,
|
||||
"storage": str,
|
||||
"time": int,
|
||||
"size": int | None,
|
||||
"count": int,
|
||||
}
|
||||
|
||||
|
||||
class TestCreateVmTakesANode:
|
||||
def test_node_is_an_optional_keyword(self):
|
||||
fn = _declared("create_vm_from_cloud_init")
|
||||
kwonly = {
|
||||
arg.arg: default
|
||||
for arg, default in zip(fn.args.kwonlyargs, fn.args.kw_defaults, strict=True)
|
||||
}
|
||||
assert "node" in kwonly
|
||||
default = kwonly["node"]
|
||||
assert isinstance(default, ast.Constant) and default.value is None
|
||||
@@ -0,0 +1,182 @@
|
||||
"""Access to a container engine: which engines a host has, and a way to each.
|
||||
|
||||
This package abstracts the connection, the driver builds it, and netOrk does
|
||||
the talking (NetOrk/netork#765, decided 2026-10-07). So nothing here knows what
|
||||
a container is. It knows how to reach the engine's API (``dial-stdio``) and how
|
||||
to run the engine's CLI with arguments netOrk chooses, and the driver alone
|
||||
knows where that CLI lives.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import OSDriver
|
||||
from napalm_device_types.channel import CommandResult
|
||||
from napalm_device_types.container_engine import (
|
||||
ContainerEngineConnection,
|
||||
ContainerEngineMixin,
|
||||
ContainerEngineUnavailable,
|
||||
)
|
||||
|
||||
|
||||
class FakeDriver(ContainerEngineMixin):
|
||||
"""A driver with a channel: records what it is asked to run."""
|
||||
|
||||
def __init__(self, present=("docker",), binary=None):
|
||||
self.present = set(present)
|
||||
self.binary = binary
|
||||
self.commands = []
|
||||
self.streams = []
|
||||
|
||||
def run_command(self, command, *, privileged=False, timeout=60, stdin=None):
|
||||
self.commands.append((command, privileged, timeout, stdin))
|
||||
found = any(command == f"command -v {b}" for b in self._binaries())
|
||||
return CommandResult(
|
||||
stdout="/usr/bin/x\n" if found else "", stderr="", exit_code=0 if found else 1
|
||||
)
|
||||
|
||||
def open_stream(self, command, *, privileged=False):
|
||||
self.streams.append((command, privileged))
|
||||
return object()
|
||||
|
||||
def _binaries(self):
|
||||
return {self._container_engine_binary(e) for e in self.present}
|
||||
|
||||
def _container_engine_binary(self, engine):
|
||||
return self.binary or super()._container_engine_binary(engine)
|
||||
|
||||
|
||||
def test_a_host_with_docker_lists_it():
|
||||
assert FakeDriver().container_engines() == [{"engine": "docker", "api": "docker-engine"}]
|
||||
|
||||
|
||||
def test_a_host_without_the_cli_lists_nothing():
|
||||
assert FakeDriver(present=()).container_engines() == []
|
||||
|
||||
|
||||
def test_the_api_is_reached_through_dial_stdio():
|
||||
driver = FakeDriver()
|
||||
|
||||
driver.open_container_engine("docker").open_api()
|
||||
|
||||
assert driver.streams == [("docker system dial-stdio", False)]
|
||||
|
||||
|
||||
def test_the_driver_decides_where_the_binary_lives():
|
||||
"""QNAP keeps docker under Container Station's path; netOrk never sees it."""
|
||||
path = "/share/CACHEDEV1_DATA/.qpkg/container-station/bin/docker"
|
||||
driver = FakeDriver(binary=path)
|
||||
|
||||
assert driver.container_engines() == [{"engine": "docker", "api": "docker-engine"}]
|
||||
driver.open_container_engine("docker").open_api()
|
||||
|
||||
assert driver.streams == [(f"{path} system dial-stdio", False)]
|
||||
|
||||
|
||||
def test_cli_arguments_are_quoted_and_prefixed_by_the_binary():
|
||||
driver = FakeDriver()
|
||||
conn = driver.open_container_engine("docker")
|
||||
|
||||
conn.run_cli(["compose", "-f", "/srv/my stack/compose.yml", "config", "--format", "json"])
|
||||
|
||||
command, privileged, timeout, stdin = driver.commands[-1]
|
||||
assert command == "docker compose -f '/srv/my stack/compose.yml' config --format json"
|
||||
assert privileged is False and stdin is None
|
||||
|
||||
|
||||
def test_cli_stdin_and_timeout_reach_the_channel():
|
||||
driver = FakeDriver()
|
||||
|
||||
driver.open_container_engine("docker").run_cli(
|
||||
["login", "--password-stdin"], stdin=b"x", timeout=30
|
||||
)
|
||||
|
||||
_command, _privileged, timeout, stdin = driver.commands[-1]
|
||||
assert (timeout, stdin) == (30, b"x")
|
||||
|
||||
|
||||
def test_a_streamed_cli_call_can_merge_stderr():
|
||||
"""compose writes its progress to stderr; a merged stream carries both."""
|
||||
driver = FakeDriver()
|
||||
conn = driver.open_container_engine("docker")
|
||||
|
||||
conn.stream_cli(["pull", "redis:7-alpine"])
|
||||
conn.stream_cli(["compose", "up", "-d", "db"], merge_stderr=True)
|
||||
|
||||
assert driver.streams == [
|
||||
("docker pull redis:7-alpine", False),
|
||||
("docker compose up -d db 2>&1", False),
|
||||
]
|
||||
|
||||
|
||||
def test_a_shell_metacharacter_in_an_argument_stays_an_argument():
|
||||
driver = FakeDriver()
|
||||
|
||||
driver.open_container_engine("docker").run_cli(["inspect", "x; rm -rf /"])
|
||||
|
||||
assert driver.commands[-1][0] == "docker inspect 'x; rm -rf /'"
|
||||
|
||||
|
||||
def test_an_unknown_engine_is_refused():
|
||||
with pytest.raises(ContainerEngineUnavailable):
|
||||
FakeDriver().open_container_engine("rkt")
|
||||
|
||||
|
||||
def test_the_connection_names_its_engine():
|
||||
conn = FakeDriver().open_container_engine("docker")
|
||||
|
||||
assert isinstance(conn, ContainerEngineConnection)
|
||||
assert conn.engine == "docker"
|
||||
|
||||
|
||||
def test_an_os_driver_does_not_get_container_engine_access_by_role():
|
||||
"""Mixed in by the driver, not by OSDriver: a Windows host is an OS driver
|
||||
too, and `hasattr(driver, "open_container_engine")` has to stay truthful."""
|
||||
assert not issubclass(OSDriver, ContainerEngineMixin)
|
||||
assert not hasattr(OSDriver, "open_container_engine")
|
||||
|
||||
|
||||
class TestPrivilegedCli:
|
||||
"""A refused engine call can be repeated as root (netork#773): the driver
|
||||
knows the binary and how it gains root, so the connection carries it."""
|
||||
|
||||
def test_run_cli_passes_privileged_to_the_channel(self):
|
||||
driver = FakeDriver()
|
||||
|
||||
driver.open_container_engine("docker").run_cli(["restart", "web"], privileged=True)
|
||||
|
||||
assert driver.commands == [("docker restart web", True, 120, None)]
|
||||
|
||||
def test_run_cli_is_unprivileged_by_default(self):
|
||||
driver = FakeDriver()
|
||||
|
||||
driver.open_container_engine("docker").run_cli(["ps"])
|
||||
|
||||
assert driver.commands[0][1] is False
|
||||
|
||||
def test_stream_cli_passes_privileged_too(self):
|
||||
driver = FakeDriver()
|
||||
|
||||
driver.open_container_engine("docker").stream_cli(["pull", "nginx"], privileged=True)
|
||||
|
||||
assert driver.streams == [("docker pull nginx", True)]
|
||||
|
||||
|
||||
class TestNoDockerModel:
|
||||
"""There is no container model here (NetOrk/netork#765): netOrk reads and
|
||||
handles containers over the Engine API, so the CLI-shaped types and the
|
||||
``get_docker_info()`` declaration that served napalm-linux' old reading are gone."""
|
||||
|
||||
def test_the_docker_types_are_gone(self):
|
||||
import napalm_device_types
|
||||
from napalm_device_types import models
|
||||
|
||||
for name in ("DockerInfoDict", "DockerContainerDict", "DockerImageDict"):
|
||||
assert not hasattr(models, name), name
|
||||
assert not hasattr(napalm_device_types, name), name
|
||||
|
||||
def test_an_os_driver_declares_no_docker_reading(self):
|
||||
from napalm_device_types import os as os_role
|
||||
|
||||
assert "get_docker_info" not in os_role.__dict__.get("OSDriver", object).__dict__
|
||||
@@ -0,0 +1,141 @@
|
||||
"""Cron expressions: what they mean, when they fire next, and as systemd ``OnCalendar``.
|
||||
|
||||
One module answers all three, because a job netOrk writes as a timer must fire
|
||||
exactly when the cron expression the user gave says, and the preview must show
|
||||
the same runs the host will make (NetOrk/netork#1933).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import UTC, datetime
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types.cron import is_reboot, next_runs, parse_cron, to_on_calendar
|
||||
|
||||
BERLIN = ZoneInfo("Europe/Berlin")
|
||||
|
||||
|
||||
class TestParse:
|
||||
def test_every_field_is_expanded_to_its_values(self):
|
||||
s = parse_cron("*/15 6-8 1,15 * *")
|
||||
assert s.minutes == frozenset({0, 15, 30, 45})
|
||||
assert s.hours == frozenset({6, 7, 8})
|
||||
assert s.days == frozenset({1, 15})
|
||||
assert s.months == frozenset(range(1, 13))
|
||||
assert s.weekdays == frozenset(range(7))
|
||||
|
||||
def test_a_range_takes_a_step(self):
|
||||
assert parse_cron("1-30/10 * * * *").minutes == frozenset({1, 11, 21})
|
||||
|
||||
def test_names_are_months_and_weekdays_in_any_case(self):
|
||||
s = parse_cron("0 0 * Jan,MAR mon-Fri")
|
||||
assert s.months == frozenset({1, 3})
|
||||
assert s.weekdays == frozenset({1, 2, 3, 4, 5})
|
||||
|
||||
def test_seven_is_sunday_too(self):
|
||||
assert parse_cron("0 0 * * 7").weekdays == frozenset({0})
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("alias", "expanded"),
|
||||
[
|
||||
("@hourly", "0 * * * *"),
|
||||
("@daily", "0 0 * * *"),
|
||||
("@midnight", "0 0 * * *"),
|
||||
("@weekly", "0 0 * * 0"),
|
||||
("@monthly", "0 0 1 * *"),
|
||||
("@yearly", "0 0 1 1 *"),
|
||||
("@annually", "0 0 1 1 *"),
|
||||
],
|
||||
)
|
||||
def test_an_alias_is_its_expression(self, alias, expanded):
|
||||
assert parse_cron(alias) == parse_cron(expanded)
|
||||
|
||||
def test_reboot_is_no_calendar(self):
|
||||
assert is_reboot("@reboot")
|
||||
assert not is_reboot("0 0 * * *")
|
||||
with pytest.raises(ValueError, match="reboot"):
|
||||
parse_cron("@reboot")
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"bad",
|
||||
[
|
||||
"",
|
||||
"* * * *",
|
||||
"* * * * * *",
|
||||
"60 * * * *",
|
||||
"* 24 * * *",
|
||||
"* * 0 * *",
|
||||
"* * * 13 *",
|
||||
"* * * * 8",
|
||||
"*/0 * * * *",
|
||||
"5-1 * * * *",
|
||||
"a * * * *",
|
||||
"@often",
|
||||
"1,,2 * * * *",
|
||||
],
|
||||
)
|
||||
def test_what_cron_would_not_take_is_refused(self, bad):
|
||||
with pytest.raises(ValueError):
|
||||
parse_cron(bad)
|
||||
|
||||
def test_day_of_month_and_weekday_are_or_only_when_both_are_restricted(self):
|
||||
assert parse_cron("0 0 1 * 1").either_day
|
||||
assert not parse_cron("0 0 */2 * 1").either_day
|
||||
assert not parse_cron("0 0 1 * *").either_day
|
||||
|
||||
|
||||
class TestNextRuns:
|
||||
def test_runs_after_the_instant_in_the_hosts_time(self):
|
||||
after = datetime(2026, 11, 3, 5, 50, tzinfo=UTC) # 06:50 in Berlin
|
||||
runs = next_runs("0 7 * * *", after, BERLIN, count=2)
|
||||
assert runs == [
|
||||
datetime(2026, 11, 3, 7, 0, tzinfo=BERLIN),
|
||||
datetime(2026, 11, 4, 7, 0, tzinfo=BERLIN),
|
||||
]
|
||||
|
||||
def test_the_instant_itself_is_not_a_next_run(self):
|
||||
after = datetime(2026, 11, 3, 6, 0, tzinfo=UTC)
|
||||
assert next_runs("0 6 * * *", after, ZoneInfo("UTC"), count=1)[0].day == 4
|
||||
|
||||
def test_either_day_fires_on_both(self):
|
||||
after = datetime(2026, 11, 30, 12, 0, tzinfo=UTC) # a Monday
|
||||
runs = next_runs("0 0 1 * 1", after, ZoneInfo("UTC"), count=3)
|
||||
assert [r.day for r in runs] == [1, 7, 14]
|
||||
|
||||
def test_a_day_that_never_comes_gives_no_runs(self):
|
||||
assert next_runs("0 0 31 2 *", datetime(2026, 1, 1, tzinfo=UTC), BERLIN) == []
|
||||
|
||||
def test_a_time_the_clock_skips_is_left_out(self):
|
||||
# 2026-03-29 02:30 does not exist in Berlin.
|
||||
after = datetime(2026, 3, 28, 12, 0, tzinfo=UTC)
|
||||
runs = next_runs("30 2 * * *", after, BERLIN, count=2)
|
||||
assert [r.day for r in runs] == [30, 31]
|
||||
|
||||
def test_reboot_has_no_next_run(self):
|
||||
assert next_runs("@reboot", datetime(2026, 1, 1, tzinfo=UTC), BERLIN) == []
|
||||
|
||||
|
||||
class TestOnCalendar:
|
||||
def test_every_field_as_its_values(self):
|
||||
assert to_on_calendar("*/15 6 * * *") == ["*-*-* 06:00,15,30,45:00"]
|
||||
|
||||
def test_a_restricted_weekday_leads(self):
|
||||
assert to_on_calendar("30 4 * * 1-5") == ["Mon,Tue,Wed,Thu,Fri *-*-* 04:30:00"]
|
||||
|
||||
def test_months_and_days(self):
|
||||
assert to_on_calendar("0 0 1,15 1,7 *") == ["*-01,07-01,15 00:00:00"]
|
||||
|
||||
def test_either_day_is_two_lines(self):
|
||||
assert to_on_calendar("0 0 1 * 0") == ["*-*-01 00:00:00", "Sun *-*-* 00:00:00"]
|
||||
|
||||
def test_both_days_with_a_star_are_one_line(self):
|
||||
assert to_on_calendar("0 0 */10 * 6") == ["Sat *-*-01,11,21,31 00:00:00"]
|
||||
|
||||
def test_an_alias(self):
|
||||
assert to_on_calendar("@daily") == ["*-*-* 00:00:00"]
|
||||
|
||||
def test_reboot_is_refused(self):
|
||||
with pytest.raises(ValueError, match="reboot"):
|
||||
to_on_calendar("@reboot")
|
||||
@@ -0,0 +1,145 @@
|
||||
"""Crontabs as text: the jobs in them, and changing one line (NetOrk/netork#1933)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types._crontab import (
|
||||
DISABLED_MARKER,
|
||||
marked_lines,
|
||||
parse_anacrontab,
|
||||
parse_crontab,
|
||||
run_script,
|
||||
switched,
|
||||
with_job,
|
||||
without_job,
|
||||
)
|
||||
|
||||
SYSTEM = """\
|
||||
# /etc/crontab: system-wide crontab
|
||||
SHELL=/bin/sh
|
||||
PATH="/usr/local/sbin:/usr/bin:/bin"
|
||||
17 * * * * root cd / && run-parts --report /etc/cron.hourly
|
||||
@reboot root /usr/local/bin/warm-cache # keeps the # in the command
|
||||
#napalm-disabled: 0 4 * * * backup /usr/local/bin/backup
|
||||
garbage line
|
||||
0 4 * * * backup /usr/local/bin/backup
|
||||
"""
|
||||
|
||||
USER = """\
|
||||
MAILTO=ops@example.org
|
||||
*/5 * * * * /usr/bin/php /var/www/cron.php
|
||||
# napalm-scheduled-job: netork-cleanup tidy the cache
|
||||
0 3 * * * find /tmp -mtime +7 -delete
|
||||
"""
|
||||
|
||||
|
||||
class TestParse:
|
||||
def test_a_system_crontab_names_the_user(self):
|
||||
jobs = parse_crontab("/etc/crontab", SYSTEM, system=True)
|
||||
first = jobs[0]
|
||||
assert (first.schedule, first.run_as) == ("17 * * * *", "root")
|
||||
assert first.command == "cd / && run-parts --report /etc/cron.hourly"
|
||||
assert first.env == {"SHELL": "/bin/sh", "PATH": "/usr/local/sbin:/usr/bin:/bin"}
|
||||
assert first.key.startswith("cron:/etc/crontab:")
|
||||
|
||||
def test_an_alias_and_a_hash_inside_the_command(self):
|
||||
reboot = parse_crontab("/etc/crontab", SYSTEM, system=True)[1]
|
||||
assert reboot.schedule == "@reboot"
|
||||
assert reboot.command == "/usr/local/bin/warm-cache # keeps the # in the command"
|
||||
|
||||
def test_a_line_switched_off_is_a_job_that_is_off(self):
|
||||
jobs = parse_crontab("/etc/crontab", SYSTEM, system=True)
|
||||
off = jobs[2]
|
||||
assert not off.enabled
|
||||
assert off.run_as == "backup"
|
||||
|
||||
def test_equal_lines_get_their_own_keys_and_switching_off_keeps_the_key(self):
|
||||
jobs = parse_crontab("/etc/crontab", SYSTEM, system=True)
|
||||
assert jobs[3].key == jobs[2].key + ":2"
|
||||
assert len(jobs) == 4 # the garbage line is left out
|
||||
|
||||
def test_a_user_crontab_runs_as_its_user(self):
|
||||
jobs = parse_crontab("/var/spool/cron/crontabs/www", USER, system=False, user="www-data")
|
||||
assert jobs[0].run_as == "www-data"
|
||||
assert jobs[0].env == {"MAILTO": "ops@example.org"}
|
||||
|
||||
def test_an_applications_job_is_keyed_by_its_name(self):
|
||||
job = parse_crontab("/x", USER, system=False, user="root")[1]
|
||||
assert (job.key, job.name, job.description) == (
|
||||
"job:netork-cleanup",
|
||||
"netork-cleanup",
|
||||
"tidy the cache",
|
||||
)
|
||||
|
||||
def test_an_anacrontab(self):
|
||||
text = "SHELL=/bin/sh\n1\t5\tcron.daily\trun-parts --report /etc/cron.daily\n@monthly 15 cron.monthly run-parts /etc/cron.monthly\nbad line here\n"
|
||||
jobs = parse_anacrontab(text)
|
||||
assert [(j.key, j.schedule, j.run_as) for j in jobs] == [
|
||||
("anacron:cron.daily", "1", "root"),
|
||||
("anacron:cron.monthly", "@monthly", "root"),
|
||||
]
|
||||
assert jobs[0].command == "run-parts --report /etc/cron.daily"
|
||||
|
||||
|
||||
class TestEdit:
|
||||
def test_switching_off_and_on_again_gives_the_text_back(self):
|
||||
jobs = parse_crontab("/etc/crontab", SYSTEM, system=True)
|
||||
off = switched(SYSTEM, jobs[0], enabled=False)
|
||||
assert DISABLED_MARKER + "17 *\t* * *\troot" in off
|
||||
again = parse_crontab("/etc/crontab", off, system=True)
|
||||
assert again[0].key == jobs[0].key and not again[0].enabled
|
||||
assert switched(off, again[0], enabled=True) == SYSTEM.replace(
|
||||
"17 *\t* * *\troot\tcd", "17 *\t* * *\troot\tcd"
|
||||
)
|
||||
|
||||
def test_an_applications_job_is_added_replaced_and_removed(self):
|
||||
lines = marked_lines("netork-x", "a job", "0 1 * * * /bin/true", enabled=True)
|
||||
added = with_job("A=1\n", [], "netork-x", lines)
|
||||
assert added == "A=1\n# napalm-scheduled-job: netork-x a job\n0 1 * * * /bin/true\n"
|
||||
jobs = parse_crontab("/x", added, system=False, user="root")
|
||||
changed = with_job(
|
||||
added,
|
||||
jobs,
|
||||
"netork-x",
|
||||
marked_lines("netork-x", "", "0 2 * * * /bin/true", enabled=False),
|
||||
)
|
||||
assert (
|
||||
changed
|
||||
== "A=1\n# napalm-scheduled-job: netork-x\n#napalm-disabled: 0 2 * * * /bin/true\n"
|
||||
)
|
||||
jobs = parse_crontab("/x", changed, system=False, user="root")
|
||||
assert without_job(changed, jobs, "netork-x") == "A=1\n"
|
||||
|
||||
def test_adding_to_a_text_without_a_last_newline(self):
|
||||
assert with_job("A=1", [], "n", ["# m", "x"]) == "A=1\n# m\nx\n"
|
||||
|
||||
def test_removing_the_only_job_leaves_nothing(self):
|
||||
text = "# napalm-scheduled-job: n\n0 1 * * * x\n"
|
||||
jobs = parse_crontab("/x", text, system=False, user="root")
|
||||
assert without_job(text, jobs, "n") == ""
|
||||
|
||||
|
||||
class TestRunScript:
|
||||
def _run(self, script: str) -> str:
|
||||
return subprocess.run(
|
||||
["sh", "-c", script], capture_output=True, text=True, check=True
|
||||
).stdout
|
||||
|
||||
def test_the_crontabs_variables_are_set(self):
|
||||
assert (
|
||||
self._run(run_script('echo "$GREETING"', {"GREETING": "hi there", "MAILTO": "x"}))
|
||||
== "hi there\n"
|
||||
)
|
||||
|
||||
def test_a_percent_starts_stdin_and_further_ones_are_newlines(self):
|
||||
assert self._run(run_script("cat%one%two", {})) == "one\ntwo\n"
|
||||
|
||||
def test_an_escaped_percent_is_a_percent(self):
|
||||
assert self._run(run_script("date -d @0 -u +\\%Y", {})) == "1970\n"
|
||||
|
||||
@pytest.mark.parametrize("command", ["echo plain", "echo 'quoted'"])
|
||||
def test_without_percent_the_command_is_as_it_is(self, command):
|
||||
assert run_script(command, {}) == command
|
||||
@@ -9,9 +9,10 @@ principle: generic vs. device-specific logic" for why this logic lives here
|
||||
and not in a vendor driver.
|
||||
"""
|
||||
|
||||
from typing import Any, Dict, List, Optional
|
||||
from typing import Any
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import DhcpServerMixin, FirewallDriver, ResidentialGatewayDriver
|
||||
from napalm_device_types.dhcp import normalize_mac
|
||||
from napalm_device_types.models import DhcpReservationDict
|
||||
@@ -33,21 +34,21 @@ def _reservation(**overrides: Any) -> DhcpReservationDict:
|
||||
class _FakeDhcpServer(DhcpServerMixin):
|
||||
"""In-memory fake — no network, no Kea/UCI specifics."""
|
||||
|
||||
def __init__(self, live: Optional[List[DhcpReservationDict]] = None) -> None:
|
||||
def __init__(self, live: list[DhcpReservationDict] | None = None) -> None:
|
||||
self.live = live or []
|
||||
self.applied: List[Dict[str, Any]] = []
|
||||
self.applied: list[dict[str, Any]] = []
|
||||
self.commits = 0
|
||||
|
||||
def get_dhcp_reservations(self) -> List[DhcpReservationDict]:
|
||||
def get_dhcp_reservations(self) -> list[DhcpReservationDict]:
|
||||
return self.live
|
||||
|
||||
def apply_dhcp_reservation(
|
||||
self, reservation: DhcpReservationDict, *, uuid: Optional[str] = None
|
||||
) -> Dict[str, Any]:
|
||||
self, reservation: DhcpReservationDict, *, uuid: str | None = None
|
||||
) -> dict[str, Any]:
|
||||
self.applied.append({"reservation": reservation, "uuid": uuid})
|
||||
return {"success": True}
|
||||
|
||||
def commit_dhcp_reservations(self) -> Dict[str, Any]:
|
||||
def commit_dhcp_reservations(self) -> dict[str, Any]:
|
||||
self.commits += 1
|
||||
return {"success": True}
|
||||
|
||||
@@ -206,3 +207,25 @@ class TestApplyReservationSet:
|
||||
assert driver.commits == 0
|
||||
assert driver.applied == []
|
||||
assert lines == ["[commit] no changes"]
|
||||
|
||||
|
||||
class TestKeysThatCannotNameOneReservation:
|
||||
def test_a_live_reservation_without_mac_is_never_matched(self):
|
||||
driver = _FakeDhcpServer([_reservation(uuid="b", mac="")])
|
||||
|
||||
diff = driver.diff_dhcp_reservations([_reservation()])
|
||||
|
||||
assert diff == {"add": [_reservation()], "update": []}
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"live, desired",
|
||||
[
|
||||
([], [_reservation(mac="")]),
|
||||
([], [_reservation(mac="AA-BB-CC-DD-EE-02"), _reservation(mac="aabb.ccdd.ee02")]),
|
||||
([_reservation(uuid="a"), _reservation(uuid="b")], [_reservation()]),
|
||||
],
|
||||
ids=["empty desired", "desired twice", "two live share it"],
|
||||
)
|
||||
def test_raises(self, live, desired):
|
||||
with pytest.raises(ValueError):
|
||||
_FakeDhcpServer(live).diff_dhcp_reservations(desired)
|
||||
@@ -0,0 +1,69 @@
|
||||
"""Two facts an HA pair of servers reports about itself.
|
||||
|
||||
`get_dhcp_ha` is what a DHCP server's own HA configuration says: the peers it
|
||||
pairs with and each peer's role, as the server words it. `get_config_sync` is
|
||||
where a host pushes its configuration and for which of its services. Both are
|
||||
declarations; what they mean for the pair -- hot standby, who to write to --
|
||||
is the caller's to work out.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import inspect
|
||||
|
||||
import napalm_device_types as package
|
||||
from napalm_device_types import ConfigSyncMixin, DhcpServerMixin, FirewallDriver
|
||||
|
||||
|
||||
class TestDeclaredOnly:
|
||||
def test_get_dhcp_ha_is_absent_until_a_driver_implements_it(self):
|
||||
assert not hasattr(DhcpServerMixin, "get_dhcp_ha")
|
||||
|
||||
def test_get_config_sync_is_absent_until_a_driver_implements_it(self):
|
||||
assert not hasattr(ConfigSyncMixin, "get_config_sync")
|
||||
|
||||
def test_config_sync_mixin_owns_nothing_callable(self):
|
||||
own = [
|
||||
name
|
||||
for name, value in vars(ConfigSyncMixin).items()
|
||||
if not name.startswith("__") and inspect.isfunction(value)
|
||||
]
|
||||
assert own == []
|
||||
|
||||
def test_the_firewall_role_does_not_carry_config_sync(self):
|
||||
"""A driver mixes the capability in; the role does not promise it."""
|
||||
assert not issubclass(FirewallDriver, ConfigSyncMixin)
|
||||
|
||||
def test_a_driver_that_implements_them_is_found_by_hasattr(self):
|
||||
class Pair(ConfigSyncMixin, DhcpServerMixin):
|
||||
def get_dhcp_ha(self):
|
||||
return {"enabled": False, "peers": []}
|
||||
|
||||
def get_config_sync(self):
|
||||
return {"peer": None, "services": [], "unmapped": []}
|
||||
|
||||
assert callable(getattr(Pair(), "get_dhcp_ha", None))
|
||||
assert callable(getattr(Pair(), "get_config_sync", None))
|
||||
|
||||
|
||||
class TestTypes:
|
||||
def test_are_exported(self):
|
||||
for name in ("ConfigSyncMixin", "ConfigSyncDict", "DhcpHaDict", "DhcpHaPeerDict"):
|
||||
assert name in package.__all__ and hasattr(package, name), name
|
||||
|
||||
def test_a_peer_needs_only_its_name_and_role(self):
|
||||
"""A future server may not have a URL to report; Kea does."""
|
||||
from napalm_device_types import DhcpHaPeerDict
|
||||
|
||||
assert DhcpHaPeerDict.__required_keys__ == frozenset({"name", "role"})
|
||||
|
||||
def test_ha_needs_only_whether_it_is_on_and_the_peers(self):
|
||||
from napalm_device_types import DhcpHaDict
|
||||
|
||||
assert DhcpHaDict.__required_keys__ == frozenset({"enabled", "peers"})
|
||||
|
||||
def test_config_sync_reports_what_it_could_not_name(self):
|
||||
"""A sync item the driver has no service name for is still a fact."""
|
||||
from napalm_device_types import ConfigSyncDict
|
||||
|
||||
assert ConfigSyncDict.__required_keys__ == frozenset({"peer", "services", "unmapped"})
|
||||
@@ -11,9 +11,10 @@ A subnet is a heavier object than a reservation: a wrong `pools` or
|
||||
below lean on the never-delete and preserve-unmanaged-options guarantees.
|
||||
"""
|
||||
|
||||
from typing import Any, Dict, List, Optional
|
||||
from typing import Any
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import DhcpServerMixin
|
||||
from napalm_device_types.models import DhcpSubnetDict
|
||||
|
||||
@@ -35,21 +36,21 @@ def _subnet(**overrides: Any) -> DhcpSubnetDict:
|
||||
|
||||
|
||||
class _FakeDhcpServer(DhcpServerMixin):
|
||||
def __init__(self, live: Optional[List[DhcpSubnetDict]] = None) -> None:
|
||||
def __init__(self, live: list[DhcpSubnetDict] | None = None) -> None:
|
||||
self.live = live or []
|
||||
self.applied: List[Dict[str, Any]] = []
|
||||
self.applied: list[dict[str, Any]] = []
|
||||
self.commits = 0
|
||||
|
||||
def get_dhcp_subnets(self) -> List[DhcpSubnetDict]:
|
||||
def get_dhcp_subnets(self) -> list[DhcpSubnetDict]:
|
||||
return self.live
|
||||
|
||||
def apply_dhcp_subnet(
|
||||
self, subnet: DhcpSubnetDict, *, uuid: Optional[str] = None
|
||||
) -> Dict[str, Any]:
|
||||
self, subnet: DhcpSubnetDict, *, uuid: str | None = None
|
||||
) -> dict[str, Any]:
|
||||
self.applied.append({"subnet": subnet, "uuid": uuid})
|
||||
return {"success": True}
|
||||
|
||||
def commit_dhcp_subnets(self) -> Dict[str, Any]:
|
||||
def commit_dhcp_subnets(self) -> dict[str, Any]:
|
||||
self.commits += 1
|
||||
return {"success": True}
|
||||
|
||||
@@ -215,9 +216,7 @@ class _BareDriver(DhcpServerMixin):
|
||||
pass
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"method", ["get_dhcp_subnets", "apply_dhcp_subnet", "commit_dhcp_subnets"]
|
||||
)
|
||||
@pytest.mark.parametrize("method", ["get_dhcp_subnets", "apply_dhcp_subnet", "commit_dhcp_subnets"])
|
||||
def test_device_specific_methods_are_absent_until_implemented(method: str) -> None:
|
||||
"""Declared under TYPE_CHECKING, so they do not exist until a driver adds them."""
|
||||
assert not hasattr(_BareDriver, method)
|
||||
@@ -226,3 +225,13 @@ def test_device_specific_methods_are_absent_until_implemented(method: str) -> No
|
||||
def test_calling_a_missing_device_method_fails_loudly() -> None:
|
||||
with pytest.raises(AttributeError):
|
||||
_BareDriver().get_dhcp_subnets()
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"desired",
|
||||
[[_subnet(subnet="")], [_subnet(), _subnet(subnet=" 10.10.20.0/24 ")]],
|
||||
ids=["empty", "twice"],
|
||||
)
|
||||
def test_a_subnet_that_cannot_name_one_raises(desired):
|
||||
with pytest.raises(ValueError):
|
||||
_FakeDhcpServer([]).diff_dhcp_subnets(desired)
|
||||
@@ -0,0 +1,63 @@
|
||||
"""Facts instead of warnings (docs/STRUCTURE.md, NetOrk/netork#1111).
|
||||
|
||||
A driver reports what it reads; the application decides what is a warning. Three
|
||||
facts that drivers so far only reported as warning codes get contracts of their
|
||||
own: the package manager's proxy, licenses and subscriptions, and the state of a
|
||||
hypervisor's hosts. All are declared only, so a capability check stays truthful.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import get_type_hints
|
||||
|
||||
from napalm_device_types import (
|
||||
HypervisorDriver,
|
||||
HypervisorHostDict,
|
||||
LicenseDict,
|
||||
LicenseMixin,
|
||||
PackageProxyMixin,
|
||||
)
|
||||
|
||||
|
||||
def test_a_license_says_what_it_is_and_whether_it_holds():
|
||||
hints = get_type_hints(LicenseDict)
|
||||
assert {"name", "status"} <= set(hints)
|
||||
assert LicenseDict.__required_keys__ == frozenset({"name", "status"})
|
||||
assert set(hints) == {"name", "status", "edition", "expires", "detail", "withholds"}
|
||||
|
||||
|
||||
def test_a_missing_license_may_say_what_the_device_withholds_without_it():
|
||||
# A Supermicro X10 with only SFT-OOB-LIC answers Redfish but keeps its drives and
|
||||
# firmware inventory until DCMS is activated: partly licensed, not unlicensed.
|
||||
hints = get_type_hints(LicenseDict)
|
||||
assert hints["withholds"] == list[str]
|
||||
assert "withholds" not in LicenseDict.__required_keys__
|
||||
|
||||
|
||||
def test_a_hypervisor_host_says_how_it_stands():
|
||||
hints = get_type_hints(HypervisorHostDict)
|
||||
assert set(hints) == {"name", "connected", "maintenance", "config_issues", "managed_by"}
|
||||
assert HypervisorHostDict.__required_keys__ == frozenset(
|
||||
{"name", "connected", "maintenance", "config_issues"}
|
||||
)
|
||||
|
||||
|
||||
def test_the_getters_are_declared_only():
|
||||
assert not hasattr(PackageProxyMixin, "get_package_proxy")
|
||||
assert not hasattr(LicenseMixin, "get_licenses")
|
||||
assert not hasattr(HypervisorDriver, "get_hypervisor_hosts")
|
||||
|
||||
|
||||
def test_a_firmware_offer_says_what_runs_and_what_is_offered():
|
||||
"""The vendor's newer firmware is a fact the driver can know (a download
|
||||
catalogue, the device's own update check); whether it is a warning is the
|
||||
application's call. Comparing vendor version strings stays with the driver,
|
||||
so it says itself whether the offer is newer."""
|
||||
from napalm_device_types import FirmwareOfferDict, FirmwareOfferMixin
|
||||
|
||||
hints = get_type_hints(FirmwareOfferDict)
|
||||
assert set(hints) == {"current_version", "latest_version", "newer", "download_url"}
|
||||
assert FirmwareOfferDict.__required_keys__ == frozenset(
|
||||
{"current_version", "latest_version", "newer"}
|
||||
)
|
||||
assert not hasattr(FirmwareOfferMixin, "get_firmware_offer")
|
||||
@@ -3,6 +3,8 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
from napalm.base import NetworkDriver
|
||||
|
||||
from napalm_device_types import (
|
||||
AccessPointDriver,
|
||||
DeviceTypeDriver,
|
||||
@@ -14,9 +16,8 @@ from napalm_device_types import (
|
||||
ResidentialGatewayDriver,
|
||||
StorageDriver,
|
||||
SwitchDriver,
|
||||
rules_max_weight,
|
||||
)
|
||||
from napalm.base import NetworkDriver
|
||||
|
||||
|
||||
# ── Hierarchy ─────────────────────────────────────────────────────────────────
|
||||
|
||||
@@ -48,6 +49,7 @@ def test_device_type_driver_inherits_network_driver():
|
||||
|
||||
# ── Default attributes ────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_defaults_on_device_type_driver():
|
||||
assert DeviceTypeDriver.VENDOR == ""
|
||||
assert DeviceTypeDriver.DRIVER_NAME == ""
|
||||
@@ -71,6 +73,7 @@ def test_fingerprint_attributes_inherited(cls):
|
||||
|
||||
# ── FingerprintRule ───────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_fingerprint_rule_defaults():
|
||||
r = FingerprintRule("fritz!box")
|
||||
assert r.pattern == "fritz!box"
|
||||
@@ -90,6 +93,54 @@ def test_fingerprint_rule_negative():
|
||||
assert r.negative is True
|
||||
|
||||
|
||||
def test_fingerprint_rule_has_no_group_by_default():
|
||||
assert FingerprintRule("fritz!box").group is None
|
||||
|
||||
|
||||
def test_fingerprint_rule_group():
|
||||
r = FingerprintRule("idrac", weight=6.0, group="landing_page")
|
||||
assert r.group == "landing_page"
|
||||
|
||||
|
||||
# ── rules_max_weight: the most a set of rules can add ─────────────────────────
|
||||
|
||||
|
||||
def test_max_weight_sums_rules_that_can_match_together():
|
||||
rules = [FingerprintRule("proxmox virtual environment", 9.0), FingerprintRule("pve", 2.0)]
|
||||
assert rules_max_weight(rules) == 11.0
|
||||
|
||||
|
||||
def test_max_weight_counts_one_rule_per_group():
|
||||
# Alternatives: a page is ATEN's or iDRAC's or iLO's, never all three. Summing
|
||||
# them made a driver that knows three BMC families lose to one that knows none.
|
||||
rules = [
|
||||
FingerprintRule("aten international", 6.0, group="landing_page"),
|
||||
FingerprintRule("idrac", 6.0, group="landing_page"),
|
||||
FingerprintRule("integrated lights-out", 4.0, group="landing_page"),
|
||||
]
|
||||
assert rules_max_weight(rules) == 6.0
|
||||
|
||||
|
||||
def test_max_weight_mixes_groups_and_single_rules():
|
||||
rules = [
|
||||
FingerprintRule("redfishversion", 10.0, group="redfish_answer"),
|
||||
FingerprintRule("@message.extendedinfo", 8.0, group="redfish_answer"),
|
||||
FingerprintRule("#serviceroot.", 5.0),
|
||||
FingerprintRule("aten international", 6.0, group="landing_page"),
|
||||
FingerprintRule("idrac", 6.0, group="landing_page"),
|
||||
]
|
||||
assert rules_max_weight(rules) == 21.0
|
||||
|
||||
|
||||
def test_max_weight_ignores_negative_rules():
|
||||
rules = [FingerprintRule("vcenter", 9.0), FingerprintRule("vmware esxi", 9.0, negative=True)]
|
||||
assert rules_max_weight(rules) == 9.0
|
||||
|
||||
|
||||
def test_max_weight_of_no_rules_is_zero():
|
||||
assert rules_max_weight([]) == 0.0
|
||||
|
||||
|
||||
def test_fingerprint_rule_is_immutable():
|
||||
r = FingerprintRule("test", weight=3.0)
|
||||
with pytest.raises(AttributeError):
|
||||
@@ -98,6 +149,7 @@ def test_fingerprint_rule_is_immutable():
|
||||
|
||||
# ── PortSpec ──────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_port_spec_defaults():
|
||||
p = PortSpec("https", 8006)
|
||||
assert p.scheme == "https"
|
||||
@@ -121,6 +173,7 @@ def test_port_spec_is_immutable():
|
||||
|
||||
# ── Concrete driver subclass ──────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_concrete_driver_overrides_fingerprint():
|
||||
class MyDriver(FirewallDriver):
|
||||
VENDOR = "Acme"
|
||||
|
||||
@@ -9,9 +9,10 @@ device-specific logic" for why this logic lives here and not in a vendor
|
||||
driver.
|
||||
"""
|
||||
|
||||
from typing import Any, Dict, List, Optional
|
||||
from typing import Any
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import FirewallDriver
|
||||
from napalm_device_types.models import FirewallRuleDict
|
||||
|
||||
@@ -39,21 +40,21 @@ def _rule(**overrides: Any) -> FirewallRuleDict:
|
||||
class _FakeFirewall(FirewallDriver):
|
||||
"""In-memory fake — no network, no OPNsense/vendor specifics."""
|
||||
|
||||
def __init__(self, live_rules: Optional[List[FirewallRuleDict]] = None) -> None:
|
||||
def __init__(self, live_rules: list[FirewallRuleDict] | None = None) -> None:
|
||||
self.live_rules = live_rules or []
|
||||
self.applied: List[Dict[str, Any]] = []
|
||||
self.applied: list[dict[str, Any]] = []
|
||||
self.committed = False
|
||||
|
||||
def get_firewall_rules(self) -> List[FirewallRuleDict]:
|
||||
def get_firewall_rules(self) -> list[FirewallRuleDict]:
|
||||
return self.live_rules
|
||||
|
||||
def apply_firewall_rule(
|
||||
self, rule: FirewallRuleDict, *, uuid: Optional[str] = None
|
||||
) -> Dict[str, Any]:
|
||||
self, rule: FirewallRuleDict, *, uuid: str | None = None
|
||||
) -> dict[str, Any]:
|
||||
self.applied.append({"rule": rule, "uuid": uuid})
|
||||
return {"success": True}
|
||||
|
||||
def commit_firewall_rules(self) -> Dict[str, Any]:
|
||||
def commit_firewall_rules(self) -> dict[str, Any]:
|
||||
self.committed = True
|
||||
return {"success": True}
|
||||
|
||||
@@ -169,3 +170,30 @@ class TestApplyFirewallRuleset:
|
||||
|
||||
assert driver.applied == []
|
||||
assert driver.committed is True
|
||||
|
||||
|
||||
class TestKeysThatCannotNameOneRule:
|
||||
"""An empty or shared description must never overwrite a rule by guess."""
|
||||
|
||||
def test_a_live_rule_without_description_is_never_matched(self):
|
||||
driver = _FakeFirewall([_rule(uuid="auto1", description="", action="block")])
|
||||
|
||||
diff = driver.diff_firewall_rules([_rule()])
|
||||
|
||||
assert diff == {"add": [_rule()], "update": []}
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"live, desired",
|
||||
[
|
||||
([], [_rule(description="")]),
|
||||
([], [_rule(), _rule(action="block")]),
|
||||
([_rule(uuid="u1"), _rule(uuid="u2")], [_rule()]),
|
||||
],
|
||||
ids=["empty desired", "desired twice", "two live rules share it"],
|
||||
)
|
||||
def test_raises(self, live, desired):
|
||||
driver = _FakeFirewall(live)
|
||||
|
||||
with pytest.raises(ValueError):
|
||||
driver.diff_firewall_rules(desired)
|
||||
assert driver.applied == []
|
||||
@@ -0,0 +1,269 @@
|
||||
"""Fix routines are templates (docs/STRUCTURE.md, NetOrk/netork#1105).
|
||||
|
||||
The template fixes the order of the steps, the driver knows how each step is done
|
||||
on its device, the caller supplies every value. None of them is mixed into a role
|
||||
base, so ``callable(getattr(driver, name, None))`` stays a truthful answer.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import (
|
||||
AccessPointDriver,
|
||||
BackupServerDriver,
|
||||
EnsurePackageMixin,
|
||||
FirewallDriver,
|
||||
HypervisorDriver,
|
||||
LldpAgentMixin,
|
||||
ManagementControllerDriver,
|
||||
MediaDriver,
|
||||
OSDriver,
|
||||
PackageProxyMixin,
|
||||
PhoneDriver,
|
||||
ResidentialGatewayDriver,
|
||||
SnmpAgentMixin,
|
||||
StorageDriver,
|
||||
SwitchDriver,
|
||||
run_fix_steps,
|
||||
)
|
||||
|
||||
ROLE_BASES = [
|
||||
AccessPointDriver, BackupServerDriver, FirewallDriver, HypervisorDriver, ManagementControllerDriver, MediaDriver,
|
||||
OSDriver, PhoneDriver, ResidentialGatewayDriver, StorageDriver, SwitchDriver,
|
||||
] # fmt: skip
|
||||
|
||||
|
||||
def _ok(output: str = ""):
|
||||
return lambda: {"ok": True, "output": output}
|
||||
|
||||
|
||||
def _fail(output: str):
|
||||
return lambda: {"ok": False, "output": output}
|
||||
|
||||
|
||||
class TestRunFixSteps:
|
||||
def test_every_step_runs_in_order_and_is_reported(self):
|
||||
result = run_fix_steps([("install", _ok("installed")), ("start", _ok("started"))])
|
||||
|
||||
assert result["success"] is True
|
||||
assert result["steps"] == [
|
||||
{"step": "install", "ok": True, "output": "installed"},
|
||||
{"step": "start", "ok": True, "output": "started"},
|
||||
]
|
||||
assert result["output"] == "[install] installed\n[start] started"
|
||||
|
||||
def test_the_first_failure_stops_the_rest(self):
|
||||
ran = []
|
||||
|
||||
def later():
|
||||
ran.append("later")
|
||||
return {"ok": True, "output": ""}
|
||||
|
||||
result = run_fix_steps([("install", _fail("no package")), ("start", later)])
|
||||
|
||||
assert result["success"] is False
|
||||
assert [s["step"] for s in result["steps"]] == ["install"]
|
||||
assert ran == []
|
||||
|
||||
def test_a_step_that_raises_is_a_failed_step(self):
|
||||
def broken():
|
||||
raise RuntimeError("connection lost")
|
||||
|
||||
result = run_fix_steps([("install", broken)])
|
||||
|
||||
assert result["success"] is False
|
||||
assert result["steps"] == [{"step": "install", "ok": False, "output": "connection lost"}]
|
||||
|
||||
|
||||
class _Agent(SnmpAgentMixin):
|
||||
def __init__(self, fail_at: str = ""):
|
||||
self.calls: list = []
|
||||
self.fail_at = fail_at
|
||||
|
||||
def _step(self, name, *args):
|
||||
self.calls.append((name, *args))
|
||||
return {"ok": name != self.fail_at, "output": name}
|
||||
|
||||
def _install_snmp_agent(self):
|
||||
return self._step("install")
|
||||
|
||||
def _write_snmp_agent_config(self, settings):
|
||||
return self._step("configure", settings)
|
||||
|
||||
def _allow_snmp_sources(self, settings):
|
||||
return self._step("firewall", settings)
|
||||
|
||||
def _start_snmp_agent(self):
|
||||
return self._step("start")
|
||||
|
||||
|
||||
class TestSnmpAgent:
|
||||
SETTINGS = {
|
||||
"community": "c0mm",
|
||||
"allowed_sources": ["10.0.0.5"],
|
||||
"contact": "ops",
|
||||
"location": "rack 1",
|
||||
}
|
||||
|
||||
def test_install_configure_open_and_start_with_the_caller_s_values(self):
|
||||
agent = _Agent()
|
||||
|
||||
result = agent.configure_snmp_agent(self.SETTINGS)
|
||||
|
||||
assert result["success"] is True
|
||||
assert agent.calls == [
|
||||
("install",),
|
||||
("configure", self.SETTINGS),
|
||||
("firewall", self.SETTINGS),
|
||||
("start",),
|
||||
]
|
||||
|
||||
def test_a_failed_install_configures_nothing(self):
|
||||
agent = _Agent(fail_at="install")
|
||||
|
||||
assert agent.configure_snmp_agent(self.SETTINGS)["success"] is False
|
||||
assert agent.calls == [("install",)]
|
||||
|
||||
|
||||
class _Packages(EnsurePackageMixin):
|
||||
def __init__(self, installed):
|
||||
self.installed = installed
|
||||
self.asked: list = []
|
||||
|
||||
def get_packages(self):
|
||||
return [{"name": n, "installed": True} for n in self.installed] + [
|
||||
{"name": "lldpd", "installed": False}
|
||||
]
|
||||
|
||||
def install_package(self, name, version=""):
|
||||
self.asked.append(name)
|
||||
|
||||
|
||||
class TestEnsurePackage:
|
||||
def test_installs_what_is_missing(self):
|
||||
driver = _Packages(installed=["curl"])
|
||||
|
||||
result = driver.ensure_package("lldpd")
|
||||
|
||||
assert result["success"] is True
|
||||
assert driver.asked == ["lldpd"]
|
||||
|
||||
def test_leaves_alone_what_is_there(self):
|
||||
driver = _Packages(installed=["lldpd"])
|
||||
|
||||
result = driver.ensure_package("lldpd")
|
||||
|
||||
assert result["success"] is True
|
||||
assert driver.asked == []
|
||||
assert "already installed" in result["output"]
|
||||
|
||||
def test_an_install_that_reports_failure_fails(self):
|
||||
driver = _Packages(installed=[])
|
||||
driver.install_package = lambda name, version="": {
|
||||
"success": False,
|
||||
"output": "E: no candidate",
|
||||
}
|
||||
|
||||
result = driver.ensure_package("lldpd")
|
||||
|
||||
assert result["success"] is False
|
||||
assert "no candidate" in result["output"]
|
||||
|
||||
|
||||
class _Lldp(LldpAgentMixin):
|
||||
"""The generic steps over ensure_package and manage_service; the driver's configure."""
|
||||
|
||||
def __init__(self, refuse: str = ""):
|
||||
self.calls: list = []
|
||||
self.refuse = refuse
|
||||
|
||||
def ensure_package(self, name):
|
||||
self.calls.append(("ensure_package", name))
|
||||
ok = self.refuse != "install"
|
||||
return {"success": ok, "steps": [], "output": "installed" if ok else "E: no candidate"}
|
||||
|
||||
def _write_lldp_agent_config(self, interfaces):
|
||||
self.calls.append(("configure", interfaces))
|
||||
return {"ok": self.refuse != "configure", "output": "interfaces set"}
|
||||
|
||||
def manage_service(self, name, action):
|
||||
self.calls.append((action, name))
|
||||
ok = self.refuse != action
|
||||
return {"success": ok, "output": "" if ok else f"cannot {action}"}
|
||||
|
||||
|
||||
class TestLldpAgent:
|
||||
def test_install_configure_enable_and_restart_on_the_caller_s_interfaces(self):
|
||||
agent = _Lldp()
|
||||
|
||||
result = agent.configure_lldp_agent(["br-lan"])
|
||||
|
||||
assert result["success"] is True
|
||||
assert agent.calls == [
|
||||
("ensure_package", "lldpd"),
|
||||
("configure", ["br-lan"]),
|
||||
("enable", "lldpd"),
|
||||
("restart", "lldpd"),
|
||||
]
|
||||
assert [s["step"] for s in result["steps"]] == ["install", "configure", "start"]
|
||||
|
||||
def test_a_failed_install_configures_nothing(self):
|
||||
agent = _Lldp(refuse="install")
|
||||
|
||||
result = agent.configure_lldp_agent(["br-lan"])
|
||||
|
||||
assert result["success"] is False
|
||||
assert agent.calls == [("ensure_package", "lldpd")]
|
||||
assert "no candidate" in result["output"]
|
||||
|
||||
def test_an_agent_that_will_not_start_at_boot_is_not_restarted(self):
|
||||
agent = _Lldp(refuse="enable")
|
||||
|
||||
result = agent.configure_lldp_agent(["br-lan"])
|
||||
|
||||
assert result["success"] is False
|
||||
assert ("restart", "lldpd") not in agent.calls
|
||||
assert "cannot enable" in result["output"]
|
||||
|
||||
def test_a_failed_restart_fails_the_fix(self):
|
||||
result = _Lldp(refuse="restart").configure_lldp_agent(["br-lan"])
|
||||
|
||||
assert result["success"] is False
|
||||
assert result["steps"][-1]["step"] == "start"
|
||||
|
||||
def test_the_package_and_service_names_are_the_driver_s(self):
|
||||
agent = _Lldp()
|
||||
agent.LLDP_AGENT_PACKAGE = "lldpd-full"
|
||||
agent.LLDP_AGENT_SERVICE = "lldpd.service"
|
||||
|
||||
agent.configure_lldp_agent(["eth0"])
|
||||
|
||||
assert ("ensure_package", "lldpd-full") in agent.calls
|
||||
assert ("restart", "lldpd.service") in agent.calls
|
||||
|
||||
def test_no_interface_is_refused_before_anything_changes(self):
|
||||
agent = _Lldp()
|
||||
|
||||
with pytest.raises(ValueError):
|
||||
agent.configure_lldp_agent([])
|
||||
assert agent.calls == []
|
||||
|
||||
|
||||
@pytest.mark.parametrize("base", ROLE_BASES, ids=lambda b: b.__name__)
|
||||
@pytest.mark.parametrize(
|
||||
"method",
|
||||
[
|
||||
"configure_snmp_agent",
|
||||
"configure_lldp_agent",
|
||||
"ensure_package",
|
||||
"configure_package_proxy",
|
||||
"grant_container_engine_access",
|
||||
],
|
||||
)
|
||||
def test_no_role_base_offers_a_fix(base, method):
|
||||
assert getattr(base, method, None) is None
|
||||
|
||||
|
||||
def test_the_package_proxy_is_declared_only():
|
||||
assert not hasattr(PackageProxyMixin, "configure_package_proxy")
|
||||
@@ -0,0 +1,501 @@
|
||||
"""get_host_firewall: a Linux host's own netfilter rules, read once, in one shape.
|
||||
|
||||
Whether a service on a host is reachable from the internet is not only decided
|
||||
by the address it listens on: the host's own firewall may drop the packets
|
||||
before they reach it. Reading that is the same on every Linux host, so the
|
||||
command and the parse live here, and a driver only carries the command across
|
||||
(NetOrk/netork#1344).
|
||||
|
||||
``nft -j list ruleset`` is the primary reading. Rules that iptables-nft wrote
|
||||
show their extensions there only as an opaque ``xt`` (``-m conntrack``,
|
||||
``-m set``), so the tables iptables manages are taken from ``iptables-save``,
|
||||
which spells them out. A legacy iptables is a ruleset of its own beside nft,
|
||||
and both apply.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from shlex import quote
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types.host_firewall import (
|
||||
HOST_FIREWALL_COMMAND,
|
||||
HostFirewallMixin,
|
||||
HostFirewallUnavailable,
|
||||
parse_host_firewall,
|
||||
)
|
||||
|
||||
IPTABLES_NFT = """# Generated by iptables-save v1.8.10 (nf_tables) on Wed Oct 21 14:52:11 2026
|
||||
*filter
|
||||
:INPUT ACCEPT [2783578:295175610]
|
||||
:FORWARD ACCEPT [0:0]
|
||||
:OUTPUT ACCEPT [0:0]
|
||||
:DOCKER-USER - [0:0]
|
||||
:WGRUNNER - [0:0]
|
||||
-A INPUT -s 10.255.255.4/32 -j WGRUNNER
|
||||
-A INPUT -i ens6 -p udp -m udp --dport 51820 -j ACCEPT
|
||||
-A INPUT -m set --match-set crowdsec-blacklists src -j DROP
|
||||
-A INPUT -i ens6 -m state --state RELATED,ESTABLISHED -j ACCEPT
|
||||
-A INPUT -i ens6 -p tcp -m multiport --dports 80,443,8000:8010 -j ACCEPT
|
||||
-A INPUT ! -i lo -p tcp -m conntrack --ctstate NEW --ctorigdstport 2222 -j LOG --log-prefix "ssh "
|
||||
-A INPUT -i ens6 -j DROP
|
||||
-A FORWARD -j DOCKER-USER
|
||||
-A DOCKER-USER -i eth+ ! -s 192.168.0.0/16 -g WGRUNNER
|
||||
-A WGRUNNER -d 10.7.224.12/32 -p tcp -m tcp --dport 10051 -j ACCEPT
|
||||
-A WGRUNNER -j REJECT --reject-with icmp-port-unreachable
|
||||
-A WGRUNNER -j RETURN
|
||||
COMMIT
|
||||
# Completed on Wed Oct 21 14:52:11 2026
|
||||
*nat
|
||||
:PREROUTING ACCEPT [0:0]
|
||||
:POSTROUTING ACCEPT [0:0]
|
||||
:DOCKER - [0:0]
|
||||
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
|
||||
-A DOCKER ! -i br-1 -p udp -m udp --dport 53 -j DNAT --to-destination 172.18.0.2:53
|
||||
COMMIT
|
||||
"""
|
||||
|
||||
IP6TABLES_NFT = """# Generated by ip6tables-save v1.8.10 (nf_tables) on Wed Oct 21 14:52:11 2026
|
||||
*filter
|
||||
:INPUT DROP [0:0]
|
||||
:FORWARD ACCEPT [0:0]
|
||||
:OUTPUT ACCEPT [0:0]
|
||||
-A INPUT -i lo -j ACCEPT
|
||||
COMMIT
|
||||
"""
|
||||
|
||||
|
||||
def _nft(*objects: dict) -> str:
|
||||
meta = {"metainfo": {"version": "1.0.9", "json_schema_version": 1}}
|
||||
return json.dumps({"nftables": [meta, *objects]})
|
||||
|
||||
|
||||
#: The iptables-nft filter table as nft shows it -- with its extensions opaque --
|
||||
#: and a native ``inet filter`` table of the host's own nftables.service.
|
||||
NFT = _nft(
|
||||
{"table": {"family": "ip", "name": "filter", "handle": 1}},
|
||||
{
|
||||
"chain": {
|
||||
"family": "ip",
|
||||
"table": "filter",
|
||||
"name": "INPUT",
|
||||
"type": "filter",
|
||||
"hook": "input",
|
||||
"prio": 0,
|
||||
"policy": "accept",
|
||||
}
|
||||
},
|
||||
{
|
||||
"rule": {
|
||||
"family": "ip",
|
||||
"table": "filter",
|
||||
"chain": "INPUT",
|
||||
"expr": [
|
||||
{"xt": {"type": "match", "name": "set"}},
|
||||
{"counter": {"packets": 0, "bytes": 0}},
|
||||
{"drop": None},
|
||||
],
|
||||
}
|
||||
},
|
||||
{"table": {"family": "inet", "name": "host", "handle": 9}},
|
||||
{
|
||||
"set": {
|
||||
"family": "inet",
|
||||
"table": "host",
|
||||
"name": "admins",
|
||||
"type": "ipv4_addr",
|
||||
"flags": ["interval"],
|
||||
"elem": [{"prefix": {"addr": "198.51.100.0", "len": 24}}],
|
||||
}
|
||||
},
|
||||
{
|
||||
"chain": {
|
||||
"family": "inet",
|
||||
"table": "host",
|
||||
"name": "input",
|
||||
"type": "filter",
|
||||
"hook": "input",
|
||||
"prio": 10,
|
||||
"policy": "drop",
|
||||
}
|
||||
},
|
||||
{"chain": {"family": "inet", "table": "host", "name": "services"}},
|
||||
{
|
||||
"rule": {
|
||||
"family": "inet",
|
||||
"table": "host",
|
||||
"chain": "input",
|
||||
"expr": [
|
||||
{
|
||||
"match": {
|
||||
"op": "in",
|
||||
"left": {"ct": {"key": "state"}},
|
||||
"right": ["established", "related"],
|
||||
}
|
||||
},
|
||||
{"accept": None},
|
||||
],
|
||||
}
|
||||
},
|
||||
{
|
||||
"rule": {
|
||||
"family": "inet",
|
||||
"table": "host",
|
||||
"chain": "input",
|
||||
"expr": [
|
||||
{"match": {"op": "==", "left": {"meta": {"key": "iif"}}, "right": "lo"}},
|
||||
{"accept": None},
|
||||
],
|
||||
}
|
||||
},
|
||||
{
|
||||
"rule": {
|
||||
"family": "inet",
|
||||
"table": "host",
|
||||
"chain": "input",
|
||||
"expr": [
|
||||
{
|
||||
"match": {
|
||||
"op": "==",
|
||||
"left": {"payload": {"protocol": "ip", "field": "saddr"}},
|
||||
"right": "@admins",
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"op": "==",
|
||||
"left": {"payload": {"protocol": "tcp", "field": "dport"}},
|
||||
"right": {"set": [22, {"range": [8000, 8010]}]},
|
||||
}
|
||||
},
|
||||
{"accept": None},
|
||||
],
|
||||
}
|
||||
},
|
||||
{
|
||||
"rule": {
|
||||
"family": "inet",
|
||||
"table": "host",
|
||||
"chain": "input",
|
||||
"expr": [
|
||||
{"match": {"op": "!=", "left": {"meta": {"key": "iifname"}}, "right": "wg*"}},
|
||||
{
|
||||
"match": {
|
||||
"op": "==",
|
||||
"left": {"payload": {"protocol": "ip6", "field": "daddr"}},
|
||||
"right": {"prefix": {"addr": "2001:db8::", "len": 32}},
|
||||
}
|
||||
},
|
||||
{"jump": {"target": "services"}},
|
||||
],
|
||||
}
|
||||
},
|
||||
{
|
||||
"rule": {
|
||||
"family": "inet",
|
||||
"table": "host",
|
||||
"chain": "input",
|
||||
"expr": [
|
||||
{"match": {"op": "==", "left": {"meta": {"key": "l4proto"}}, "right": "udp"}},
|
||||
{"limit": {"rate": 10, "per": "second"}},
|
||||
{"log": {"prefix": "udp "}},
|
||||
{"reject": {"type": "icmpx", "expr": "port-unreachable"}},
|
||||
],
|
||||
}
|
||||
},
|
||||
{
|
||||
"rule": {
|
||||
"family": "inet",
|
||||
"table": "host",
|
||||
"chain": "input",
|
||||
"expr": [
|
||||
{
|
||||
"match": {
|
||||
"op": "==",
|
||||
"left": {"fib": {"result": "type", "flags": ["daddr"]}},
|
||||
"right": "local",
|
||||
}
|
||||
},
|
||||
{"goto": {"target": "services"}},
|
||||
],
|
||||
}
|
||||
},
|
||||
{
|
||||
"rule": {
|
||||
"family": "inet",
|
||||
"table": "host",
|
||||
"chain": "services",
|
||||
"expr": [
|
||||
{
|
||||
"match": {
|
||||
"op": "==",
|
||||
"left": {"payload": {"protocol": "udp", "field": "dport"}},
|
||||
"right": 53,
|
||||
}
|
||||
},
|
||||
{"queue": {"num": 1}},
|
||||
],
|
||||
}
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def _report(
|
||||
*,
|
||||
nft: str | None = NFT,
|
||||
nft_rc: int = 0,
|
||||
ipt: str | None = IPTABLES_NFT,
|
||||
ip6t: str | None = IP6TABLES_NFT,
|
||||
) -> str:
|
||||
parts = ["$ sh -c ...", "FW_BEGIN"]
|
||||
if nft is not None:
|
||||
parts += ["[nft]", nft, f"__FW_RC={nft_rc}"]
|
||||
if ipt is not None:
|
||||
parts += ["[iptables-save]", ipt, "__FW_RC=0"]
|
||||
if ip6t is not None:
|
||||
parts += ["[ip6tables-save]", ip6t, "__FW_RC=0"]
|
||||
parts.append("FW_END")
|
||||
return "\n".join(parts) + "\n"
|
||||
|
||||
|
||||
def _chain(reading, family, table, name):
|
||||
(chain,) = [
|
||||
c
|
||||
for c in reading["chains"]
|
||||
if (c["family"], c["table"], c["name"]) == (family, table, name)
|
||||
]
|
||||
return chain
|
||||
|
||||
|
||||
def _fields(rule):
|
||||
return [(m["field"], m["op"], m["values"]) for m in rule["matches"]]
|
||||
|
||||
|
||||
class TestTheSources:
|
||||
def test_iptables_tables_come_from_iptables_save_the_rest_from_nft(self):
|
||||
reading = parse_host_firewall(_report())
|
||||
|
||||
assert reading["sources"] == ["nftables", "iptables-nft"]
|
||||
tables = {(c["family"], c["table"]) for c in reading["chains"]}
|
||||
assert tables == {("ip", "filter"), ("ip", "nat"), ("ip6", "filter"), ("inet", "host")}
|
||||
# The filter table's INPUT is iptables-save's: its set match is spelled
|
||||
# out there, and nft's opaque copy is not added a second time.
|
||||
assert len(_chain(reading, "ip", "filter", "INPUT")["rules"]) == 7
|
||||
|
||||
def test_a_legacy_iptables_is_a_ruleset_of_its_own(self):
|
||||
legacy = IPTABLES_NFT.replace("(nf_tables)", "(legacy)")
|
||||
|
||||
reading = parse_host_firewall(_report(ipt=legacy, ip6t=None))
|
||||
|
||||
assert reading["sources"] == ["nftables", "iptables-legacy"]
|
||||
inputs = [c for c in reading["chains"] if c["name"] == "INPUT" and c["family"] == "ip"]
|
||||
assert len(inputs) == 2 # nft's own ip filter INPUT and legacy's both apply
|
||||
|
||||
def test_nft_alone(self):
|
||||
reading = parse_host_firewall(_report(ipt=None, ip6t=None))
|
||||
|
||||
assert reading["sources"] == ["nftables"]
|
||||
assert _chain(reading, "ip", "filter", "INPUT")["rules"][0]["matches"] == [
|
||||
{"field": "unknown", "op": "==", "values": ["xt match set"]}
|
||||
]
|
||||
|
||||
def test_iptables_alone_when_nft_fails(self):
|
||||
reading = parse_host_firewall(_report(nft="Operation not permitted", nft_rc=1))
|
||||
|
||||
assert reading["sources"] == ["iptables-nft"]
|
||||
|
||||
def test_no_tool_that_could_be_read(self):
|
||||
with pytest.raises(HostFirewallUnavailable):
|
||||
parse_host_firewall(_report(nft=None, ipt=None, ip6t=None))
|
||||
|
||||
def test_a_failed_reading_is_no_reading(self):
|
||||
with pytest.raises(HostFirewallUnavailable):
|
||||
parse_host_firewall(_report(nft="permission denied", nft_rc=1, ipt=None, ip6t=None))
|
||||
|
||||
def test_a_report_cut_short_raises(self):
|
||||
with pytest.raises(ValueError):
|
||||
parse_host_firewall(_report().replace("FW_END", ""))
|
||||
|
||||
def test_an_empty_ruleset_is_a_host_without_rules(self):
|
||||
reading = parse_host_firewall(_report(nft=_nft(), ipt=None, ip6t=None))
|
||||
|
||||
assert reading == {"sources": ["nftables"], "chains": []}
|
||||
|
||||
|
||||
class TestIptablesSave:
|
||||
def setup_method(self):
|
||||
self.reading = parse_host_firewall(_report(nft=None))
|
||||
|
||||
def test_a_built_in_chain_has_its_hook_and_policy(self):
|
||||
chain = _chain(self.reading, "ip", "filter", "INPUT")
|
||||
assert (chain["type"], chain["hook"], chain["priority"], chain["policy"]) == (
|
||||
"filter",
|
||||
"input",
|
||||
0,
|
||||
"accept",
|
||||
)
|
||||
assert _chain(self.reading, "ip6", "filter", "INPUT")["policy"] == "drop"
|
||||
nat = _chain(self.reading, "ip", "nat", "PREROUTING")
|
||||
assert (nat["type"], nat["hook"], nat["priority"]) == ("nat", "prerouting", -100)
|
||||
|
||||
def test_a_user_chain_has_none(self):
|
||||
chain = _chain(self.reading, "ip", "filter", "WGRUNNER")
|
||||
assert (chain["type"], chain["hook"], chain["priority"], chain["policy"]) == (
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
)
|
||||
|
||||
def test_matches_and_verdicts(self):
|
||||
rules = _chain(self.reading, "ip", "filter", "INPUT")["rules"]
|
||||
assert _fields(rules[0]) == [("saddr", "==", ["10.255.255.4/32"])]
|
||||
assert (rules[0]["verdict"], rules[0]["target"]) == ("jump", "WGRUNNER")
|
||||
assert _fields(rules[1]) == [
|
||||
("iifname", "==", ["ens6"]),
|
||||
("l4proto", "==", ["udp"]),
|
||||
("dport", "==", ["51820"]),
|
||||
]
|
||||
assert rules[1]["verdict"] == "accept"
|
||||
assert _fields(rules[2]) == [
|
||||
("unknown", "==", ["-m set --match-set crowdsec-blacklists src"]),
|
||||
]
|
||||
assert _fields(rules[3]) == [
|
||||
("iifname", "==", ["ens6"]),
|
||||
("ct_state", "==", ["related", "established"]),
|
||||
]
|
||||
assert _fields(rules[4]) == [
|
||||
("iifname", "==", ["ens6"]),
|
||||
("l4proto", "==", ["tcp"]),
|
||||
("dport", "==", ["80", "443", "8000-8010"]),
|
||||
]
|
||||
assert _fields(rules[5]) == [
|
||||
("iifname", "!=", ["lo"]),
|
||||
("l4proto", "==", ["tcp"]),
|
||||
("ct_state", "==", ["new"]),
|
||||
("ct_orig_dport", "==", ["2222"]),
|
||||
]
|
||||
assert rules[5]["verdict"] == "continue" # LOG decides nothing
|
||||
assert rules[6]["verdict"] == "drop"
|
||||
|
||||
def test_negation_wildcard_and_goto(self):
|
||||
(rule,) = _chain(self.reading, "ip", "filter", "DOCKER-USER")["rules"]
|
||||
assert _fields(rule) == [("iifname", "==", ["eth*"]), ("saddr", "!=", ["192.168.0.0/16"])]
|
||||
assert (rule["verdict"], rule["target"]) == ("goto", "WGRUNNER")
|
||||
|
||||
def test_reject_return_and_nat_targets(self):
|
||||
rules = _chain(self.reading, "ip", "filter", "WGRUNNER")["rules"]
|
||||
assert [r["verdict"] for r in rules] == ["accept", "reject", "return"]
|
||||
(dnat,) = _chain(self.reading, "ip", "nat", "DOCKER")["rules"]
|
||||
assert (dnat["verdict"], dnat["target"]) == ("other", "DNAT")
|
||||
assert _chain(self.reading, "ip", "nat", "PREROUTING")["rules"][0]["matches"] == [
|
||||
{"field": "unknown", "op": "==", "values": ["-m addrtype --dst-type LOCAL"]}
|
||||
]
|
||||
|
||||
def test_the_line_is_kept_as_the_rules_text(self):
|
||||
rule = _chain(self.reading, "ip", "filter", "INPUT")["rules"][6]
|
||||
assert rule["text"] == "-A INPUT -i ens6 -j DROP"
|
||||
|
||||
|
||||
class TestNftJson:
|
||||
def setup_method(self):
|
||||
self.chains = {
|
||||
c["name"]: c
|
||||
for c in parse_host_firewall(_report(ipt=None, ip6t=None))["chains"]
|
||||
if c["table"] == "host"
|
||||
}
|
||||
|
||||
def test_a_base_chain_and_a_regular_one(self):
|
||||
base, regular = self.chains["input"], self.chains["services"]
|
||||
assert (base["family"], base["type"], base["hook"], base["priority"], base["policy"]) == (
|
||||
"inet",
|
||||
"filter",
|
||||
"input",
|
||||
10,
|
||||
"drop",
|
||||
)
|
||||
assert (regular["hook"], regular["policy"]) == (None, None)
|
||||
|
||||
def test_matches(self):
|
||||
rules = self.chains["input"]["rules"]
|
||||
assert _fields(rules[0]) == [("ct_state", "==", ["established", "related"])]
|
||||
assert _fields(rules[1]) == [("iifname", "==", ["lo"])]
|
||||
# A named set is resolved from its elements; tcp dport implies tcp.
|
||||
assert _fields(rules[2]) == [
|
||||
("saddr", "==", ["198.51.100.0/24"]),
|
||||
("l4proto", "==", ["tcp"]),
|
||||
("dport", "==", ["22", "8000-8010"]),
|
||||
]
|
||||
assert _fields(rules[3]) == [("iifname", "!=", ["wg*"]), ("daddr", "==", ["2001:db8::/32"])]
|
||||
assert (rules[3]["verdict"], rules[3]["target"]) == ("jump", "services")
|
||||
|
||||
def test_a_statement_that_takes_only_some_packets_is_unknown(self):
|
||||
rule = self.chains["input"]["rules"][4]
|
||||
assert _fields(rule) == [("l4proto", "==", ["udp"]), ("unknown", "==", ["limit"])]
|
||||
assert rule["verdict"] == "reject"
|
||||
|
||||
def test_an_expression_it_does_not_model_is_unknown(self):
|
||||
rule = self.chains["input"]["rules"][5]
|
||||
assert rule["matches"][0]["field"] == "unknown"
|
||||
assert (rule["verdict"], rule["target"]) == ("goto", "services")
|
||||
|
||||
def test_a_verdict_it_does_not_model_is_other(self):
|
||||
(rule,) = self.chains["services"]["rules"]
|
||||
assert rule["verdict"] == "other"
|
||||
|
||||
def test_each_rule_reads_as_a_line(self):
|
||||
assert self.chains["input"]["rules"][2]["text"] == (
|
||||
"ip saddr 198.51.100.0/24 meta l4proto tcp tcp dport {22, 8000-8010} accept"
|
||||
)
|
||||
|
||||
def test_an_unresolvable_set_is_unknown(self):
|
||||
nft = NFT.replace('"@admins"', '"@missing"')
|
||||
rules = [
|
||||
c
|
||||
for c in parse_host_firewall(_report(nft=nft, ipt=None, ip6t=None))["chains"]
|
||||
if c["name"] == "input"
|
||||
][0]["rules"]
|
||||
assert rules[2]["matches"][0] == {
|
||||
"field": "unknown",
|
||||
"op": "==",
|
||||
"values": ["ip saddr @missing"],
|
||||
}
|
||||
|
||||
|
||||
class _Host(HostFirewallMixin):
|
||||
def __init__(self, output: str | None = None, fail: Exception | None = None):
|
||||
self.calls: list[tuple[str, bool]] = []
|
||||
self._output, self._fail = output, fail
|
||||
|
||||
def _run_host_firewall_command(self, command: str, *, privileged: bool) -> str:
|
||||
self.calls.append((command, privileged))
|
||||
if self._fail:
|
||||
raise self._fail
|
||||
return self._output or ""
|
||||
|
||||
|
||||
class TestTheMixin:
|
||||
def test_one_privileged_command(self):
|
||||
host = _Host(_report())
|
||||
|
||||
reading = host.get_host_firewall()
|
||||
|
||||
assert host.calls == [(f"sh -c {quote(HOST_FIREWALL_COMMAND)}", True)]
|
||||
assert reading["sources"] == ["nftables", "iptables-nft"]
|
||||
|
||||
def test_without_root_there_is_nothing_to_read(self):
|
||||
# nft and iptables-save both need CAP_NET_ADMIN; a reading without root
|
||||
# would be an empty ruleset that is not the host's.
|
||||
host = _Host("FW_BEGIN\n[nft]\nnetlink: Operation not permitted\n__FW_RC=1\nFW_END\n")
|
||||
|
||||
with pytest.raises(HostFirewallUnavailable):
|
||||
host.get_host_firewall()
|
||||
assert len(host.calls) == 1
|
||||
|
||||
def test_the_command_is_framed_and_read_only(self):
|
||||
assert "nft -j list ruleset" in HOST_FIREWALL_COMMAND
|
||||
assert "iptables-save" in HOST_FIREWALL_COMMAND
|
||||
for word in (" -F", " -X", "flush", "delete", "-A ", "-I ", "restore"):
|
||||
assert word not in HOST_FIREWALL_COMMAND
|
||||
@@ -0,0 +1,27 @@
|
||||
"""shutdown_host: the contract for shutting the device itself down cleanly.
|
||||
|
||||
netOrk's graph workflows switch a hypervisor off when it has nothing to do
|
||||
(NetOrk/netork#1636): through its own API, never by pulling the power. Like
|
||||
``reboot_host`` it is declared, not implemented, so ``hasattr`` tells netOrk
|
||||
whether a driver can.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import inspect
|
||||
|
||||
from napalm_device_types import DeviceTypeDriver, HostShutdownMixin
|
||||
|
||||
|
||||
def test_every_device_type_driver_carries_the_declaration():
|
||||
assert issubclass(DeviceTypeDriver, HostShutdownMixin)
|
||||
|
||||
|
||||
def test_declared_not_implemented():
|
||||
assert not hasattr(DeviceTypeDriver, "shutdown_host")
|
||||
|
||||
|
||||
def test_declaration_documents_the_contract():
|
||||
source = inspect.getsource(HostShutdownMixin)
|
||||
assert "def shutdown_host(self) -> None" in source
|
||||
assert "RuntimeError" in source
|
||||
@@ -0,0 +1,572 @@
|
||||
"""Host status: does the host need a reboot, and does it patch itself?
|
||||
|
||||
A patch run that installed a new kernel has not closed anything until the host
|
||||
boots it, so "reboot required" is part of being patched. Whether the host
|
||||
installs updates on its own (unattended-upgrades, dnf-automatic) is what netOrk
|
||||
shows next to the window it governs. Both are read the same way on every Linux
|
||||
host, so the command and its parse live here once (netOrk MVP 5).
|
||||
|
||||
The fixtures are the real states of six hosts on netOrk's test server.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import OSDriver
|
||||
from napalm_device_types.host_status import (
|
||||
BUSY_CHECK,
|
||||
HOST_STATUS_COMMAND,
|
||||
HostStatusMixin,
|
||||
kernel_reboot_pending,
|
||||
parse_host_status,
|
||||
)
|
||||
|
||||
|
||||
def _wire(
|
||||
*,
|
||||
reboot_file=False,
|
||||
running="6.8.0-142-generic",
|
||||
modules=("6.8.0-139-generic", "6.8.0-142-generic"),
|
||||
needs_restarting=None,
|
||||
periodic=None,
|
||||
stamp=None,
|
||||
timer="enabled",
|
||||
unattended=True,
|
||||
dnf_timers=("not-found", "not-found"),
|
||||
dnf_conf=(),
|
||||
dpkg=None,
|
||||
journal=(),
|
||||
busy=False,
|
||||
):
|
||||
lines = ["HSTAT_BEGIN"]
|
||||
if reboot_file:
|
||||
lines.append("[reboot-required]")
|
||||
if needs_restarting is not None:
|
||||
lines += ["[needs-restarting]", str(needs_restarting)]
|
||||
lines += ["[kernel]", running, "[modules]", *modules]
|
||||
if periodic is not None:
|
||||
lines += ["[apt-config]", *periodic]
|
||||
if unattended:
|
||||
lines.append("[unattended-upgrade]")
|
||||
lines += ["[upgrade-stamp]", *([str(stamp)] if stamp else [])]
|
||||
if dpkg is not None:
|
||||
if busy:
|
||||
lines.append("[dpkg-busy]")
|
||||
lines += ["[dpkg-journal]", *journal, "[dpkg-audit]", *dpkg]
|
||||
lines += [
|
||||
"[timers]",
|
||||
f"apt-daily-upgrade.timer {timer}",
|
||||
f"dnf-automatic.timer {dnf_timers[0]}",
|
||||
f"dnf-automatic-install.timer {dnf_timers[1]}",
|
||||
"[dnf-automatic]",
|
||||
*dnf_conf,
|
||||
"HSTAT_END",
|
||||
]
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
UNATTENDED = ['APT::Periodic::Update-Package-Lists "1";', 'APT::Periodic::Unattended-Upgrade "1";']
|
||||
|
||||
|
||||
class TestRebootRequired:
|
||||
def test_the_reboot_required_file_says_so(self):
|
||||
status = parse_host_status(_wire(reboot_file=True))
|
||||
|
||||
assert status["reboot_required"] is True
|
||||
assert "reboot-required" in status["reboot_reason"]
|
||||
|
||||
def test_a_newer_installed_kernel_than_the_running_one(self):
|
||||
"""z2m-garden: running 6.8.0-139, 6.8.0-142 installed."""
|
||||
status = parse_host_status(
|
||||
_wire(
|
||||
running="6.8.0-139-generic",
|
||||
modules=("6.8.0-87-generic", "6.8.0-139-generic", "6.8.0-142-generic"),
|
||||
)
|
||||
)
|
||||
|
||||
assert status["reboot_required"] is True
|
||||
assert "6.8.0-142-generic" in status["reboot_reason"]
|
||||
|
||||
def test_the_newest_kernel_running_needs_none(self):
|
||||
"""vault-01: 6.8.0-142 running and newest; 6.8.0-94 sorts below it."""
|
||||
status = parse_host_status(
|
||||
_wire(running="6.8.0-142-generic", modules=("6.8.0-94-generic", "6.8.0-142-generic"))
|
||||
)
|
||||
|
||||
assert status["reboot_required"] is False
|
||||
assert status["reboot_reason"] is None
|
||||
|
||||
def test_needs_restarting_exit_1_means_reboot(self):
|
||||
assert parse_host_status(_wire(needs_restarting=1))["reboot_required"] is True
|
||||
|
||||
def test_needs_restarting_exit_0_does_not(self):
|
||||
assert parse_host_status(_wire(needs_restarting=0))["reboot_required"] is False
|
||||
|
||||
def test_no_kernel_information_is_unknown(self):
|
||||
"""A container has no /lib/modules of its own."""
|
||||
status = parse_host_status(_wire(modules=()))
|
||||
|
||||
assert status["reboot_required"] is None
|
||||
|
||||
|
||||
class TestKernelRebootPending:
|
||||
@pytest.mark.parametrize(
|
||||
("running", "installed", "newer"),
|
||||
[
|
||||
# Raspberry Pi: two flavours side by side; only the running one counts.
|
||||
(
|
||||
"6.18.33+rpt-rpi-v8",
|
||||
[
|
||||
"6.12.75+rpt-rpi-2712",
|
||||
"6.12.75+rpt-rpi-v8",
|
||||
"6.18.33+rpt-rpi-2712",
|
||||
"6.18.33+rpt-rpi-v8",
|
||||
],
|
||||
None,
|
||||
),
|
||||
# Debian (OMV): 7.1.8 installed while 7.1.3 runs.
|
||||
(
|
||||
"7.1.3+deb13-amd64",
|
||||
["6.12.57+deb13-amd64", "7.1.3+deb13-amd64", "7.1.8+deb13-amd64"],
|
||||
"7.1.8+deb13-amd64",
|
||||
),
|
||||
# Proxmox: 7.0.14-19 is newer than 7.0.2-6, numerically.
|
||||
("7.0.14-19-pve", ["7.0.14-19-pve", "7.0.2-6-pve"], None),
|
||||
# Arch: the running kernel's modules were replaced by the upgrade.
|
||||
("6.10.5-arch1-1", ["6.10.9-arch1-1"], "6.10.9-arch1-1"),
|
||||
# RHEL/Fedora: the release changes with every update, the arch does not.
|
||||
(
|
||||
"5.14.0-427.13.1.el9_4.x86_64",
|
||||
["5.14.0-427.13.1.el9_4.x86_64", "5.14.0-503.11.1.el9_5.x86_64"],
|
||||
"5.14.0-503.11.1.el9_5.x86_64",
|
||||
),
|
||||
("6.9.7-200.fc40.x86_64", ["6.9.7-200.fc40.x86_64"], None),
|
||||
],
|
||||
)
|
||||
def test_the_newer_kernel_of_the_running_flavour(self, running, installed, newer):
|
||||
assert kernel_reboot_pending(running, installed) == newer
|
||||
|
||||
|
||||
class TestAutoUpdates:
|
||||
def test_unattended_upgrades_switched_on(self):
|
||||
assert parse_host_status(_wire(periodic=UNATTENDED))["auto_updates"] is True
|
||||
|
||||
def test_apt_without_the_setting_does_not_patch_itself(self):
|
||||
"""Proxmox and Raspberry Pi OS: apt-config answers, the setting is absent."""
|
||||
assert parse_host_status(_wire(periodic=[]))["auto_updates"] is False
|
||||
|
||||
def test_switched_off_by_zero(self):
|
||||
off = ['APT::Periodic::Unattended-Upgrade "0";']
|
||||
|
||||
assert parse_host_status(_wire(periodic=off))["auto_updates"] is False
|
||||
|
||||
def test_a_disabled_timer_stops_it_even_when_configured(self):
|
||||
status = parse_host_status(_wire(periodic=UNATTENDED, timer="disabled"))
|
||||
|
||||
assert status["auto_updates"] is False
|
||||
|
||||
def test_removed_but_not_purged_does_not_patch_itself(self):
|
||||
"""``apt remove unattended-upgrades`` leaves its conffile, and its "1", behind."""
|
||||
status = parse_host_status(_wire(periodic=UNATTENDED, unattended=False))
|
||||
|
||||
assert status["auto_updates"] is False
|
||||
|
||||
def test_dnf_automatic_install_timer_always_installs(self):
|
||||
status = parse_host_status(_wire(dnf_timers=("not-found", "enabled")))
|
||||
|
||||
assert status["auto_updates"] is True
|
||||
|
||||
def test_dnf_automatic_with_apply_updates(self):
|
||||
conf = ["apply_updates = yes"]
|
||||
status = parse_host_status(_wire(dnf_timers=("enabled", "not-found"), dnf_conf=conf))
|
||||
|
||||
assert status["auto_updates"] is True
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"conf", [[], ["apply_updates = no"], ["apply_updates=yes", "apply_updates = False"]]
|
||||
)
|
||||
def test_dnf_automatic_by_default_only_downloads(self, conf):
|
||||
"""dnf-automatic.timer installs only with apply_updates; the default is no.
|
||||
dnf5's file comes last and wins."""
|
||||
status = parse_host_status(_wire(dnf_timers=("enabled", "not-found"), dnf_conf=conf))
|
||||
|
||||
assert status["auto_updates"] is False
|
||||
|
||||
def test_the_command_reads_apply_updates_of_dnf4_then_dnf5(self):
|
||||
conf = "/etc/dnf/automatic.conf /etc/dnf/dnf5-plugins/automatic.conf"
|
||||
assert conf in HOST_STATUS_COMMAND
|
||||
|
||||
@pytest.mark.parametrize("installed", [True, False])
|
||||
def test_the_command_asks_whether_unattended_upgrade_is_installed(self, tmp_path, installed):
|
||||
"""Run for real, with a stand-in apt-config that still says "1"."""
|
||||
if os.path.exists("/usr/bin/unattended-upgrade"):
|
||||
pytest.skip("this host has unattended-upgrade installed")
|
||||
apt_config = tmp_path / "apt-config"
|
||||
apt_config.write_text("#!/bin/sh\necho 'APT::Periodic::Unattended-Upgrade \"1\";'\n")
|
||||
stubs = [apt_config]
|
||||
if installed:
|
||||
stubs.append(tmp_path / "unattended-upgrade")
|
||||
stubs[-1].write_text("#!/bin/sh\nexit 0\n")
|
||||
for stub in stubs:
|
||||
stub.chmod(0o755)
|
||||
env = {**os.environ, "PATH": f"{tmp_path}:{os.environ.get('PATH', '/usr/bin:/bin')}"}
|
||||
out = subprocess.run(
|
||||
["sh", "-c", HOST_STATUS_COMMAND], capture_output=True, text=True, env=env
|
||||
).stdout
|
||||
|
||||
assert ("[unattended-upgrade]" in out) is installed
|
||||
|
||||
def test_neither_apt_nor_dnf_is_unknown(self):
|
||||
assert parse_host_status(_wire())["auto_updates"] is None
|
||||
|
||||
|
||||
class TestTheReport:
|
||||
def test_a_cut_short_report_raises(self):
|
||||
with pytest.raises(ValueError):
|
||||
parse_host_status(_wire().replace("HSTAT_END\n", ""))
|
||||
|
||||
def test_the_frame_is_not_in_the_command_itself(self):
|
||||
assert "HSTAT_BEGIN" not in HOST_STATUS_COMMAND
|
||||
assert "HSTAT_END" not in HOST_STATUS_COMMAND
|
||||
|
||||
def test_it_runs_without_a_terminal(self):
|
||||
"""No colour codes from ls, nothing a screen scraper could take for a prompt."""
|
||||
assert HOST_STATUS_COMMAND.rstrip().endswith("| cat")
|
||||
|
||||
def test_terminal_codes_are_dropped(self):
|
||||
status = parse_host_status(
|
||||
"\x1b[0m" + _wire(reboot_file=True).replace("HSTAT_END", "\x1b>HSTAT_END")
|
||||
)
|
||||
|
||||
assert status["reboot_required"] is True
|
||||
|
||||
def test_it_changes_nothing(self):
|
||||
for word in ("rm ", "apt-get ", "dnf install", "systemctl start", "reboot"):
|
||||
assert word not in HOST_STATUS_COMMAND.replace("reboot-required", "")
|
||||
|
||||
|
||||
class _Driver(HostStatusMixin):
|
||||
def __init__(self, reply: str) -> None:
|
||||
self.reply = reply
|
||||
self.commands: list = []
|
||||
|
||||
def _run_host_status_command(self, command: str) -> str:
|
||||
self.commands.append(command)
|
||||
return self.reply
|
||||
|
||||
|
||||
class TestHostStatusMixin:
|
||||
def test_a_driver_supplies_only_the_transport(self):
|
||||
driver = _Driver(_wire(reboot_file=True, periodic=UNATTENDED))
|
||||
|
||||
status = driver.get_host_status()
|
||||
|
||||
assert driver.commands == [HOST_STATUS_COMMAND]
|
||||
assert (status["reboot_required"], status["auto_updates"]) == (True, True)
|
||||
|
||||
def test_not_every_os_driver_has_it(self):
|
||||
assert not hasattr(OSDriver, "get_host_status")
|
||||
|
||||
|
||||
class TestFreeBSD:
|
||||
"""freebsd-version -k names the installed kernel, -r the running one; a
|
||||
patch that replaced the kernel shows as the two differing until the host
|
||||
boots it. Recorded on a FreeBSD 15.1 host (NetOrk/netork#799)."""
|
||||
|
||||
#: What the command printed there before this section existed.
|
||||
BEFORE = (
|
||||
"HSTAT_BEGIN\n[kernel]\n15.1-RELEASE\n[modules]\n[timers]\n"
|
||||
"apt-daily-upgrade.timer \ndnf-automatic.timer \ndnf-automatic-install.timer \n"
|
||||
"HSTAT_END\n"
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _report(installed: str, running: str) -> str:
|
||||
return (
|
||||
f"HSTAT_BEGIN\n[kernel]\n{running}\n[modules]\n"
|
||||
f"[freebsd-kernel]\n{installed}\n[freebsd-running]\n{running}\n"
|
||||
"[timers]\napt-daily-upgrade.timer \nHSTAT_END\n"
|
||||
)
|
||||
|
||||
def test_without_the_section_it_is_unknown(self):
|
||||
assert parse_host_status(self.BEFORE)["reboot_required"] is None
|
||||
|
||||
def test_the_running_kernel_is_the_installed_one(self):
|
||||
status = parse_host_status(self._report("15.1-RELEASE-p4", "15.1-RELEASE-p4"))
|
||||
assert status["reboot_required"] is False
|
||||
assert status["reboot_reason"] is None
|
||||
|
||||
def test_a_patched_kernel_waits_for_a_reboot(self):
|
||||
status = parse_host_status(self._report("15.1-RELEASE-p5", "15.1-RELEASE-p4"))
|
||||
assert status["reboot_required"] is True
|
||||
assert status["reboot_reason"] == (
|
||||
"kernel 15.1-RELEASE-p5 installed, 15.1-RELEASE-p4 running"
|
||||
)
|
||||
|
||||
def test_freebsd_does_not_patch_itself_as_far_as_netork_can_tell(self):
|
||||
status = parse_host_status(self._report("15.1-RELEASE", "15.1-RELEASE"))
|
||||
assert status["auto_updates"] is None
|
||||
|
||||
def test_the_command_asks_only_where_freebsd_version_exists(self):
|
||||
assert "command -v freebsd-version" in HOST_STATUS_COMMAND
|
||||
assert "freebsd-version -k" in HOST_STATUS_COMMAND
|
||||
assert "freebsd-version -r" in HOST_STATUS_COMMAND
|
||||
|
||||
|
||||
class TestInterruptedDpkg:
|
||||
"""dpkg left halfway: apt refuses to install anything until
|
||||
``dpkg --configure -a`` has run, and the daily apt timers report success
|
||||
all the same (NetOrk/netork#738).
|
||||
|
||||
The fixture is aris (Ubuntu 24.04): unattended-upgrades was cut off on
|
||||
2026-07-03 while update-initramfs built the initrd for 6.8.0-134, and the
|
||||
kernel package stayed half-configured for three months.
|
||||
"""
|
||||
|
||||
ARIS = ("iF linux-image-6.8.0-134-generic",)
|
||||
|
||||
def test_a_half_configured_kernel_is_an_interrupted_dpkg(self):
|
||||
status = parse_host_status(_wire(dpkg=self.ARIS))
|
||||
|
||||
assert status["package_manager_interrupted"] is True
|
||||
assert status["interrupted_packages"] == [
|
||||
{"name": "linux-image-6.8.0-134-generic", "state": "half-configured"}
|
||||
]
|
||||
|
||||
def test_a_clean_dpkg_is_not_interrupted(self):
|
||||
status = parse_host_status(_wire(dpkg=()))
|
||||
|
||||
assert status["package_manager_interrupted"] is False
|
||||
assert status["interrupted_packages"] == []
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("line", "state"),
|
||||
[
|
||||
("iH base-files", "half-installed"),
|
||||
("iU base-files", "unpacked"),
|
||||
("iF base-files", "half-configured"),
|
||||
("iW base-files", "triggers-awaited"),
|
||||
("it base-files", "triggers-pending"),
|
||||
("iUR base-files", "reinstall-required"),
|
||||
("hF base-files", "half-configured"),
|
||||
],
|
||||
)
|
||||
def test_every_state_an_interrupted_run_leaves(self, line, state):
|
||||
status = parse_host_status(_wire(dpkg=(line,)))
|
||||
|
||||
assert status["interrupted_packages"] == [{"name": "base-files", "state": state}]
|
||||
|
||||
def test_a_journal_left_behind_is_interrupted_on_its_own(self):
|
||||
"""What apt itself checks: numbered files in /var/lib/dpkg/updates."""
|
||||
status = parse_host_status(_wire(dpkg=(), journal=("0000", "0001")))
|
||||
|
||||
assert status["package_manager_interrupted"] is True
|
||||
assert status["interrupted_packages"] == []
|
||||
|
||||
def test_only_numbered_journal_files_count(self):
|
||||
"""apt ignores anything else there, such as dpkg's tmp.i."""
|
||||
status = parse_host_status(_wire(dpkg=(), journal=("tmp.i",)))
|
||||
|
||||
assert status["package_manager_interrupted"] is False
|
||||
|
||||
def test_a_running_package_manager_makes_it_unknown(self):
|
||||
"""Mid-upgrade, packages are unpacked and not yet configured, also
|
||||
between two of apt's dpkg calls. That is work in progress, not an
|
||||
interruption."""
|
||||
status = parse_host_status(_wire(dpkg=("iU base-files",), busy=True))
|
||||
|
||||
assert status["package_manager_interrupted"] is None
|
||||
assert status["interrupted_packages"] == []
|
||||
|
||||
def test_without_dpkg_it_is_unknown(self):
|
||||
status = parse_host_status(_wire())
|
||||
|
||||
assert status["package_manager_interrupted"] is None
|
||||
assert status["interrupted_packages"] == []
|
||||
|
||||
def test_the_command_asks_dpkg_only_where_it_exists(self):
|
||||
assert "command -v dpkg-query" in HOST_STATUS_COMMAND
|
||||
assert "/var/lib/dpkg/updates" in HOST_STATUS_COMMAND
|
||||
|
||||
def test_it_never_repairs(self):
|
||||
assert "--configure" not in HOST_STATUS_COMMAND
|
||||
|
||||
|
||||
class TestAutoUpdatesLastSuccess:
|
||||
"""apt.systemd.daily touches upgrade-stamp only after unattended-upgrade
|
||||
succeeded. update-success-stamp says nothing: netOrk's own daily
|
||||
``apt-get update`` touches it as well (NetOrk/netork#738)."""
|
||||
|
||||
def test_the_upgrade_stamp_is_the_last_successful_run(self):
|
||||
status = parse_host_status(_wire(periodic=UNATTENDED, stamp=1751525820))
|
||||
|
||||
assert status["auto_updates_last_success"] == 1751525820
|
||||
assert status["auto_updates_interval"] == 86400
|
||||
|
||||
def test_no_stamp_is_unknown(self):
|
||||
status = parse_host_status(_wire(periodic=UNATTENDED))
|
||||
|
||||
assert status["auto_updates_last_success"] is None
|
||||
|
||||
def test_without_apt_both_are_unknown(self):
|
||||
status = parse_host_status(_wire())
|
||||
|
||||
assert status["auto_updates_last_success"] is None
|
||||
assert status["auto_updates_interval"] is None
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("value", "seconds"),
|
||||
[
|
||||
('"1"', 86400),
|
||||
('"7d"', 7 * 86400),
|
||||
('"12h"', 43200),
|
||||
('"30m"', 1800),
|
||||
('"90s"', 90),
|
||||
('"always"', 0),
|
||||
('"0"', None),
|
||||
('"weekly"', None),
|
||||
],
|
||||
)
|
||||
def test_the_interval_in_apts_units(self, value, seconds):
|
||||
periodic = [f"APT::Periodic::Unattended-Upgrade {value};"]
|
||||
|
||||
assert parse_host_status(_wire(periodic=periodic))["auto_updates_interval"] == seconds
|
||||
|
||||
def test_the_command_reads_the_upgrade_stamp_only(self):
|
||||
assert "/var/lib/apt/periodic/upgrade-stamp" in HOST_STATUS_COMMAND
|
||||
assert "update-success-stamp" not in HOST_STATUS_COMMAND
|
||||
|
||||
|
||||
class TestBusyCheck:
|
||||
"""Whether a package manager is working right now, run through ``sh`` with a
|
||||
stand-in ``ps``. On Ubuntu, unattended-upgrades keeps
|
||||
``unattended-upgrade-shutdown --wait-for-signal`` running all the time; its
|
||||
process name is cut to 15 characters, ``unattended-upgr``, the same as a real
|
||||
run's. Taken for a run, it made every such host's dpkg state unknown -- aris
|
||||
among them (NetOrk/netork#738)."""
|
||||
|
||||
UBUNTU_IDLE = [
|
||||
("systemd", "/sbin/init"),
|
||||
(
|
||||
"unattended-upgr",
|
||||
"/usr/bin/python3 /usr/share/unattended-upgrades/unattended-upgrade-shutdown --wait-for-signal",
|
||||
),
|
||||
("sshd", "sshd: netork [priv]"),
|
||||
]
|
||||
|
||||
@staticmethod
|
||||
def _busy(tmp_path, processes) -> bool:
|
||||
comm = "\n".join(c for c, _ in processes)
|
||||
args = "\n".join(a for _, a in processes)
|
||||
ps = tmp_path / "ps"
|
||||
ps.write_text(
|
||||
"#!/bin/sh\n"
|
||||
f"case \"$*\" in *comm=*) printf '%s\\n' '{comm}';; *) printf '%s\\n' '{args}';; esac\n"
|
||||
)
|
||||
ps.chmod(0o755)
|
||||
env = {"PATH": f"{tmp_path}:/usr/bin:/bin"}
|
||||
out = subprocess.run(["sh", "-c", BUSY_CHECK], capture_output=True, text=True, env=env)
|
||||
return "[dpkg-busy]" in out.stdout
|
||||
|
||||
def test_the_shutdown_helper_of_an_idle_ubuntu_host_is_no_run(self, tmp_path):
|
||||
assert self._busy(tmp_path, self.UBUNTU_IDLE) is False
|
||||
|
||||
def test_unattended_upgrade_running(self, tmp_path):
|
||||
run = ("unattended-upgr", "/usr/bin/python3 /usr/bin/unattended-upgrade")
|
||||
|
||||
assert self._busy(tmp_path, [*self.UBUNTU_IDLE, run]) is True
|
||||
|
||||
@pytest.mark.parametrize("name", ["dpkg", "apt", "apt-get", "aptitude"])
|
||||
def test_dpkg_and_apt(self, tmp_path, name):
|
||||
assert self._busy(tmp_path, [*self.UBUNTU_IDLE, (name, f"/usr/bin/{name} upgrade")]) is True
|
||||
|
||||
def test_it_is_part_of_the_command(self):
|
||||
assert BUSY_CHECK in HOST_STATUS_COMMAND
|
||||
|
||||
def test_the_command_does_not_match_itself(self):
|
||||
"""Run over an exec channel, the command is the command line of the shell
|
||||
that runs it, and ``ps -o args`` lists it."""
|
||||
assert not re.search(r"(^|[ /])unattended-upgrade( |$)", HOST_STATUS_COMMAND)
|
||||
|
||||
|
||||
class TestFilesystems:
|
||||
"""How full the filesystems an upgrade writes to are: ``/``, ``/boot`` and
|
||||
``/var``, read with ``df -Pk`` and no root.
|
||||
|
||||
pve-dual's root filesystem stood at 94 % with about 400 MB free for a week;
|
||||
a kernel upgrade failed on it with "No space left on device" and left dpkg
|
||||
halfway, and nothing had said the disk was full (NetOrk/netork#862). The
|
||||
figures below are shaped after it (6.5 GiB, 400 MiB available): ext4 keeps
|
||||
5 % for root, so what is available is less than total minus used.
|
||||
"""
|
||||
|
||||
HEADER = "Filesystem 1024-blocks Used Available Capacity Mounted on"
|
||||
PVE_ROOT = "/dev/mapper/pve-root 6815744 6065357 409600 94% /"
|
||||
BOOT = "/dev/sda2 1046512 104860 941652 11% /boot"
|
||||
|
||||
@staticmethod
|
||||
def _report(*df_lines: str) -> str:
|
||||
return _wire().replace("[timers]", "\n".join(["[filesystems]", *df_lines, "[timers]"]))
|
||||
|
||||
def test_the_root_filesystem_in_bytes(self):
|
||||
status = parse_host_status(self._report(self.HEADER, self.PVE_ROOT))
|
||||
|
||||
assert status["filesystems"] == [
|
||||
{
|
||||
"mountpoint": "/",
|
||||
"total": 6815744 * 1024,
|
||||
"used": 6065357 * 1024,
|
||||
"available": 409600 * 1024,
|
||||
}
|
||||
]
|
||||
|
||||
def test_a_separate_boot_partition(self):
|
||||
status = parse_host_status(self._report(self.HEADER, self.PVE_ROOT, self.BOOT))
|
||||
|
||||
assert [fs["mountpoint"] for fs in status["filesystems"]] == ["/", "/boot"]
|
||||
|
||||
def test_a_directory_on_root_is_root_once(self):
|
||||
"""``df /boot /var`` names the filesystem they live on, here ``/`` three times."""
|
||||
status = parse_host_status(
|
||||
self._report(self.HEADER, self.PVE_ROOT, self.PVE_ROOT, self.PVE_ROOT)
|
||||
)
|
||||
|
||||
assert [fs["mountpoint"] for fs in status["filesystems"]] == ["/"]
|
||||
|
||||
def test_a_long_device_name_and_a_blank_in_the_mountpoint(self):
|
||||
line = "/dev/disk/by-id/a very long name 1000 600 400 60% /mnt/my disk"
|
||||
|
||||
(fs,) = parse_host_status(self._report(self.HEADER, line))["filesystems"]
|
||||
|
||||
assert (fs["mountpoint"], fs["available"]) == ("/mnt/my disk", 400 * 1024)
|
||||
|
||||
def test_a_filesystem_without_size_is_left_out(self):
|
||||
"""What a pseudo filesystem reports; it cannot run full."""
|
||||
status = parse_host_status(self._report(self.HEADER, "none 0 0 0 - /"))
|
||||
|
||||
assert status["filesystems"] is None
|
||||
|
||||
def test_without_the_section_it_is_unknown(self):
|
||||
"""An older command, or no df: None, not "nothing is full"."""
|
||||
assert parse_host_status(_wire())["filesystems"] is None
|
||||
|
||||
def test_df_that_printed_nothing_is_unknown(self):
|
||||
assert parse_host_status(self._report())["filesystems"] is None
|
||||
|
||||
def test_the_command_asks_for_the_three_without_root(self):
|
||||
assert "df -Pk / /boot /var" in HOST_STATUS_COMMAND
|
||||
assert "sudo" not in HOST_STATUS_COMMAND
|
||||
|
||||
def test_the_command_reads_them_on_a_real_shell(self):
|
||||
"""POSIX df the way a host's sh runs it: ``/`` is always there, whatever it is."""
|
||||
out = subprocess.run(
|
||||
["sh", "-c", HOST_STATUS_COMMAND], capture_output=True, text=True, timeout=30
|
||||
)
|
||||
|
||||
mountpoints = [fs["mountpoint"] for fs in parse_host_status(out.stdout)["filesystems"]]
|
||||
assert "/" in mountpoints
|
||||
@@ -0,0 +1,159 @@
|
||||
"""get_kernel_facts: what the running kernel has built and loaded.
|
||||
|
||||
A kernel CVE's preconditions ask whether a module is loaded or a build option
|
||||
set. Reading that is the same on every Linux host -- one read-only command and
|
||||
its parse -- so both live here once, and a driver only carries the command
|
||||
across (#268 in netOrk).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import gzip
|
||||
import os
|
||||
import subprocess
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import OSDriver
|
||||
from napalm_device_types.kernel import (
|
||||
KERNEL_FACTS_COMMAND,
|
||||
KernelFactsMixin,
|
||||
module_name,
|
||||
parse_kernel_facts,
|
||||
)
|
||||
|
||||
REPORT = """[release]
|
||||
6.1.0-25-amd64
|
||||
[loaded]
|
||||
tipc
|
||||
nf_tables
|
||||
[builtin]
|
||||
kernel/net/ipv4/tcp_cubic.ko
|
||||
kernel/drivers/char/tpm/tpm-tis.ko
|
||||
[available]
|
||||
kernel/net/tipc/tipc.ko.xz
|
||||
kernel/net/can/can-raw.ko.zst
|
||||
kernel/net/netfilter/nf_tables.ko
|
||||
[config]
|
||||
CONFIG_TIPC=m
|
||||
CONFIG_BPF_JIT=y
|
||||
CONFIG_DEFAULT_HOSTNAME="(none)"
|
||||
CONFIG_HZ=250
|
||||
"""
|
||||
|
||||
|
||||
def _wire(report: str, *, noise: str = "") -> str:
|
||||
"""The report as the command prints it: framed, gzipped, base64 in lines."""
|
||||
payload = base64.encodebytes(gzip.compress(report.encode())).decode()
|
||||
return f"{noise}KFACTS_BEGIN\n{payload}KFACTS_END\n"
|
||||
|
||||
|
||||
class TestParsing:
|
||||
def test_every_section_is_read(self):
|
||||
facts = parse_kernel_facts(_wire(REPORT))
|
||||
|
||||
assert facts["release"] == "6.1.0-25-amd64"
|
||||
assert facts["loaded"] == ["nf_tables", "tipc"]
|
||||
assert facts["builtin"] == ["tcp_cubic", "tpm_tis"]
|
||||
assert facts["available"] == ["can_raw", "nf_tables", "tipc"]
|
||||
assert facts["config"] == {
|
||||
"CONFIG_TIPC": "m",
|
||||
"CONFIG_BPF_JIT": "y",
|
||||
"CONFIG_DEFAULT_HOSTNAME": "(none)",
|
||||
"CONFIG_HZ": "250",
|
||||
}
|
||||
|
||||
def test_a_section_never_printed_is_none_not_empty(self):
|
||||
"""``None`` is "could not read"; an empty list would claim "read it,
|
||||
and there is nothing" -- and that is what turns a module into
|
||||
``not_met`` downstream."""
|
||||
facts = parse_kernel_facts(_wire("[release]\n6.1.0\n[loaded]\n"))
|
||||
|
||||
assert facts["loaded"] == []
|
||||
assert facts["builtin"] is None
|
||||
assert facts["available"] is None
|
||||
assert facts["config"] is None
|
||||
|
||||
def test_whatever_surrounds_the_frame_is_ignored(self):
|
||||
"""A screen-scraping transport may echo the command or a banner."""
|
||||
noise = "Last login: today\nprintf '%s%s\\n' KFACTS_ BEGIN; ...\n"
|
||||
|
||||
assert parse_kernel_facts(_wire(REPORT, noise=noise))["release"] == "6.1.0-25-amd64"
|
||||
|
||||
def test_output_without_the_frame_raises(self):
|
||||
with pytest.raises(ValueError):
|
||||
parse_kernel_facts("sh: gzip: not found\n")
|
||||
|
||||
def test_an_empty_payload_raises(self):
|
||||
"""gzip or base64 missing on the host: the frame is there, the report is not."""
|
||||
with pytest.raises(ValueError):
|
||||
parse_kernel_facts("KFACTS_BEGIN\nKFACTS_END\n")
|
||||
empty = base64.b64encode(gzip.compress(b"")).decode()
|
||||
with pytest.raises(ValueError):
|
||||
parse_kernel_facts(f"KFACTS_BEGIN\n{empty}\nKFACTS_END\n")
|
||||
|
||||
def test_a_damaged_payload_raises(self):
|
||||
with pytest.raises(ValueError):
|
||||
parse_kernel_facts("KFACTS_BEGIN\nnot base64 at all!\nKFACTS_END\n")
|
||||
|
||||
|
||||
class TestModuleNames:
|
||||
@pytest.mark.parametrize(
|
||||
"raw, name",
|
||||
[
|
||||
("tipc", "tipc"),
|
||||
("kernel/net/tipc/tipc.ko", "tipc"),
|
||||
("kernel/net/tipc/tipc.ko.zst", "tipc"),
|
||||
("kernel/net/can/can-raw.ko.xz", "can_raw"),
|
||||
("CAN-RAW", "can_raw"),
|
||||
(" nf_tables ", "nf_tables"),
|
||||
],
|
||||
)
|
||||
def test_dash_and_underscore_are_one_name(self, raw, name):
|
||||
"""The kernel treats ``-`` and ``_`` in module names as the same."""
|
||||
assert module_name(raw) == name
|
||||
|
||||
|
||||
class TestTheCommand:
|
||||
def test_the_frame_is_not_in_the_command_itself(self):
|
||||
"""An echoing transport prints the command back; the markers must only
|
||||
appear once the command has run."""
|
||||
assert "KFACTS_BEGIN" not in KERNEL_FACTS_COMMAND
|
||||
assert "KFACTS_END" not in KERNEL_FACTS_COMMAND
|
||||
|
||||
def test_it_writes_nothing(self):
|
||||
for verb in ("modprobe", "insmod", "rmmod", "sudo", " > ", ">>"):
|
||||
assert verb not in KERNEL_FACTS_COMMAND
|
||||
|
||||
@pytest.mark.skipif(os.uname().sysname != "Linux", reason="reads a Linux kernel")
|
||||
def test_it_runs_and_parses_on_this_host(self):
|
||||
out = subprocess.run(
|
||||
["sh", "-c", KERNEL_FACTS_COMMAND], capture_output=True, text=True, timeout=60
|
||||
).stdout
|
||||
|
||||
facts = parse_kernel_facts(out)
|
||||
|
||||
assert facts["release"] == os.uname().release
|
||||
assert facts["loaded"] is None or all(isinstance(m, str) for m in facts["loaded"])
|
||||
|
||||
|
||||
class TestTheTemplate:
|
||||
def test_a_driver_supplies_only_the_transport(self):
|
||||
class Driver(KernelFactsMixin):
|
||||
def _run_kernel_facts_command(self, command: str) -> str:
|
||||
self.sent = command
|
||||
return _wire(REPORT)
|
||||
|
||||
driver = Driver()
|
||||
facts = driver.get_kernel_facts()
|
||||
|
||||
assert driver.sent == KERNEL_FACTS_COMMAND
|
||||
assert facts["release"] == "6.1.0-25-amd64"
|
||||
|
||||
def test_not_every_os_driver_has_it(self):
|
||||
"""A Windows host is an OSDriver too, and has no Linux kernel to read:
|
||||
``hasattr`` has to stay a truthful answer, so the drivers that can mix
|
||||
this in themselves."""
|
||||
assert not issubclass(OSDriver, KernelFactsMixin)
|
||||
assert not hasattr(OSDriver, "get_kernel_facts")
|
||||
@@ -3,7 +3,9 @@
|
||||
from napalm_device_types import add_lag_interfaces
|
||||
|
||||
|
||||
def _port(is_up: bool = True, is_enabled: bool = True, speed: float = 1000.0, trunk_group: str = "") -> dict:
|
||||
def _port(
|
||||
is_up: bool = True, is_enabled: bool = True, speed: float = 1000.0, trunk_group: str = ""
|
||||
) -> dict:
|
||||
port = {
|
||||
"is_up": is_up,
|
||||
"is_enabled": is_enabled,
|
||||
@@ -74,3 +76,12 @@ def test_does_not_modify_its_input():
|
||||
add_lag_interfaces(ifaces)
|
||||
|
||||
assert list(ifaces) == ["3"]
|
||||
|
||||
|
||||
def test_an_unknown_member_speed_does_not_make_the_lag_negative():
|
||||
ifaces = {
|
||||
"3": _port(speed=-1.0, trunk_group="Trk3"),
|
||||
"4": _port(speed=1000.0, trunk_group="Trk3"),
|
||||
}
|
||||
|
||||
assert add_lag_interfaces(ifaces)["Trk3"]["speed"] == 1000.0
|
||||
@@ -0,0 +1,397 @@
|
||||
"""get_listening_sockets: what listens on which address, and which service it is.
|
||||
|
||||
Whether a service is reachable from outside its host is decided by what it
|
||||
listens on -- ``0.0.0.0:5432`` is, ``127.0.0.1:5432`` is not. Reading that is
|
||||
the same on every Linux host: ``ss`` for the sockets, ``/proc/<pid>/cgroup``
|
||||
for the systemd unit or container a process belongs to. So both live here once,
|
||||
and a driver only carries the command across (#658 in netOrk).
|
||||
|
||||
A host without ``ss`` -- OpenWrt's busybox, an old net-tools box -- is read
|
||||
with ``netstat -lntup`` instead, and on OpenWrt the cgroup names the procd
|
||||
service (#673 in netOrk).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import OSDriver
|
||||
from napalm_device_types.listening import (
|
||||
LISTENING_SOCKETS_COMMAND,
|
||||
ListeningSocketsMixin,
|
||||
ListeningSocketsUnavailable,
|
||||
parse_listening_sockets,
|
||||
)
|
||||
|
||||
CID = "4f1c" + "0" * 60
|
||||
SS = """Netid State Recv-Q Send-Q Local Address:Port Peer Address:PortProcess
|
||||
udp UNCONN 0 0 127.0.0.53%lo:53 0.0.0.0:* users:(("systemd-resolve",pid=612,fd=13))
|
||||
udp UNCONN 0 0 0.0.0.0%ens18:68 0.0.0.0:* users:(("systemd-network",pid=590,fd=22))
|
||||
tcp LISTEN 0 4096 0.0.0.0:5432 0.0.0.0:* users:(("postgres",pid=812,fd=6))
|
||||
tcp LISTEN 0 128 [::]:22 [::]:* users:(("sshd",pid=700,fd=4))
|
||||
tcp LISTEN 0 511 *:80 *:* users:(("nginx",pid=901,fd=6),("nginx",pid=900,fd=6))
|
||||
tcp LISTEN 0 4096 [::ffff:127.0.0.1]:8125 *:* users:(("statsd",pid=950,fd=3))
|
||||
tcp LISTEN 0 4096 0.0.0.0:8080 0.0.0.0:* users:(("docker-proxy",pid=1200,fd=4))
|
||||
tcp LISTEN 0 4096 0.0.0.0:9100 0.0.0.0:* users:(("node_exporter",pid=1300,fd=3))
|
||||
tcp LISTEN 0 64 0.0.0.0:2049 0.0.0.0:*
|
||||
"""
|
||||
CGROUPS = f"""612 0::/system.slice/systemd-resolved.service
|
||||
590 0::/system.slice/systemd-networkd.service
|
||||
812 0::/system.slice/system-postgresql.slice/postgresql@16-main.service
|
||||
700 0::/system.slice/ssh.service
|
||||
900 0::/system.slice/nginx.service
|
||||
901 0::/system.slice/nginx.service
|
||||
950 0::/user.slice/user-1000.slice/session-3.scope
|
||||
1200 0::/system.slice/docker.service
|
||||
1300 0::/system.slice/docker-{CID}.scope
|
||||
"""
|
||||
|
||||
|
||||
def _wire(ss: str = SS, cgroups: str = CGROUPS, *, rc: int = 0, noise: str = "") -> str:
|
||||
"""The report as the command prints it, framed."""
|
||||
return f"{noise}SOCK_BEGIN\n[ss]\n{ss}__SS_RC={rc}\n[cgroups]\n{cgroups}SOCK_END\n"
|
||||
|
||||
|
||||
def _by_port(sockets: list) -> dict:
|
||||
return {(s["proto"], s["port"], s["address"]): s for s in sockets}
|
||||
|
||||
|
||||
class TestParsing:
|
||||
def test_a_socket_comes_with_its_process_and_unit(self):
|
||||
sockets = _by_port(parse_listening_sockets(_wire()))
|
||||
|
||||
assert sockets[("tcp", 5432, "0.0.0.0")] == {
|
||||
"proto": "tcp",
|
||||
"address": "0.0.0.0",
|
||||
"port": 5432,
|
||||
"interface": None,
|
||||
"process": "postgres",
|
||||
"pid": 812,
|
||||
"unit": "postgresql@16-main",
|
||||
"container_id": None,
|
||||
}
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"local, address, interface",
|
||||
[
|
||||
("[::]:22", "::", None),
|
||||
("*:80", "*", None),
|
||||
("127.0.0.53%lo:53", "127.0.0.53", "lo"),
|
||||
("0.0.0.0%ens18:68", "0.0.0.0", "ens18"),
|
||||
("[::ffff:127.0.0.1]:8125", "::ffff:127.0.0.1", None),
|
||||
("[fe80::1%eth0]:546", "fe80::1", "eth0"),
|
||||
("[fe80::1]%eth0:546", "fe80::1", "eth0"), # what iproute2 ss prints
|
||||
(":::22", "::", None), # iproute2 4.9 prints IPv6 without brackets
|
||||
],
|
||||
)
|
||||
def test_every_address_form(self, local: str, address: str, interface):
|
||||
line = f'tcp LISTEN 0 128 {local} *:* users:(("x",pid=1,fd=3))\n'
|
||||
|
||||
[socket] = parse_listening_sockets(_wire(line, "1 0::/system.slice/x.service\n"))
|
||||
|
||||
assert (socket["address"], socket["interface"]) == (address, interface)
|
||||
|
||||
def test_the_first_of_several_processes_names_the_socket(self):
|
||||
nginx = _by_port(parse_listening_sockets(_wire()))[("tcp", 80, "*")]
|
||||
|
||||
assert (nginx["process"], nginx["pid"], nginx["unit"]) == ("nginx", 901, "nginx")
|
||||
|
||||
def test_a_socket_without_a_process_is_kept(self):
|
||||
"""The kernel's nfsd, or every socket of another user without root."""
|
||||
nfs = _by_port(parse_listening_sockets(_wire()))[("tcp", 2049, "0.0.0.0")]
|
||||
|
||||
assert (nfs["process"], nfs["pid"], nfs["unit"], nfs["container_id"]) == (
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
)
|
||||
|
||||
def test_the_header_is_no_socket(self):
|
||||
sockets = parse_listening_sockets(_wire())
|
||||
|
||||
assert len(sockets) == 9
|
||||
assert all(s["proto"] in ("tcp", "udp") for s in sockets)
|
||||
|
||||
def test_sorted_by_protocol_port_and_address(self):
|
||||
keys = [(s["proto"], s["port"], s["address"]) for s in parse_listening_sockets(_wire())]
|
||||
|
||||
assert keys == sorted(keys)
|
||||
|
||||
def test_nothing_listening_is_an_empty_list(self):
|
||||
assert parse_listening_sockets(_wire(SS.splitlines(True)[0], "")) == []
|
||||
|
||||
|
||||
class TestCgroups:
|
||||
def _unit_and_container(self, cgroup_lines: str) -> tuple:
|
||||
line = 'tcp LISTEN 0 128 0.0.0.0:1 0.0.0.0:* users:(("p",pid=5,fd=3))\n'
|
||||
[socket] = parse_listening_sockets(_wire(line, cgroup_lines))
|
||||
return socket["unit"], socket["container_id"]
|
||||
|
||||
def test_a_container_on_the_host_network(self):
|
||||
assert self._unit_and_container(f"5 0::/system.slice/docker-{CID}.scope\n") == (
|
||||
None,
|
||||
CID,
|
||||
)
|
||||
|
||||
def test_a_container_under_the_cgroupfs_driver(self):
|
||||
assert self._unit_and_container(f"5 0::/docker/{CID}\n") == (None, CID)
|
||||
|
||||
def test_cgroup_v1_reads_the_systemd_hierarchy(self):
|
||||
lines = "5 12:pids:/user.slice\n5 1:name=systemd:/system.slice/ssh.service\n"
|
||||
|
||||
assert self._unit_and_container(lines) == ("ssh", None)
|
||||
|
||||
def test_an_escaped_template_instance(self):
|
||||
lines = "5 0::/system.slice/system-wg\\x2dquick.slice/wg-quick@wg0.service\n"
|
||||
|
||||
assert self._unit_and_container(lines) == ("wg-quick@wg0", None)
|
||||
|
||||
def test_a_procd_service_on_openwrt(self):
|
||||
"""procd puts every instance into /services/<name>/<instance>, a jailed
|
||||
one too -- its PID is not the one procd reports, its cgroup is."""
|
||||
assert self._unit_and_container("5 0::/services/dnsmasq/cfg01411c\n") == (
|
||||
"dnsmasq",
|
||||
None,
|
||||
)
|
||||
|
||||
def test_a_login_session_is_no_unit(self):
|
||||
assert self._unit_and_container("5 0::/user.slice/user-1000.slice/session-3.scope\n") == (
|
||||
None,
|
||||
None,
|
||||
)
|
||||
|
||||
def test_a_process_gone_before_its_cgroup_was_read(self):
|
||||
assert self._unit_and_container("") == (None, None)
|
||||
|
||||
def test_the_docker_daemons_own_proxy_stays_raw(self):
|
||||
"""docker-proxy runs in docker.service. Telling it apart from a real
|
||||
listener is the consumer's business; the reading reports what is there."""
|
||||
proxy = _by_port(parse_listening_sockets(_wire()))[("tcp", 8080, "0.0.0.0")]
|
||||
|
||||
assert (proxy["process"], proxy["unit"]) == ("docker-proxy", "docker")
|
||||
|
||||
|
||||
class TestFailures:
|
||||
def test_whatever_surrounds_the_frame_is_ignored(self):
|
||||
noisy = _wire(noise="$ sh -c '...'\nWelcome to Ubuntu\n")
|
||||
|
||||
assert len(parse_listening_sockets(noisy)) == 9
|
||||
|
||||
def test_output_without_the_frame_raises(self):
|
||||
with pytest.raises(ValueError):
|
||||
parse_listening_sockets("sudo: a password is required\n")
|
||||
|
||||
def test_a_report_cut_short_raises(self):
|
||||
with pytest.raises(ValueError):
|
||||
parse_listening_sockets(_wire()[: -len("SOCK_END\n")])
|
||||
|
||||
def test_a_host_without_ss_or_netstat_is_unavailable(self):
|
||||
with pytest.raises(ListeningSocketsUnavailable):
|
||||
parse_listening_sockets("SOCK_BEGIN\n[no-ss]\nSOCK_END\n")
|
||||
|
||||
def test_ss_failing_raises(self):
|
||||
with pytest.raises(ValueError):
|
||||
parse_listening_sockets(_wire("ss: invalid option -- 'p'\n", "", rc=1))
|
||||
|
||||
def test_unavailable_is_not_a_value_error(self):
|
||||
"""A caller retries a broken report without privilege; a host without
|
||||
ss has nothing to retry."""
|
||||
assert not issubclass(ListeningSocketsUnavailable, ValueError)
|
||||
|
||||
|
||||
# busybox netstat on OpenWrt 25.12, addresses replaced by documentation ones.
|
||||
BUSYBOX = """Active Internet connections (only servers)
|
||||
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
|
||||
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1604/dropbear
|
||||
tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 1969/uhttpd
|
||||
tcp 0 0 192.0.2.15:53 0.0.0.0:* LISTEN 1495/dnsmasq
|
||||
tcp 0 0 127.0.0.1:53 0.0.0.0:* LISTEN 1495/dnsmasq
|
||||
tcp 0 0 :::22 :::* LISTEN 1604/dropbear
|
||||
tcp 0 0 fe80::1:53 :::* LISTEN 1495/dnsmasq
|
||||
tcp 0 0 ::1:53 :::* LISTEN 1495/dnsmasq
|
||||
udp 0 0 192.0.2.15:53 0.0.0.0:* 1495/dnsmasq
|
||||
udp 0 0 0.0.0.0:161 0.0.0.0:* 3173/snmpd
|
||||
udp 0 0 0.0.0.0:5353 0.0.0.0:* -
|
||||
"""
|
||||
PROCD = """1495 0::/services/dnsmasq/cfg01411c
|
||||
1604 0::/services/dropbear/instance1
|
||||
1969 0::/services/uhttpd/instance1
|
||||
3173 0::/services/snmpd/instance1
|
||||
"""
|
||||
# net-tools netstat on an old Debian: tcp6/udp6, and a program name with a space.
|
||||
NET_TOOLS = """Active Internet connections (only servers)
|
||||
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
|
||||
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 700/sshd: /usr/sbin
|
||||
tcp6 0 0 :::22 :::* LISTEN 700/sshd: /usr/sbin
|
||||
udp6 0 0 :::5353 :::* -
|
||||
"""
|
||||
|
||||
|
||||
def _netstat(out: str = BUSYBOX, cgroups: str = PROCD, *, rc: int = 0) -> str:
|
||||
return f"SOCK_BEGIN\n[netstat]\n{out}__SS_RC={rc}\n[cgroups]\n{cgroups}SOCK_END\n"
|
||||
|
||||
|
||||
class TestNetstat:
|
||||
"""A host without ss (#673 in netOrk): OpenWrt's busybox, old net-tools."""
|
||||
|
||||
def test_a_socket_comes_with_its_process_and_procd_service(self):
|
||||
sockets = _by_port(parse_listening_sockets(_netstat()))
|
||||
|
||||
assert sockets[("tcp", 22, "0.0.0.0")] == {
|
||||
"proto": "tcp",
|
||||
"address": "0.0.0.0",
|
||||
"port": 22,
|
||||
"interface": None,
|
||||
"process": "dropbear",
|
||||
"pid": 1604,
|
||||
"unit": "dropbear",
|
||||
"container_id": None,
|
||||
}
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"proto, port, address",
|
||||
[("tcp", 22, "::"), ("tcp", 53, "fe80::1"), ("tcp", 53, "::1"), ("tcp", 53, "192.0.2.15")],
|
||||
)
|
||||
def test_every_address_form(self, proto, port, address):
|
||||
assert (proto, port, address) in _by_port(parse_listening_sockets(_netstat()))
|
||||
|
||||
def test_a_udp_socket_has_no_state_column(self):
|
||||
snmpd = _by_port(parse_listening_sockets(_netstat()))[("udp", 161, "0.0.0.0")]
|
||||
|
||||
assert (snmpd["process"], snmpd["pid"], snmpd["unit"]) == ("snmpd", 3173, "snmpd")
|
||||
|
||||
def test_a_socket_without_a_process_is_kept(self):
|
||||
mdns = _by_port(parse_listening_sockets(_netstat()))[("udp", 5353, "0.0.0.0")]
|
||||
|
||||
assert (mdns["process"], mdns["pid"], mdns["unit"]) == (None, None, None)
|
||||
|
||||
def test_the_headers_are_no_sockets(self):
|
||||
assert len(parse_listening_sockets(_netstat())) == 10
|
||||
|
||||
def test_net_tools_names_ipv6_and_programs_its_own_way(self):
|
||||
sockets = _by_port(parse_listening_sockets(_netstat(NET_TOOLS, "")))
|
||||
|
||||
assert sockets[("tcp", 22, "::")]["process"] == "sshd"
|
||||
assert sockets[("tcp", 22, "::")]["pid"] == 700
|
||||
assert ("udp", 5353, "::") in sockets
|
||||
|
||||
def test_netstat_failing_raises(self):
|
||||
with pytest.raises(ValueError):
|
||||
parse_listening_sockets(_netstat("netstat: invalid option -- 'p'\n", "", rc=1))
|
||||
|
||||
|
||||
class TestTheNetstatFallback:
|
||||
"""The command itself, on a host where ss is missing and netstat is not."""
|
||||
|
||||
@pytest.mark.skipif(
|
||||
any(os.path.exists(f"{d}/ss") for d in ("/usr/sbin", "/sbin")),
|
||||
reason="ss sits on the PATH the command always adds",
|
||||
)
|
||||
def test_it_reads_netstat_when_there_is_no_ss(self, tmp_path):
|
||||
for tool in ("grep", "cut", "sort", "sed", "tr", "cat"):
|
||||
(tmp_path / tool).symlink_to(shutil.which(tool))
|
||||
stub = tmp_path / "netstat"
|
||||
stub.write_text(f"#!/bin/sh\ncat <<'EOF'\n{BUSYBOX}EOF\n")
|
||||
stub.chmod(0o755)
|
||||
|
||||
out = subprocess.run(
|
||||
["/bin/sh", "-c", LISTENING_SOCKETS_COMMAND],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
env={"PATH": str(tmp_path)},
|
||||
timeout=30,
|
||||
).stdout
|
||||
|
||||
assert "[netstat]" in out
|
||||
sockets = _by_port(parse_listening_sockets(out))
|
||||
assert sockets[("tcp", 443, "0.0.0.0")]["process"] == "uhttpd"
|
||||
assert len(sockets) == 10
|
||||
|
||||
|
||||
class TestTheCommand:
|
||||
def test_the_frame_is_not_in_the_command_itself(self):
|
||||
"""An echoing transport prints the command back; the markers must only
|
||||
appear once the command has run."""
|
||||
assert "SOCK_BEGIN" not in LISTENING_SOCKETS_COMMAND
|
||||
assert "SOCK_END" not in LISTENING_SOCKETS_COMMAND
|
||||
|
||||
def test_it_writes_and_changes_nothing(self):
|
||||
for verb in ("sudo", " > ", ">>", "kill", "rm ", "systemctl "):
|
||||
assert verb not in LISTENING_SOCKETS_COMMAND
|
||||
|
||||
def test_it_is_posix_sh(self):
|
||||
result = subprocess.run(
|
||||
["sh", "-n", "-c", LISTENING_SOCKETS_COMMAND], capture_output=True, text=True
|
||||
)
|
||||
assert result.returncode == 0, result.stderr
|
||||
|
||||
@pytest.mark.skipif(shutil.which("ss") is None, reason="needs ss on this host")
|
||||
def test_it_runs_and_parses_on_this_host(self):
|
||||
out = subprocess.run(
|
||||
["sh", "-c", LISTENING_SOCKETS_COMMAND], capture_output=True, text=True, timeout=60
|
||||
).stdout
|
||||
|
||||
sockets = parse_listening_sockets(out)
|
||||
|
||||
assert all(s["proto"] in ("tcp", "udp") and 0 < s["port"] < 65536 for s in sockets)
|
||||
|
||||
|
||||
class _Driver(ListeningSocketsMixin):
|
||||
def __init__(self, privileged: str, unprivileged: str = "") -> None:
|
||||
self.answers = {True: privileged, False: unprivileged}
|
||||
self.calls: list = []
|
||||
|
||||
def _run_listening_sockets_command(self, command: str, *, privileged: bool) -> str:
|
||||
self.calls.append((command, privileged))
|
||||
return self.answers[privileged]
|
||||
|
||||
|
||||
class TestTheTemplate:
|
||||
def test_a_driver_supplies_only_the_transport(self):
|
||||
driver = _Driver(_wire())
|
||||
|
||||
reading = driver.get_listening_sockets()
|
||||
|
||||
assert reading["attributed"] is True
|
||||
assert len(reading["sockets"]) == 9
|
||||
[(command, privileged)] = driver.calls
|
||||
assert privileged is True
|
||||
|
||||
def test_the_command_reaches_root_as_one_shell(self):
|
||||
"""``sudo -n a; b`` runs only ``a`` as root: the whole script goes as
|
||||
one ``sh -c`` argument."""
|
||||
driver = _Driver(_wire())
|
||||
driver.get_listening_sockets()
|
||||
|
||||
[(command, _privileged)] = driver.calls
|
||||
assert command.startswith("sh -c '")
|
||||
assert subprocess.run(["sh", "-n", "-c", command]).returncode == 0
|
||||
|
||||
def test_without_root_it_reads_what_it_can(self):
|
||||
"""Without sudo, ss names only the user's own processes: the sockets
|
||||
are still worth having, marked as not attributed."""
|
||||
driver = _Driver("sudo: a password is required\n", _wire(cgroups=""))
|
||||
|
||||
reading = driver.get_listening_sockets()
|
||||
|
||||
assert reading["attributed"] is False
|
||||
assert len(reading["sockets"]) == 9
|
||||
assert [p for _c, p in driver.calls] == [True, False]
|
||||
|
||||
def test_a_host_without_either_is_not_asked_twice(self):
|
||||
driver = _Driver("SOCK_BEGIN\n[no-ss]\nSOCK_END\n")
|
||||
|
||||
with pytest.raises(ListeningSocketsUnavailable):
|
||||
driver.get_listening_sockets()
|
||||
assert len(driver.calls) == 1
|
||||
|
||||
def test_not_every_os_driver_has_it(self):
|
||||
"""A Windows host is an OSDriver too and has no ss: ``hasattr`` has to
|
||||
stay a truthful answer, so the drivers that can mix this in themselves."""
|
||||
assert not issubclass(OSDriver, ListeningSocketsMixin)
|
||||
assert not hasattr(OSDriver, "get_listening_sockets")
|
||||
@@ -0,0 +1,127 @@
|
||||
"""LLDP rows projected into NAPALM's ``get_lldp_neighbors`` and ``get_lldp_neighbors_detail``.
|
||||
|
||||
A driver reads its neighbours into :class:`LldpNeighborRow` (from ``lldpctl``,
|
||||
an API, a CLI) and the two NAPALM shapes come from those rows the same way for
|
||||
every driver (NAPALM/napalm-device-types#10).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import get_type_hints
|
||||
|
||||
from napalm_device_types import LldpNeighborRow, lldp_neighbors, lldp_neighbors_detail
|
||||
|
||||
|
||||
def _row(local_port: str, name: str, **fields: object) -> LldpNeighborRow:
|
||||
row: LldpNeighborRow = {
|
||||
"local_port": local_port,
|
||||
"chassis_id": "",
|
||||
"chassis_id_type": "",
|
||||
"port_id": "",
|
||||
"port_id_type": "",
|
||||
"port_description": "",
|
||||
"system_name": name,
|
||||
"system_description": "",
|
||||
"management_addresses": [],
|
||||
"capabilities": [],
|
||||
"enabled_capabilities": [],
|
||||
}
|
||||
row.update(fields) # type: ignore[typeddict-item]
|
||||
return row
|
||||
|
||||
|
||||
PHONE = _row(
|
||||
"eth0.1",
|
||||
"desk-phone",
|
||||
chassis_id="00:11:22:33:44:AB",
|
||||
chassis_id_type="mac",
|
||||
port_id="00:11:22:33:44:01",
|
||||
port_id_type="mac",
|
||||
port_description="WAN",
|
||||
system_description="SIP phone",
|
||||
capabilities=["bridge", "telephone"],
|
||||
enabled_capabilities=["telephone"],
|
||||
)
|
||||
PC = _row("eth0.1", "desk-pc", port_id="eno1", port_id_type="ifname")
|
||||
SWITCH = _row("eth1", "access-sw", port_id="gi1/0/7", port_id_type="local")
|
||||
|
||||
|
||||
def test_a_row_names_its_fields():
|
||||
assert set(get_type_hints(LldpNeighborRow)) == {
|
||||
"local_port",
|
||||
"chassis_id",
|
||||
"chassis_id_type",
|
||||
"port_id",
|
||||
"port_id_type",
|
||||
"port_description",
|
||||
"system_name",
|
||||
"system_description",
|
||||
"management_addresses",
|
||||
"capabilities",
|
||||
"enabled_capabilities",
|
||||
}
|
||||
|
||||
|
||||
class TestNeighbors:
|
||||
def test_every_neighbour_of_a_port_is_listed(self):
|
||||
assert lldp_neighbors([PHONE, PC, SWITCH]) == {
|
||||
"eth0.1": [
|
||||
{"hostname": "desk-phone", "port": "00:11:22:33:44:01", "mac": "00:11:22:33:44:ab"},
|
||||
{"hostname": "desk-pc", "port": "eno1"},
|
||||
],
|
||||
"eth1": [{"hostname": "access-sw", "port": "gi1/0/7"}],
|
||||
}
|
||||
|
||||
def test_no_rows_is_no_neighbour(self):
|
||||
assert lldp_neighbors([]) == {}
|
||||
|
||||
def test_a_chassis_mac_is_reported_in_lower_case(self):
|
||||
# netOrk matches a neighbour by this MAC first (NAPALM/napalm-openwrt#31).
|
||||
assert lldp_neighbors([PHONE])["eth0.1"][0]["mac"] == "00:11:22:33:44:ab"
|
||||
|
||||
def test_a_chassis_named_otherwise_has_no_mac(self):
|
||||
switch = _row("eth1", "sw", chassis_id="00:11:22:33:44:55", chassis_id_type="local")
|
||||
|
||||
assert "mac" not in lldp_neighbors([switch])["eth1"][0]
|
||||
|
||||
def test_a_mac_id_that_is_no_mac_address_is_left_out(self):
|
||||
odd = _row("eth1", "sw", chassis_id="00:11", chassis_id_type="mac")
|
||||
|
||||
assert "mac" not in lldp_neighbors([odd])["eth1"][0]
|
||||
|
||||
|
||||
class TestNeighborsDetail:
|
||||
def test_every_field_napalm_names(self):
|
||||
assert lldp_neighbors_detail([PHONE]) == {
|
||||
"eth0.1": [
|
||||
{
|
||||
"parent_interface": "",
|
||||
"remote_port": "00:11:22:33:44:01",
|
||||
"remote_port_description": "WAN",
|
||||
"remote_chassis_id": "00:11:22:33:44:AB",
|
||||
"remote_system_name": "desk-phone",
|
||||
"remote_system_description": "SIP phone",
|
||||
"remote_system_capab": ["bridge", "telephone"],
|
||||
"remote_system_enable_capab": ["telephone"],
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
def test_every_neighbour_of_a_port_is_listed(self):
|
||||
detail = lldp_neighbors_detail([PHONE, PC, SWITCH])
|
||||
|
||||
assert [n["remote_system_name"] for n in detail["eth0.1"]] == ["desk-phone", "desk-pc"]
|
||||
assert [n["remote_system_name"] for n in detail["eth1"]] == ["access-sw"]
|
||||
|
||||
def test_one_interface_only(self):
|
||||
assert list(lldp_neighbors_detail([PHONE, PC, SWITCH], "eth1")) == ["eth1"]
|
||||
|
||||
def test_an_interface_without_neighbours_is_empty(self):
|
||||
assert lldp_neighbors_detail([PHONE], "eth9") == {}
|
||||
|
||||
def test_the_lists_are_the_rows_own_copies(self):
|
||||
detail = lldp_neighbors_detail([PHONE])
|
||||
|
||||
detail["eth0.1"][0]["remote_system_capab"].append("router")
|
||||
|
||||
assert PHONE["capabilities"] == ["bridge", "telephone"]
|
||||
@@ -0,0 +1,308 @@
|
||||
"""``lldpctl -f keyvalue``: one row per LLDP neighbour, for every driver that reads lldpd.
|
||||
|
||||
napalm-linux, napalm-openwrt and napalm-proxmox each parsed this output on their
|
||||
own, and each copy grouped the lines by interface, so two neighbours on one port
|
||||
merged (NAPALM/napalm-openwrt#9, NAPALM/napalm-linux#58), split ``eth0.1`` at its
|
||||
dot (NAPALM/napalm-openwrt#26) and looked for ``chassis.id.value``, a key lldpd
|
||||
never prints. The parse lives here once now.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_device_types import LLDPCTL_KEYVALUE_COMMAND, parse_lldpctl_keyvalue
|
||||
|
||||
# The recording napalm-openwrt's tests have used since the driver was written
|
||||
# (tests/unit/test_driver.py, ``LLDPCTL_KV``): one neighbour on eth0. Its two
|
||||
# ``chassis.cap.*`` lines are not lldpd's format (lldpd prints one
|
||||
# ``chassis.<Capability>.enabled=on|off`` line per capability), so they yield none.
|
||||
OPENWRT_ONE_NEIGHBOUR = """\
|
||||
lldp.eth0.via=LLDP
|
||||
lldp.eth0.rid=1
|
||||
lldp.eth0.age=0 day, 01:23:45
|
||||
lldp.eth0.chassis.mac=00:aa:bb:cc:dd:ee
|
||||
lldp.eth0.chassis.name=core-router
|
||||
lldp.eth0.chassis.descr=RouterOS 7.x
|
||||
lldp.eth0.chassis.mgmt-ip=10.0.0.1
|
||||
lldp.eth0.chassis.cap.available=Router, Bridge
|
||||
lldp.eth0.chassis.cap.enabled=Router
|
||||
lldp.eth0.port.ifname=ether1
|
||||
lldp.eth0.port.descr=uplink
|
||||
"""
|
||||
|
||||
# Laid out the way lldpd 1.0's keyvalue writer prints ``show neighbors details``
|
||||
# (src/client/display.c ``display_interface``, ``display_chassis``,
|
||||
# ``display_port``, ``display_vlans``, ``display_med``; src/client/kv_writer.c):
|
||||
# a neighbour opens with the local interface's optional ``alias`` and ``via``,
|
||||
# an id is printed under its type (``chassis.mac``, ``port.local``), a
|
||||
# capability as ``chassis.<Symbol>.enabled``, a management address once per
|
||||
# address. Not recorded from a device.
|
||||
#
|
||||
# On ``eth0.1`` (a VLAN interface, its name holds a dot) a phone and a PC behind
|
||||
# an unmanaged switch; on ``eth1`` a switch that names its port by a local id and
|
||||
# itself by an address.
|
||||
TWO_ON_ONE_DOTTED_PORT = """\
|
||||
lldp.eth0.1.alias=desk
|
||||
lldp.eth0.1.via=LLDP
|
||||
lldp.eth0.1.rid=1
|
||||
lldp.eth0.1.age=0 day, 00:10:00
|
||||
lldp.eth0.1.chassis.mac=00:11:22:33:44:01
|
||||
lldp.eth0.1.chassis.name=desk-phone
|
||||
lldp.eth0.1.chassis.descr=SIP phone, firmware a=1
|
||||
lldp.eth0.1.chassis.mgmt-ip=10.0.0.21
|
||||
lldp.eth0.1.chassis.mgmt-iface=2
|
||||
lldp.eth0.1.chassis.mgmt-ip=fe80::211:22ff:fe33:4401
|
||||
lldp.eth0.1.chassis.mgmt-iface=2
|
||||
lldp.eth0.1.chassis.Bridge.enabled=off
|
||||
lldp.eth0.1.chassis.Tel.enabled=on
|
||||
lldp.eth0.1.port.mac=00:11:22:33:44:01
|
||||
lldp.eth0.1.port.descr=WAN
|
||||
lldp.eth0.1.port.ttl=120
|
||||
lldp.eth0.1.port.auto-negotiation.supported=yes
|
||||
lldp.eth0.1.port.auto-negotiation.enabled=yes
|
||||
lldp.eth0.1.port.auto-negotiation.100Base-TX.hd=yes
|
||||
lldp.eth0.1.port.auto-negotiation.100Base-TX.fd=yes
|
||||
lldp.eth0.1.port.auto-negotiation.current=100BaseTXFD - 2 pair category 5 UTP, full duplex mode
|
||||
lldp.eth0.1.vlan.vlan-id=10
|
||||
lldp.eth0.1.vlan.pvid=yes
|
||||
lldp.eth0.1.vlan=voice
|
||||
lldp.eth0.1.lldp-med.device-type=Communication Device Endpoint (Class III)
|
||||
lldp.eth0.1.lldp-med.Capabilities.available=yes
|
||||
lldp.eth0.1.lldp-med.inventory.software=2.1.4
|
||||
lldp.eth0.1.alias=desk
|
||||
lldp.eth0.1.via=LLDP
|
||||
lldp.eth0.1.rid=2
|
||||
lldp.eth0.1.age=0 day, 00:09:00
|
||||
lldp.eth0.1.chassis.mac=00:11:22:33:44:02
|
||||
lldp.eth0.1.chassis.name=desk-pc
|
||||
lldp.eth0.1.chassis.descr=Debian GNU/Linux 12 (bookworm) Linux 6.1.0-25-amd64
|
||||
lldp.eth0.1.chassis.Bridge.enabled=off
|
||||
lldp.eth0.1.chassis.Router.enabled=off
|
||||
lldp.eth0.1.chassis.Wlan.enabled=off
|
||||
lldp.eth0.1.chassis.Station.enabled=on
|
||||
lldp.eth0.1.port.ifname=eno1
|
||||
lldp.eth0.1.port.ttl=120
|
||||
lldp.eth0.1.unknown-tlvs.unknown-tlv.oui=00,80,C2
|
||||
lldp.eth0.1.unknown-tlvs.unknown-tlv.subtype=9
|
||||
lldp.eth0.1.unknown-tlvs.unknown-tlv.len=1
|
||||
lldp.eth0.1.unknown-tlvs.unknown-tlv=01
|
||||
lldp.eth1.via=LLDP
|
||||
lldp.eth1.rid=3
|
||||
lldp.eth1.age=1 day, 02:00:00
|
||||
lldp.eth1.chassis.ip=192.0.2.7
|
||||
lldp.eth1.chassis.name=access-sw
|
||||
lldp.eth1.chassis.Bridge.enabled=on
|
||||
lldp.eth1.chassis.Router.enabled=off
|
||||
lldp.eth1.chassis.CVLAN.enabled=on
|
||||
lldp.eth1.port.local=gi1/0/7
|
||||
lldp.eth1.port.descr=GigabitEthernet1/0/7
|
||||
lldp.eth1.ppvid.supported=no
|
||||
lldp.eth1.ppvid.enabled=no
|
||||
lldp.eth1.pi=88cc
|
||||
"""
|
||||
|
||||
|
||||
def _rows(output: str = TWO_ON_ONE_DOTTED_PORT) -> list[dict[str, object]]:
|
||||
return [dict(row) for row in parse_lldpctl_keyvalue(output)]
|
||||
|
||||
|
||||
class TestOneRowPerNeighbour:
|
||||
def test_the_openwrt_recording_is_one_neighbour_with_its_fields(self):
|
||||
assert _rows(OPENWRT_ONE_NEIGHBOUR) == [
|
||||
{
|
||||
"local_port": "eth0",
|
||||
"chassis_id": "00:aa:bb:cc:dd:ee",
|
||||
"chassis_id_type": "mac",
|
||||
"port_id": "ether1",
|
||||
"port_id_type": "ifname",
|
||||
"port_description": "uplink",
|
||||
"system_name": "core-router",
|
||||
"system_description": "RouterOS 7.x",
|
||||
"management_addresses": ["10.0.0.1"],
|
||||
"capabilities": [],
|
||||
"enabled_capabilities": [],
|
||||
}
|
||||
]
|
||||
|
||||
def test_two_neighbours_on_one_port_stay_apart(self):
|
||||
rows = _rows()
|
||||
|
||||
assert [(r["local_port"], r["system_name"]) for r in rows] == [
|
||||
("eth0.1", "desk-phone"),
|
||||
("eth0.1", "desk-pc"),
|
||||
("eth1", "access-sw"),
|
||||
]
|
||||
|
||||
def test_a_neighbour_keeps_only_its_own_fields(self):
|
||||
pc = _rows()[1]
|
||||
|
||||
assert pc["chassis_id"] == "00:11:22:33:44:02"
|
||||
assert pc["port_description"] == ""
|
||||
assert pc["management_addresses"] == []
|
||||
|
||||
def test_neighbours_without_alias_are_split_at_via(self):
|
||||
output = OPENWRT_ONE_NEIGHBOUR + OPENWRT_ONE_NEIGHBOUR.replace("core-router", "edge")
|
||||
|
||||
assert [r["system_name"] for r in _rows(output)] == ["core-router", "edge"]
|
||||
|
||||
|
||||
class TestInterfaceName:
|
||||
def test_a_dotted_interface_name_survives(self):
|
||||
assert {r["local_port"] for r in _rows()} == {"eth0.1", "eth1"}
|
||||
|
||||
def test_an_interface_named_like_a_field_survives(self):
|
||||
output = (
|
||||
"lldp.vlan.100.via=LLDP\n"
|
||||
"lldp.vlan.100.chassis.mac=00:aa:bb:cc:dd:01\n"
|
||||
"lldp.vlan.100.port.ifname=port1\n"
|
||||
)
|
||||
|
||||
(row,) = _rows(output)
|
||||
|
||||
assert (row["local_port"], row["chassis_id"], row["port_id"]) == (
|
||||
"vlan.100",
|
||||
"00:aa:bb:cc:dd:01",
|
||||
"port1",
|
||||
)
|
||||
|
||||
def test_without_via_the_name_ends_at_the_first_field(self):
|
||||
output = "lldp.br-lan.10.chassis.name=sw\nlldp.br-lan.10.port.ifname=1\n"
|
||||
|
||||
(row,) = _rows(output)
|
||||
|
||||
assert (row["local_port"], row["system_name"], row["port_id"]) == ("br-lan.10", "sw", "1")
|
||||
|
||||
|
||||
class TestIds:
|
||||
def test_a_port_given_by_its_mac_is_read(self):
|
||||
phone = _rows()[0]
|
||||
|
||||
assert (phone["port_id"], phone["port_id_type"]) == ("00:11:22:33:44:01", "mac")
|
||||
|
||||
def test_a_port_given_by_a_local_id_is_read(self):
|
||||
switch = _rows()[2]
|
||||
|
||||
assert (switch["port_id"], switch["port_id_type"]) == ("gi1/0/7", "local")
|
||||
|
||||
def test_a_chassis_given_by_its_address_is_read(self):
|
||||
switch = _rows()[2]
|
||||
|
||||
assert (switch["chassis_id"], switch["chassis_id_type"]) == ("192.0.2.7", "ip")
|
||||
|
||||
@pytest.mark.parametrize("kind", ["local", "ifname", "ifalias", "unhandled"])
|
||||
def test_every_id_type_lldpd_names_is_read(self, kind):
|
||||
output = f"lldp.eth0.via=LLDP\nlldp.eth0.chassis.{kind}=c1\nlldp.eth0.port.{kind}=p1\n"
|
||||
|
||||
(row,) = _rows(output)
|
||||
|
||||
assert (row["chassis_id"], row["chassis_id_type"]) == ("c1", kind)
|
||||
assert (row["port_id"], row["port_id_type"]) == ("p1", kind)
|
||||
|
||||
def test_a_neighbour_without_ids_has_empty_ones(self):
|
||||
(row,) = _rows("lldp.eth0.via=CDPv2\nlldp.eth0.chassis.name=sw\n")
|
||||
|
||||
assert (row["chassis_id"], row["chassis_id_type"]) == ("", "")
|
||||
assert (row["port_id"], row["port_id_type"]) == ("", "")
|
||||
|
||||
|
||||
class TestOtherFields:
|
||||
def test_every_management_address_is_kept(self):
|
||||
phone = _rows()[0]
|
||||
|
||||
assert phone["management_addresses"] == ["10.0.0.21", "fe80::211:22ff:fe33:4401"]
|
||||
|
||||
def test_a_value_may_hold_an_equals_sign(self):
|
||||
assert _rows()[0]["system_description"] == "SIP phone, firmware a=1"
|
||||
|
||||
def test_capabilities_in_napalm_names_and_which_are_enabled(self):
|
||||
phone, pc, switch = _rows()
|
||||
|
||||
assert phone["capabilities"] == ["bridge", "telephone"]
|
||||
assert phone["enabled_capabilities"] == ["telephone"]
|
||||
assert pc["capabilities"] == ["bridge", "router", "wlan-access-point", "station"]
|
||||
assert pc["enabled_capabilities"] == ["station"]
|
||||
assert switch["capabilities"] == ["bridge", "router", "cvlan"]
|
||||
assert switch["enabled_capabilities"] == ["bridge", "cvlan"]
|
||||
|
||||
def test_every_napalm_capability_name(self):
|
||||
symbols = ["Other", "Repeater", "Bridge", "Router", "Wlan", "Tel", "Docsis", "Station"]
|
||||
output = "lldp.eth0.via=LLDP\n" + "".join(
|
||||
f"lldp.eth0.chassis.{s}.enabled=on\n" for s in symbols
|
||||
)
|
||||
|
||||
(row,) = _rows(output)
|
||||
|
||||
assert row["capabilities"] == [
|
||||
"other",
|
||||
"repeater",
|
||||
"bridge",
|
||||
"router",
|
||||
"wlan-access-point",
|
||||
"telephone",
|
||||
"docsis-cable-device",
|
||||
"station",
|
||||
]
|
||||
assert row["enabled_capabilities"] == row["capabilities"]
|
||||
|
||||
def test_a_capability_line_needs_on_or_off(self):
|
||||
(row,) = _rows(OPENWRT_ONE_NEIGHBOUR)
|
||||
|
||||
assert row["capabilities"] == []
|
||||
|
||||
|
||||
class TestReadOrFailure:
|
||||
"""A failed read raises; ``[]`` only when lldpctl ran and listed nobody."""
|
||||
|
||||
def test_no_output_is_no_neighbour(self):
|
||||
assert parse_lldpctl_keyvalue("") == []
|
||||
|
||||
def test_a_zero_exit_status_without_neighbours_is_no_neighbour(self):
|
||||
assert parse_lldpctl_keyvalue("__LLDPCTL_RC=0\n") == []
|
||||
|
||||
def test_a_missing_lldpctl_raises(self):
|
||||
with pytest.raises(ValueError, match="lldpctl: not found"):
|
||||
parse_lldpctl_keyvalue("sh: lldpctl: not found\n")
|
||||
|
||||
def test_a_non_zero_exit_status_raises_with_what_lldpctl_said(self):
|
||||
output = (
|
||||
"[WARN/control] unable to connect to socket /var/run/lldpd.socket: "
|
||||
"No such file or directory\n"
|
||||
"__LLDPCTL_RC=1\n"
|
||||
)
|
||||
|
||||
with pytest.raises(ValueError, match="unable to connect to socket"):
|
||||
parse_lldpctl_keyvalue(output)
|
||||
|
||||
def test_a_non_zero_exit_status_without_a_message_names_the_status(self):
|
||||
with pytest.raises(ValueError, match="exit status 127"):
|
||||
parse_lldpctl_keyvalue("__LLDPCTL_RC=127\n")
|
||||
|
||||
def test_with_status_zero_a_warning_beside_the_neighbours_is_ignored(self):
|
||||
output = (
|
||||
"lldpctl -f keyvalue 2>&1; echo __LLDPCTL_RC=$?\n"
|
||||
"[WARN/lldpctl] something lldpd wants to say\n"
|
||||
+ OPENWRT_ONE_NEIGHBOUR
|
||||
+ "__LLDPCTL_RC=0\n"
|
||||
)
|
||||
|
||||
assert [r["system_name"] for r in _rows(output)] == ["core-router"]
|
||||
|
||||
def test_a_line_outside_lldp_is_no_neighbour(self):
|
||||
output = "lldpd.config.tx-delay=30\n" + OPENWRT_ONE_NEIGHBOUR
|
||||
|
||||
assert len(_rows(output)) == 1
|
||||
|
||||
def test_a_key_without_a_known_field_is_skipped(self):
|
||||
output = "lldp.summary=1\nlldp.eth9.future-field=x\n" + OPENWRT_ONE_NEIGHBOUR
|
||||
|
||||
assert [r["local_port"] for r in _rows(output)] == ["eth0"]
|
||||
|
||||
|
||||
class TestCommand:
|
||||
def test_the_command_reads_the_neighbours_in_keyvalue_form(self):
|
||||
assert LLDPCTL_KEYVALUE_COMMAND.startswith("lldpctl -f keyvalue ")
|
||||
|
||||
def test_the_command_brings_back_the_error_text_and_the_exit_status(self):
|
||||
assert "2>&1" in LLDPCTL_KEYVALUE_COMMAND
|
||||
assert LLDPCTL_KEYVALUE_COMMAND.endswith("echo __LLDPCTL_RC=$?")
|
||||
File diff suppressed because it is too large.
Load diff
Loaded 100 of 131 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user