Compare commits

...
55 Commits
Author SHA1 Message Date
christianmanivong e2aebeadeb Merge pull request 'chore(release): 4.23.0' (#75) from chore/release-4.23.0 into main
driver / lint (push) Successful in 19s
driver / test (3.13) (push) Successful in 1m16s
driver / build (push) Successful in 30s
driver / security (push) Successful in 1m6s
driver / publish (push) Successful in 44s
driver / types (push) Successful in 1m18s
driver / lowest (push) Successful in 32s
driver / standard (push) Successful in 21s
driver / test (3.12) (push) Successful in 1m24s
CI / driver (push) Successful in 3m29s
2026-11-01 23:26:04 +00:00
christianmanivong 36cf117b95 chore(release): 4.23.0
driver / build (pull_request) Successful in 31s
driver / security (pull_request) Successful in 1m0s
CI / driver (pull_request) Successful in 2m22s
driver / standard (pull_request) Successful in 16s
driver / types (pull_request) Successful in 57s
driver / test (3.12) (pull_request) Successful in 1m2s
driver / test (3.13) (pull_request) Successful in 58s
driver / lowest (pull_request) Successful in 31s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 13s
2026-11-02 00:21:38 +01:00
christianmanivong 02d461036b Merge pull request 'feat: TLS certificates by trust on first use' (#74) from feat/tls-fingerprint-tofu into main
driver / lint (push) Successful in 15s
driver / test (3.13) (push) Successful in 57s
driver / lowest (push) Successful in 34s
driver / build (push) Successful in 27s
driver / security (push) Successful in 1m3s
driver / publish (push) Skipped
driver / types (push) Successful in 1m5s
driver / standard (push) Successful in 16s
driver / test (3.12) (push) Successful in 1m4s
CI / driver (push) Successful in 2m16s
2026-11-01 23:21:03 +00:00
christianmanivong 63039ce758 fix(types): the adapter's ignore holds with and without requests' stubs
driver / build (pull_request) Successful in 26s
driver / security (pull_request) Successful in 53s
CI / driver (pull_request) Successful in 2m12s
driver / standard (pull_request) Successful in 15s
driver / types (pull_request) Successful in 57s
driver / test (3.13) (pull_request) Successful in 1m4s
driver / test (3.12) (pull_request) Successful in 1m10s
driver / lowest (pull_request) Successful in 30s
driver / lint (pull_request) Successful in 14s
driver / publish (pull_request) Skipped
CI installs types-requests, so HTTPAdapter is typed there and the
ignore for subclassing an untyped base is unused; locally it is needed.
2026-11-02 00:15:55 +01:00
christianmanivong 516fb18c0e Merge remote-tracking branch 'origin/main' into feat/tls-fingerprint-tofu
driver / build (pull_request) Successful in 28s
driver / security (pull_request) Successful in 59s
CI / driver (pull_request) Failing after 2m34s
driver / standard (pull_request) Successful in 15s
driver / test (3.13) (pull_request) Successful in 1m14s
driver / types (pull_request) Failing after 1m35s
driver / test (3.12) (pull_request) Successful in 1m27s
driver / lowest (pull_request) Successful in 34s
driver / lint (pull_request) Successful in 14s
driver / publish (pull_request) Skipped
# Conflicts:
#	CHANGELOG.md
2026-11-02 00:06:53 +01:00
christianmanivong 8d32e70cd7 Merge pull request 'feat(fixes): configure_lldp_agent(interfaces) template' (#73) from feat/lldp-agent-template into main
driver / lint (push) Successful in 25s
driver / lowest (push) Successful in 34s
driver / test (3.13) (push) Successful in 1m8s
driver / build (push) Successful in 37s
driver / security (push) Successful in 1m15s
driver / publish (push) Skipped
driver / types (push) Successful in 1m15s
driver / standard (push) Successful in 24s
driver / test (3.12) (push) Failing after 6s
CI / driver (push) Failing after 2m19s
2026-11-01 23:06:48 +00:00
christianmanivong 51e8c849bb Merge remote-tracking branch 'origin/main' into feat/lldp-agent-template
driver / build (pull_request) Successful in 33s
driver / security (pull_request) Successful in 1m10s
CI / driver (pull_request) Successful in 2m45s
driver / standard (pull_request) Successful in 18s
driver / types (pull_request) Successful in 1m6s
driver / test (3.13) (pull_request) Successful in 1m6s
driver / test (3.12) (pull_request) Successful in 1m14s
driver / lowest (pull_request) Successful in 38s
driver / lint (pull_request) Successful in 14s
driver / publish (pull_request) Skipped
# Conflicts:
#	CHANGELOG.md
2026-11-02 00:01:46 +01:00
christianmanivong 6aadcc3d91 Merge pull request 'feat(snmp): let a health profile mark a value unknown' (#72) from feat/health-profile-unknown into main
driver / lint (push) Successful in 17s
driver / test (3.13) (push) Successful in 1m15s
driver / security (push) Failing after 3s
driver / lowest (push) Successful in 41s
driver / build (push) Successful in 31s
driver / publish (push) Skipped
driver / types (push) Successful in 1m4s
driver / standard (push) Successful in 19s
driver / test (3.12) (push) Successful in 1m17s
CI / driver (push) Failing after 2m6s
2026-11-01 23:01:40 +00:00
christianmanivong e98d4763f3 Merge remote-tracking branch 'origin/main' into feat/health-profile-unknown
driver / build (pull_request) Successful in 29s
driver / security (pull_request) Successful in 1m11s
CI / driver (pull_request) Successful in 2m57s
driver / standard (pull_request) Successful in 19s
driver / types (pull_request) Successful in 1m7s
driver / test (3.12) (pull_request) Successful in 1m8s
driver / test (3.13) (pull_request) Successful in 1m9s
driver / lowest (pull_request) Successful in 40s
driver / lint (pull_request) Successful in 18s
driver / publish (pull_request) Skipped
# Conflicts:
#	CHANGELOG.md
2026-11-01 23:56:39 +01:00
christianmanivong 24f4be3b7e Merge pull request 'feat: VMProvisionIncomplete names the VM a failed creation left behind' (#70) from feat/vm-create-partial-failure into main
driver / lint (push) Successful in 18s
driver / test (3.13) (push) Successful in 1m9s
driver / lowest (push) Successful in 35s
driver / build (push) Successful in 38s
driver / security (push) Successful in 1m10s
driver / publish (push) Skipped
driver / types (push) Successful in 1m6s
driver / standard (push) Successful in 19s
driver / test (3.12) (push) Successful in 1m16s
CI / driver (push) Successful in 2m38s
2026-11-01 22:56:34 +00:00
christianmanivong f4354f498e refactor: the SNMP health profile shapes in snmp_health_profile.py
models.py is over the file size limit and may not grow; the unknown
key pushed it past its recorded length. SnmpHealthProfile,
SnmpValueSpec, SnmpMemorySpec and SnmpInterfacesSpec move to
napalm_device_types/snmp_health_profile.py. The package exports them
as before and models re-exports them, so existing imports keep working.
The baseline records the smaller models.py.
2026-11-01 23:05:13 +01:00
christianmanivong 73771aba00 docs: a tighter create_vm_from_cloud_init docstring
driver / build (pull_request) Successful in 30s
driver / security (pull_request) Successful in 1m6s
CI / driver (pull_request) Successful in 2m52s
driver / standard (pull_request) Successful in 16s
driver / types (pull_request) Successful in 1m7s
driver / test (3.12) (pull_request) Successful in 1m12s
driver / lowest (pull_request) Successful in 35s
driver / test (3.13) (pull_request) Successful in 1m11s
driver / lint (pull_request) Successful in 18s
driver / publish (pull_request) Skipped
The VMProvisionIncomplete entry pushed the method past its recorded
length in the quality baseline. The image-cache note joins the
paragraph it repeated, and the Raises entry takes two lines; nothing
it says is dropped.
2026-11-01 23:02:46 +01:00
christianmanivong c9571d59f1 Merge remote-tracking branch 'origin/main' into feat/tls-fingerprint-tofu
# Conflicts:
#	CHANGELOG.md
2026-11-01 23:01:38 +01:00
christianmanivong cf862af026 Merge remote-tracking branch 'origin/main' into feat/lldp-agent-template
# Conflicts:
#	CHANGELOG.md
2026-11-01 23:01:38 +01:00
christianmanivong 83ea13f88e Merge remote-tracking branch 'origin/main' into feat/health-profile-unknown
# Conflicts:
#	CHANGELOG.md
2026-11-01 23:01:38 +01:00
christianmanivong 4938056d20 Merge remote-tracking branch 'origin/main' into feat/vm-create-partial-failure
# Conflicts:
#	CHANGELOG.md
2026-11-01 23:01:38 +01:00
christianmanivong 9c3be46d8a Merge pull request 'feat: HA state, backups, quorum votes, migrate_vm and node maintenance' (#71) from feat/vm-lifecycle into main
driver / lint (push) Successful in 15s
driver / test (3.13) (push) Successful in 53s
driver / lowest (push) Successful in 30s
driver / build (push) Successful in 27s
driver / security (push) Successful in 56s
driver / publish (push) Skipped
driver / types (push) Successful in 58s
driver / standard (push) Successful in 16s
driver / test (3.12) (push) Successful in 1m3s
CI / driver (push) Successful in 2m29s
2026-11-01 21:48:11 +00:00
christianmanivong a6289cb24a feat: HA state, backups, quorum votes, migrate_vm and node maintenance
driver / build (pull_request) Successful in 27s
driver / security (pull_request) Successful in 1m3s
CI / driver (pull_request) Successful in 2m30s
driver / standard (pull_request) Successful in 15s
driver / types (pull_request) Successful in 58s
driver / test (3.12) (pull_request) Successful in 59s
driver / lowest (pull_request) Successful in 30s
driver / test (3.13) (pull_request) Successful in 58s
driver / lint (pull_request) Successful in 12s
driver / publish (pull_request) Skipped
get_clusters() may also report the cluster's HA resources, backup jobs, the
newest backup per guest and storage, and each node's quorum votes (all
NotRequired; None means not read). HypervisorDriver declares
get_vm_migration_preflight(), migrate_vm() and set_node_maintenance(); the
preflight shape lives in vm_migration.py.

Part of NetOrk/netork#1820 (stage 7: #759, #760, #761).
2026-11-01 22:15:01 +01:00
christianmanivong 46c1c9dc4e feat: TLS certificates by trust on first use
driver / build (pull_request) Successful in 34s
driver / security (pull_request) Successful in 1m13s
CI / driver (pull_request) Failing after 3m4s
driver / standard (pull_request) Successful in 17s
driver / types (pull_request) Failing after 1m10s
driver / test (3.12) (pull_request) Successful in 1m14s
driver / test (3.13) (pull_request) Successful in 1m4s
driver / lowest (pull_request) Successful in 38s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 20s
Devices ship self-signed certificates, so a driver either fails the CA
check on every one of them or turns verification off and sends the
login to whatever answers on the address. tls_certificates pins the
certificate by its SHA-256 fingerprint instead, shaped like
ssh_host_keys: the caller passes the fingerprint to expect by port
(optional_args["tls_fingerprints"]), the driver refuses any other
certificate before a request carries credentials, and every certificate
seen goes to optional_args["tls_certificate_observer"]. Without a
fingerprint the first certificate is accepted.

CertificateCheck checks the certificate in a handshake of its own, then
pins a requests session to it (urllib3's assert_fingerprint, direct and
through a proxy), so no other host can step in between the check and
the login.

Refs NAPALM/napalm-hpe-aruba-procurve#17
2026-11-01 22:13:20 +01:00
christianmanivong bc17862a44 feat(fixes): configure_lldp_agent(interfaces) template
driver / build (pull_request) Successful in 28s
driver / security (pull_request) Successful in 58s
CI / driver (pull_request) Successful in 2m32s
driver / standard (pull_request) Successful in 15s
driver / types (pull_request) Successful in 57s
driver / test (3.12) (pull_request) Successful in 1m7s
driver / test (3.13) (pull_request) Successful in 56s
driver / lowest (pull_request) Successful in 30s
driver / lint (pull_request) Successful in 12s
driver / publish (pull_request) Skipped
LLDP getters used to install, start and reconfigure lldpd on their own,
on every poll. Getters only read now, so starting the agent needs an
action the application calls. The template installs the agent through
ensure_package, has the driver write the interfaces it runs on, then
enables and restarts it through manage_service. The caller names the
interfaces: an agent on every interface also advertises on a WAN port.

Refs NAPALM/napalm-openwrt#8
2026-11-01 22:11:21 +01:00
christianmanivong 9f6ffa8a25 feat(snmp): let a health profile mark a value unknown
driver / build (pull_request) Successful in 27s
driver / security (pull_request) Successful in 1m1s
CI / driver (pull_request) Failing after 2m30s
driver / standard (pull_request) Failing after 15s
driver / types (pull_request) Successful in 58s
driver / test (3.12) (pull_request) Successful in 1m3s
driver / test (3.13) (pull_request) Successful in 56s
driver / lowest (pull_request) Successful in 35s
driver / lint (pull_request) Successful in 13s
driver / publish (pull_request) Skipped
SnmpValueSpec gains `unknown`. A driver whose device's agent answers a
value wrongly says so in its SNMP_HEALTH_PROFILE, and the application
reports no value for it instead of the standard reading or a default.
OpenBSD's net-snmp reports the CPU as never idle, so a reading of
ssCpuIdle shows every OpenBSD host at 100 % (NAPALM/napalm-bsd#7).
2026-11-01 22:11:19 +01:00
christianmanivong 4150570a94 feat: VMProvisionIncomplete names the VM a failed creation left behind
driver / build (pull_request) Successful in 35s
driver / security (pull_request) Successful in 1m15s
CI / driver (pull_request) Failing after 3m1s
driver / standard (pull_request) Failing after 17s
driver / types (pull_request) Successful in 1m5s
driver / test (3.12) (pull_request) Successful in 1m13s
driver / test (3.13) (pull_request) Successful in 1m6s
driver / lowest (pull_request) Successful in 37s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 16s
create_vm_from_cloud_init raised a plain error once the VM existed, so the
caller had no id to remove it by. The new RuntimeError carries vmid and
node, with the failure as its cause; a failure before the VM exists raises
as before.

Refs NAPALM/napalm-proxmox#28
2026-11-01 22:09:01 +01:00
christianmanivong a809e6c02f Merge pull request 'feat: get_clusters() and create_vm_from_cloud_init(node=)' (#68) from feat/cluster-inventory into main
driver / lint (push) Successful in 20s
driver / test (3.13) (push) Successful in 1m21s
driver / build (push) Successful in 35s
driver / security (push) Successful in 1m16s
driver / publish (push) Skipped
driver / types (push) Successful in 1m30s
driver / lowest (push) Successful in 39s
driver / standard (push) Failing after 7s
driver / test (3.12) (push) Successful in 1m26s
CI / driver (push) Failing after 2m49s
2026-11-01 18:41:05 +00:00
christianmanivong 5b894913e1 refactor: the cluster shapes in clusters.py, a shorter provisioning docstring
driver / build (pull_request) Successful in 27s
driver / security (pull_request) Successful in 1m4s
CI / driver (pull_request) Successful in 2m16s
driver / standard (pull_request) Successful in 15s
driver / types (pull_request) Successful in 1m1s
driver / test (3.13) (pull_request) Successful in 57s
driver / test (3.12) (pull_request) Successful in 1m4s
driver / lowest (pull_request) Successful in 32s
driver / lint (pull_request) Successful in 14s
driver / publish (pull_request) Skipped
models.py is over the file size limit and may not grow, and the declaration
of create_vm_from_cloud_init() may not get longer: the four cluster shapes
move to napalm_device_types/clusters.py (exported from the package as
before), and the docstring says the same in fewer lines.

Refs NetOrk/netork#484
2026-11-01 17:08:10 +01:00
christianmanivong 00aa5efef0 feat: get_clusters() and create_vm_from_cloud_init(node=)
driver / build (pull_request) Successful in 28s
driver / security (pull_request) Successful in 1m0s
CI / driver (pull_request) Failing after 2m32s
driver / standard (pull_request) Failing after 19s
driver / types (pull_request) Successful in 1m10s
driver / test (3.13) (pull_request) Successful in 1m13s
driver / test (3.12) (pull_request) Successful in 1m19s
driver / lowest (pull_request) Successful in 32s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 16s
A hypervisor reports every cluster its connection can see: the cluster's
name and quorum, its nodes (offline ones too, the local one marked) with
CPU and memory, and the storages and networks with the nodes that see
each. A standalone host is a cluster of one without a name; a manager
reports each cluster and standalone host it runs. Declared only, so a
capability check stays truthful.

create_vm_from_cloud_init() takes the node to create the VM on; a driver
that can only create on its own node refuses another.

Refs NetOrk/netork#484, NetOrk/netork#1822
2026-11-01 16:38:38 +01:00
christianmanivong 97cb8779c5 Merge pull request 'feat: netOrk's own account through the current login (NetOrk/netork#1817)' (#67) from feature/system-user-1817 into main
driver / lint (push) Successful in 15s
driver / test (3.13) (push) Successful in 1m0s
driver / lowest (push) Successful in 30s
driver / build (push) Successful in 26s
driver / security (push) Successful in 1m0s
driver / publish (push) Skipped
driver / types (push) Successful in 57s
driver / standard (push) Successful in 16s
driver / test (3.12) (push) Successful in 1m6s
CI / driver (push) Successful in 2m16s
2026-11-01 12:39:41 +00:00
christianmanivong 47751d12d6 fix: another service's account of the same name is left alone (NetOrk/netork#1817)
driver / security (pull_request) Successful in 55s
driver / build (pull_request) Successful in 26s
CI / driver (pull_request) Successful in 56s
driver / standard (pull_request) Successful in 15s
driver / types (pull_request) Successful in 1m1s
driver / test (3.13) (pull_request) Successful in 59s
driver / lowest (pull_request) Successful in 30s
driver / test (3.12) (pull_request) Successful in 56s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 14s
On bw-01, netork is the satellite's own user (nologin, in docker).
add_service_account opened it, gave it the passwordless sudo rule and netOrk's
key, and only the login as it failed. An account that is there and cannot log
in (nologin, false) is now refused before anything changes.
2026-11-01 13:36:38 +01:00
christianmanivong 3f59023f46 feat: a fixed uid for netOrk's own account (NetOrk/netork#1817)
driver / security (pull_request) Successful in 1m3s
driver / build (pull_request) Successful in 26s
CI / driver (pull_request) Successful in 1m10s
driver / standard (pull_request) Successful in 16s
driver / types (pull_request) Successful in 1m8s
driver / test (3.13) (pull_request) Successful in 1m2s
driver / lowest (pull_request) Successful in 31s
driver / test (3.12) (pull_request) Successful in 1m10s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 14s
add_service_account(uid=) makes a new account with that uid once no other
account has it, through the driver's _account_create_with_uid; an account that
is there keeps its own, as a uid is never changed under files that carry it.
The infrastructure's build standard wants the same uid on every host.
2026-11-01 12:42:30 +01:00
christianmanivong 666be091e3 feat: netOrk's own account through the current login (NetOrk/netork#1817)
driver / build (pull_request) Successful in 38s
driver / security (pull_request) Successful in 1m19s
CI / driver (pull_request) Failing after 2m35s
driver / standard (pull_request) Successful in 21s
driver / types (pull_request) Successful in 1m10s
driver / test (3.12) (pull_request) Successful in 1m9s
driver / test (3.13) (pull_request) Successful in 1m6s
driver / lowest (pull_request) Successful in 50s
driver / lint (pull_request) Failing after 7s
driver / publish (pull_request) Skipped
add_service_account(name, public_key) creates or reopens the account through
the login the driver uses today: a key and no password, and a rule of its own
in 85-service-<name> that lets it run everything as root without a password,
checked with visudo and sudo -l. Root is refused; run again with the account
the driver already logs in with, it only adds what is missing.
2026-11-01 12:03:01 +01:00
christianmanivong 25cc264306 Merge pull request 'fix(apt_proxy): https sources take the http proxy too (NetOrk/netork#1811)' (#64) from fix/apt-https-proxy into main
driver / lint (push) Successful in 15s
driver / test (3.13) (push) Successful in 1m4s
driver / lowest (push) Successful in 30s
driver / build (push) Successful in 26s
driver / security (push) Successful in 59s
driver / publish (push) Skipped
driver / types (push) Successful in 1m4s
driver / standard (push) Successful in 15s
driver / test (3.12) (push) Successful in 1m10s
CI / driver (push) Successful in 2m18s
2026-11-01 07:25:56 +00:00
christianmanivong 86959b7599 Merge remote-tracking branch 'origin/main' into fix/apt-https-proxy
driver / build (pull_request) Successful in 33s
driver / security (pull_request) Successful in 1m7s
CI / driver (pull_request) Successful in 2m50s
driver / standard (pull_request) Successful in 16s
driver / types (pull_request) Successful in 1m11s
driver / test (3.12) (pull_request) Successful in 1m12s
driver / test (3.13) (pull_request) Successful in 1m0s
driver / lowest (pull_request) Successful in 33s
driver / lint (pull_request) Successful in 15s
driver / publish (pull_request) Skipped
2026-11-01 08:21:18 +01:00
christianmanivong 8664accbc4 Merge pull request 'refactor(ssh_server): split three functions under the standard's line limit (#65)' (#66) from fix/ssh-server-ratchet into main
driver / lint (push) Successful in 17s
driver / test (3.13) (push) Successful in 1m7s
driver / lowest (push) Successful in 31s
driver / build (push) Successful in 28s
driver / security (push) Successful in 1m16s
driver / publish (push) Skipped
driver / types (push) Successful in 1m1s
driver / standard (push) Successful in 21s
driver / test (3.12) (push) Successful in 1m3s
CI / driver (push) Successful in 2m37s
2026-11-01 07:21:11 +00:00
christianmanivong 449a6b13ca refactor(ssh_server): split three functions under the standard's line limit (#65)
driver / build (pull_request) Successful in 33s
driver / security (pull_request) Successful in 1m9s
CI / driver (pull_request) Successful in 3m9s
driver / standard (pull_request) Successful in 17s
driver / types (pull_request) Successful in 1m18s
driver / test (3.12) (pull_request) Successful in 1m28s
driver / test (3.13) (pull_request) Successful in 1m18s
driver / lowest (pull_request) Successful in 35s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 15s
harden_ssh_server (70), _apply_or_restore (51) and exempt_from_ssh_penalties
(81) had grown past the 50-line limit with #63, and `driver / standard` was
red on main. The reading and planning, the putting back, the session's
address, the reasons to leave the exemption and its write move into methods
of their own. Same behaviour, same tests; harden_ssh_server leaves the
baseline.
2026-11-01 08:17:27 +01:00
christianmanivong c71ddd7fe6 refactor(apt_proxy): smaller parts, under the standard's complexity limit
driver / build (pull_request) Successful in 55s
driver / security (pull_request) Successful in 1m33s
CI / driver (pull_request) Failing after 3m0s
driver / standard (pull_request) Failing after 20s
driver / types (pull_request) Successful in 1m8s
driver / test (3.13) (pull_request) Successful in 1m6s
driver / test (3.12) (pull_request) Successful in 1m12s
driver / lowest (pull_request) Successful in 47s
driver / lint (pull_request) Successful in 17s
driver / publish (pull_request) Skipped
The quality ratchet counted _dump, parse_apt_proxy and _https_direct over
complexity 10 (NetOrk/netork#1811). Same behaviour, same tests.
2026-11-01 08:13:20 +01:00
christianmanivong 3b6c5f9585 fix(apt_proxy): https sources take the http proxy too (NetOrk/netork#1811)
driver / build (pull_request) Successful in 28s
driver / security (pull_request) Successful in 1m1s
CI / driver (pull_request) Failing after 2m22s
driver / standard (pull_request) Failing after 18s
driver / types (pull_request) Successful in 1m7s
driver / test (3.13) (pull_request) Successful in 1m3s
driver / test (3.12) (pull_request) Successful in 1m7s
driver / lowest (pull_request) Successful in 33s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 17s
apt's https method asks Acquire::https::Proxy and then Acquire::http::Proxy,
for the host and then for all. An 00proxy with the http line alone therefore
sent download.docker.com and every other https source to apt-cacher-ng, which
answers 403 CONNECT denied without a PassThroughPattern, and the whole upgrade
failed.

- The report carries https_proxy, https_per_host and https_settings;
  https_proxy_for(report, host) is the proxy apt takes for one https host.
- check_https_source() fetches one https source the way apt does.
- configure_package_proxy gains an https step: one https source through the
  new proxy. Where the proxy refuses it, 00proxy also says
  Acquire::https::Proxy "DIRECT", later https lines to the same proxy are
  commented out, and the source is fetched direct. An HTTP error from the
  mirror counts as let through.
- apt_fetch_command takes https URLs; comment_out_command takes a scheme.
2026-11-01 07:58:05 +01:00
christianmanivong 1c71fe6896 Merge pull request 'feat: LocalAccountsMixin, personal accounts added and locked again (NetOrk/netork#1793)' (#62) from feat/local-accounts into main
driver / lint (push) Successful in 14s
driver / lowest (push) Failing after 5s
driver / test (3.13) (push) Successful in 1m2s
driver / build (push) Successful in 31s
driver / security (push) Successful in 1m11s
driver / publish (push) Skipped
driver / types (push) Successful in 1m6s
driver / standard (push) Failing after 16s
driver / test (3.12) (push) Successful in 1m4s
CI / driver (push) Failing after 2m47s
Reviewed-on: #62
2026-10-31 23:45:34 +00:00
christianmanivong 9159106837 Merge remote-tracking branch 'origin/main' into feat/local-accounts
driver / build (pull_request) Successful in 32s
driver / security (pull_request) Successful in 59s
CI / driver (pull_request) Failing after 2m35s
driver / standard (pull_request) Failing after 18s
driver / types (pull_request) Successful in 59s
driver / test (3.12) (pull_request) Successful in 1m7s
driver / lowest (pull_request) Successful in 31s
driver / test (3.13) (pull_request) Successful in 58s
driver / lint (pull_request) Successful in 13s
driver / publish (pull_request) Skipped
# Conflicts:
#	CHANGELOG.md
2026-11-01 00:45:06 +01:00
christianmanivong 28ff14589b Merge pull request 'feat: the managing client and sshd's per-source penalties (NetOrk/netork#1800)' (#63) from feat/ssh-penalties-1800 into main
driver / lint (push) Successful in 14s
driver / test (3.13) (push) Successful in 58s
driver / lowest (push) Successful in 30s
driver / build (push) Successful in 30s
driver / security (push) Successful in 1m4s
driver / publish (push) Skipped
driver / types (push) Successful in 1m2s
driver / standard (push) Failing after 17s
driver / test (3.12) (push) Successful in 1m4s
CI / driver (push) Failing after 2m21s
Reviewed-on: #63
2026-10-31 23:29:09 +00:00
christianmanivong 5fffb3b886 feat: the managing client and sshd's per-source penalties (NetOrk/netork#1800)
driver / build (pull_request) Successful in 30s
driver / security (pull_request) Successful in 1m4s
CI / driver (pull_request) Failing after 2m24s
driver / standard (pull_request) Failing after 9s
driver / types (pull_request) Successful in 1m14s
driver / test (3.12) (pull_request) Successful in 1m11s
driver / test (3.13) (pull_request) Successful in 1m7s
driver / lowest (pull_request) Successful in 33s
driver / lint (pull_request) Successful in 22s
driver / publish (pull_request) Skipped
OpenSSH 9.8+ penalises a source whose connections do not authenticate --
a scanner's probes -- and then drops every connection from it, the
managing client's included.

- OpenSshServerMixin.exempt_from_ssh_penalties puts the address the host
  sees the client at ($SSH_CONNECTION) into PerSourcePenaltyExemptList,
  next to the host's own exemptions, in its own drop-in. Only where sshd
  has the penalties (an older one refuses the directive) and includes the
  drop-in directory; checked with sshd -t, sshd -T and a new session, and
  taken back on any failure. The check-and-restore step of
  harden_ssh_server is shared for it.
- retry_refused tries a connection again after 1, 3, 7 and 15 s while
  sshd closes it before its banner; HostKeyCheck.connect uses it.
2026-11-01 00:24:11 +01:00
christianmanivong 2846824a21 feat: a host may add a key file sshd reads (NetOrk/netork#1793)
driver / build (pull_request) Successful in 32s
driver / security (pull_request) Successful in 1m5s
CI / driver (pull_request) Successful in 2m45s
driver / standard (pull_request) Successful in 17s
driver / types (pull_request) Successful in 1m6s
driver / test (3.12) (pull_request) Successful in 1m14s
driver / lowest (pull_request) Successful in 39s
driver / test (3.13) (pull_request) Successful in 1m23s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 14s
_account_key_files says which files hold an account's keys; the default is
~/.ssh/authorized_keys and authorized_keys2. OpenMediaVault adds the file it
deploys under /var/lib/openmediavault/ssh/authorized_keys, so reading an
account back sees every key sshd would take.
2026-10-31 23:31:05 +01:00
christianmanivong 0545d1d814 feat: a driver may keep the account keys elsewhere (NetOrk/netork#1793)
driver / build (pull_request) Successful in 34s
driver / security (pull_request) Successful in 1m9s
CI / driver (pull_request) Successful in 2m44s
driver / standard (pull_request) Successful in 16s
driver / types (pull_request) Successful in 1m6s
driver / test (3.13) (pull_request) Successful in 1m8s
driver / test (3.12) (pull_request) Successful in 1m14s
driver / lowest (pull_request) Successful in 36s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 13s
OpenMediaVault keeps a user's keys in its own database and deploys them
to /var/lib/openmediavault/ssh/authorized_keys, and creates a home only
when its home directory setting is on. The key store becomes three hooks
with the ~/.ssh default: _account_read_keys, _account_add_key and
_account_remove_keys, each given the account's home.
2026-10-31 23:29:46 +01:00
christianmanivong 49c65cbd5d feat: LocalAccountsMixin, personal accounts added and locked again (NetOrk/netork#1793)
driver / build (pull_request) Successful in 36s
driver / security (pull_request) Successful in 1m21s
CI / driver (pull_request) Successful in 3m6s
driver / standard (pull_request) Successful in 25s
driver / types (pull_request) Successful in 1m36s
driver / test (3.13) (pull_request) Successful in 1m15s
driver / test (3.12) (pull_request) Successful in 1m41s
driver / lowest (pull_request) Successful in 46s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 30s
get_local_accounts(names) reads a person's account back: groups, lock,
sudo, key files. add_local_account() creates or reopens one with the
person's key, optional password hash and optional sudo rule;
lock_local_account() takes the keys and sudo away and refuses every
login, keeping the home.

Every command root runs is one fixed command, so a sudoers allowlist can
name it. The key files are read and written as the person (sudo -n -u),
never as root, since a link in a home the person owns may point anywhere.
The sudo rule is a drop-in of its own, checked with visudo before and
sudo -l after; a rule elsewhere (cloud-init's) is reported, not edited.
The login user's own account and system accounts are refused.

A driver supplies creating, opening, locking, reading the lock and
leaving a group.
2026-10-31 23:24:27 +01:00
christianmanivong 888a7229dc Merge pull request 'feat: SudoHistoryMixin, the sudo password left in shell histories (NetOrk/netork#1090)' (#60) from feat/sudo-history-cleanup into main
driver / lint (push) Successful in 16s
driver / test (3.13) (push) Successful in 1m3s
driver / lowest (push) Successful in 32s
driver / build (push) Successful in 34s
driver / security (push) Successful in 1m5s
driver / publish (push) Skipped
driver / types (push) Successful in 1m1s
driver / standard (push) Successful in 17s
driver / test (3.12) (push) Successful in 1m10s
CI / driver (push) Successful in 2m25s
2026-10-30 20:47:48 +00:00
christianmanivong 0f0064c2d7 test: the fake host never answers with this machine's uid
driver / build (pull_request) Successful in 30s
driver / security (pull_request) Successful in 1m2s
CI / driver (pull_request) Successful in 2m32s
driver / standard (pull_request) Successful in 19s
driver / types (pull_request) Successful in 1m14s
driver / test (3.12) (pull_request) Successful in 1m14s
driver / test (3.13) (pull_request) Successful in 1m16s
driver / lowest (pull_request) Successful in 36s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 15s
CI runs as root, which the owner step skips, so three owner tests failed there
while they passed for a normal user.
2026-10-30 21:44:40 +01:00
christianmanivong cb90fa9835 feat: SudoHistoryMixin, the sudo password left in shell histories (NetOrk/netork#1090)
driver / build (pull_request) Successful in 29s
driver / security (pull_request) Successful in 1m8s
CI / driver (pull_request) Failing after 2m24s
driver / standard (pull_request) Failing after 7s
driver / types (pull_request) Successful in 1m8s
driver / test (3.12) (pull_request) Failing after 1m9s
driver / test (3.13) (pull_request) Failing after 1m3s
driver / lowest (pull_request) Failing after 33s
driver / lint (pull_request) Successful in 20s
driver / publish (pull_request) Skipped
A driver that typed `echo '<password>' | sudo -S -p "" <command>` into an
interactive shell left that line, password and all, in the login user's
shell history once per command, and where only the part before the first
`&&` ran as root, files root should own were left to the login user.

get_history_sudo_lines() counts the lines per history file with grep -c,
as the login user, without reading any back. remove_history_sudo_lines()
deletes exactly those lines with sed -i, overwrites the backup with
shred -u, and as root gives the driver's SUDO_ROOT_FILES back to root:root
where each is a regular file, not a link, owned by the login user. It
reports each file's lines before and after. Nothing goes to stdin.

The driver names the marker and the files and supplies run_command.
2026-10-30 21:09:24 +01:00
christianmanivong 551ed0a29d Merge pull request 'feat: NetworkTargetDict.uplinks, the NICs a bridge leaves through (NetOrk/netork#1325)' (#58) from feat/network-target-uplinks into main
driver / lint (push) Successful in 16s
driver / test (3.13) (push) Successful in 1m8s
driver / lowest (push) Successful in 31s
driver / build (push) Successful in 32s
driver / security (push) Successful in 1m8s
driver / publish (push) Skipped
driver / types (push) Successful in 1m3s
driver / standard (push) Successful in 18s
driver / test (3.12) (push) Successful in 1m16s
CI / driver (push) Successful in 2m34s
2026-10-30 09:54:19 +00:00
christianmanivong 0dc1a598e2 Merge remote-tracking branch 'origin/main' into feat/network-target-uplinks
driver / build (pull_request) Successful in 28s
driver / security (pull_request) Successful in 1m5s
CI / driver (pull_request) Successful in 3m1s
driver / standard (pull_request) Successful in 24s
driver / types (pull_request) Successful in 1m2s
driver / test (3.12) (pull_request) Successful in 1m9s
driver / lowest (pull_request) Successful in 32s
driver / test (3.13) (pull_request) Successful in 59s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 16s
# Conflicts:
#	CHANGELOG.md
2026-10-30 10:20:48 +01:00
christianmanivong 131346c228 Merge pull request 'feat: ARP_AGE_UNKNOWN, the age of a neighbour the device does not report' (#57) from fix/arp-unknown-age into main
driver / lint (push) Successful in 17s
driver / test (3.13) (push) Successful in 1m7s
driver / lowest (push) Successful in 33s
driver / build (push) Successful in 42s
driver / security (push) Successful in 1m16s
driver / publish (push) Skipped
driver / types (push) Successful in 1m5s
driver / standard (push) Successful in 19s
driver / test (3.12) (push) Successful in 1m12s
CI / driver (push) Successful in 3m5s
2026-10-30 09:20:21 +00:00
christianmanivong a5b7b540c6 feat: NetworkTargetDict.uplinks, the NICs a bridge leaves through (NetOrk/netork#1325)
driver / build (pull_request) Successful in 32s
driver / security (pull_request) Successful in 1m12s
CI / driver (pull_request) Successful in 2m47s
driver / standard (pull_request) Successful in 18s
driver / types (pull_request) Successful in 1m1s
driver / test (3.12) (pull_request) Successful in 1m3s
driver / test (3.13) (pull_request) Successful in 59s
driver / lowest (pull_request) Successful in 36s
driver / lint (pull_request) Successful in 14s
driver / publish (pull_request) Skipped
A VM on a VLAN-aware bridge without a VLAN tag only gets an address when
the switch port behind the bridge carries an untagged network. To find
that port, the caller needs the bridge's physical NICs: their LLDP
neighbours name the switch and the port.

uplinks lists them in groups, one per way out of the bridge: a bond's
slaves together, a single NIC alone. A bond is one link, so a caller can
let one slave's neighbour speak for the others when lldpd sees only one.
Internal, VLAN and virtual ports are no uplinks. A missing key means the
driver cannot tell; an empty list is a bridge without uplink.
2026-10-30 09:27:23 +01:00
christianmanivong 874c6c5a92 feat: ARP_AGE_UNKNOWN, the age of a neighbour the device does not report
driver / build (pull_request) Successful in 35s
driver / security (pull_request) Successful in 1m15s
CI / driver (pull_request) Successful in 3m3s
driver / standard (pull_request) Successful in 17s
driver / types (pull_request) Successful in 1m12s
driver / test (3.12) (pull_request) Successful in 1m13s
driver / test (3.13) (pull_request) Successful in 1m8s
driver / lowest (pull_request) Successful in 31s
driver / lint (pull_request) Successful in 18s
driver / publish (pull_request) Skipped
NAPALM requires an age on every entry of get_arp_table and
get_ipv6_neighbors_table. Many devices do not report one, and drivers
filled in 0.0, which means "confirmed just now". ARP_AGE_UNKNOWN (-1.0, as
NAPALM's IOS driver reports) is the value for "not reported"; the README
states the contract.

NetOrk/netork#1327
2026-10-30 08:37:08 +01:00
christianmanivong 145c2c668b Merge pull request 'chore: Python 3.12 is the floor (NAPALM/driver-standard#6)' (#56) from chore/python-312-floor into main
driver / lint (push) Successful in 13s
driver / test (3.13) (push) Successful in 57s
driver / lowest (push) Successful in 30s
driver / build (push) Successful in 27s
driver / security (push) Successful in 59s
driver / publish (push) Skipped
driver / types (push) Successful in 55s
driver / standard (push) Successful in 17s
driver / test (3.12) (push) Successful in 58s
CI / driver (push) Successful in 2m25s
2026-10-27 10:19:37 +00:00
Christian Manivong 1cef3385e8 fix: the floor on napalm is 5.1 (NAPALM/driver-standard#6)
driver / build (pull_request) Successful in 29s
driver / security (pull_request) Successful in 1m1s
CI / driver (pull_request) Successful in 2m49s
driver / standard (pull_request) Successful in 21s
driver / types (pull_request) Successful in 1m5s
driver / test (3.12) (pull_request) Successful in 1m13s
driver / test (3.13) (pull_request) Successful in 55s
driver / lowest (pull_request) Successful in 28s
driver / publish (pull_request) Skipped
driver / lint (pull_request) Successful in 14s
napalm 5.0 and older import pkg_resources, which a Python 3.12 venv no
longer has, so the lowest job failed on them.
2026-10-27 10:49:27 +01:00
Christian Manivong fbd3cf0129 chore: Python 3.12 is the floor (NAPALM/driver-standard#6)
driver / build (pull_request) Successful in 31s
driver / security (pull_request) Successful in 1m13s
CI / driver (pull_request) Failing after 2m50s
driver / standard (pull_request) Successful in 21s
driver / types (pull_request) Successful in 1m3s
driver / test (3.12) (pull_request) Successful in 1m2s
driver / lowest (pull_request) Failing after 30s
driver / test (3.13) (pull_request) Successful in 1m0s
driver / lint (pull_request) Successful in 14s
driver / publish (pull_request) Skipped
netOrk runs every driver in its own containers on Python 3.12, so 3.10
and 3.11 are no longer supported or tested. requires-python, mypy and
ruff move to 3.12, along with what ruff's pyupgrade rules then rewrite.
2026-10-27 10:18:55 +01:00
christianmanivong e400c4359c Merge pull request 'feat: shutdown_host and begin_stop_vm, shutting down without waiting' (#55) from feat/shutdown-requests into main
driver / lint (push) Successful in 14s
driver / test (3.13) (push) Successful in 1m7s
driver / build (push) Successful in 30s
driver / security (push) Successful in 1m9s
driver / publish (push) Skipped
driver / types (push) Successful in 1m25s
driver / test (3.11) (push) Successful in 1m21s
driver / lowest (push) Successful in 34s
driver / standard (push) Successful in 15s
driver / test (3.10) (push) Successful in 1m23s
driver / test (3.12) (push) Successful in 1m22s
CI / driver (push) Successful in 3m20s
2026-10-27 07:41:23 +00:00
christianmanivong 4d3574cd60 feat: shutdown_host and begin_stop_vm, shutting down without waiting
driver / test (3.10) (pull_request) Failing after 6s
driver / test (3.11) (pull_request) Successful in 1m13s
driver / build (pull_request) Successful in 29s
driver / security (pull_request) Successful in 1m8s
CI / driver (pull_request) Failing after 2m41s
driver / standard (pull_request) Successful in 20s
driver / types (pull_request) Successful in 1m4s
driver / test (3.12) (pull_request) Successful in 1m8s
driver / test (3.13) (pull_request) Successful in 1m1s
driver / lowest (pull_request) Successful in 43s
driver / lint (pull_request) Successful in 18s
driver / publish (pull_request) Skipped
HostShutdownMixin.shutdown_host(), on DeviceTypeDriver: a device shuts
itself down cleanly through its own means; declared, so hasattr says
whether a driver can. There is no forced variant.

VmStopRequestMixin.begin_stop_vm(name): ask for a VM's clean shutdown
and return at once, for a caller that waits on a timer and reads
get_vm_status instead of blocking in stop_vm. No role demands it.

For netOrk's graph workflows, which switch a hypervisor off when CI has
nothing for it (NetOrk/netork#1636).
2026-10-27 00:36:54 +01:00
41 changed files with 5556 additions and 210 deletions

No files matched your search

+146
View File
@@ -7,6 +7,152 @@ recorded here; their history is the git log.
## [Unreleased]
## [4.23.0] - 2026-11-02
### Changed
- Python 3.12 is the floor (`requires-python = ">=3.12"`): netOrk runs every driver
on 3.12, so 3.10 and 3.11 are no longer tested (NAPALM/driver-standard#6).
- The floor on napalm is 5.1: older releases import `pkg_resources`, which a Python
3.12 venv no longer has.
- `requests` is a declared dependency (floor 2.25.0), for `tls_certificates`; it came
with napalm before.
- The SNMP health profile shapes (`SnmpHealthProfile`, `SnmpValueSpec`,
`SnmpMemorySpec`, `SnmpInterfacesSpec`) are defined in
`napalm_device_types.snmp_health_profile`. The package and `models` still export
them, so an existing import keeps working.
### Added
- `HypervisorDriver.get_vm_migration_preflight(name)`, `migrate_vm(name, target_node,
online=True, target_storage=None, timeout=3600)` and `set_node_maintenance(node,
enabled)`: which nodes of its cluster a guest could move to and what blocks it
(`MigrationPreflightDict`: `allowed_nodes`, `refused_nodes` with the platform's
reason, `local_disks` the migration copies, `local_resources` that block it), the
move itself, blocking until the platform's task ended and raising with the
platform's own cause, and a node's HA maintenance mode. Declared only; netOrk
probes each with `hasattr`.
- `get_clusters()` may report HA, backups and quorum votes: `ClusterDict.ha_resources`
(`HAResourceDict`: requested and current state, node, PVE 8 group, PVE 9 rules),
`backup_jobs` (`BackupJobDict`: schedule, storage, mode, the guests a job covers)
and `backups` (`BackupDict`: per guest and storage the newest backup, its size
and how many there are), and `ClusterNodeDict.votes`. All `NotRequired`, so
drivers written before stay valid; None means could not be read, `[]` read and
empty.
- `napalm_device_types.tls_certificates`: TLS certificates by trust on first use, the
counterpart of `ssh_host_keys` for a driver's HTTPS API. A driver takes the
fingerprint to expect from `optional_args["tls_fingerprints"]` (by port, the SHA-256
of the DER certificate, `SHA256:AB:CD:…`) and refuses any other certificate with
`TlsCertificateMismatch` (a `ConnectionException` naming both fingerprints) before a
password is sent; no CA is consulted, so a self-signed certificate verifies. Without
a fingerprint the first certificate is accepted. Every certificate seen goes to
`optional_args["tls_certificate_observer"]`. `CertificateCheck.verify_server()`
checks the certificate in a handshake of its own, `pin(session)` makes every later
connection of a `requests` session present the same one, `refusing()` turns a refusal
there into the mismatch, `observe()` reports it once the login stands
(NAPALM/napalm-hpe-aruba-procurve#17).
- `LldpAgentMixin.configure_lldp_agent(interfaces)` (`napalm_device_types.fixes`): run
an LLDP agent on the interfaces the caller names. The template installs the agent
(`ensure_package(LLDP_AGENT_PACKAGE)`), has the driver write the interface list
(`_write_lldp_agent_config`), then enables and restarts the service
(`manage_service(LLDP_AGENT_SERVICE, …)`); both names default to `lldpd`. No
interface is a ValueError before anything changes. It is the action for what LLDP
getters used to do on their own (NAPALM/napalm-openwrt#8).
- `SnmpValueSpec.unknown`: a driver whose device's agent answers a value wrongly says
so in its `SNMP_HEALTH_PROFILE` (`"cpu": {"unknown": True}`), and the application
reports no value for it -- not the standard reading, not 0. OpenBSD's net-snmp,
which reports the CPU as never idle, is the case (NAPALM/napalm-bsd#7).
- `VMProvisionIncomplete` (`napalm_device_types.provisioning`): what
`create_vm_from_cloud_init` raises when it fails after the VM was created. A
`RuntimeError` with the failure as its cause and the half-built VM's `vmid` and
`node`, so the caller can remove it with `destroy_vm` (NAPALM/napalm-proxmox#28).
- `HypervisorDriver.get_clusters()` and its shapes `ClusterDict`, `ClusterNodeDict`,
`ClusterStorageDict`, `ClusterNetworkDict`: every cluster the connection can see,
with its nodes (offline ones included, `local` on the node the connection is),
their CPU and memory, and the storages and networks with the nodes that see each.
A standalone host is a cluster of one with no name; a manager reports each cluster
and standalone host it runs. Declared only (NetOrk/netork#484, #1821).
- `create_vm_from_cloud_init(node=…)`: the cluster node to create the VM on. None
leaves it to the driver; a driver that can only create on its own node raises
ValueError for another (NetOrk/netork#1822).
- `LocalAccountsMixin.add_service_account(name, public_key)`: netOrk's own account,
created through the account the driver logs in with today. It takes a key and no
password, and a rule of its own in `85-service-<name>` lets it run everything as
root without a password (`NOPASSWD: ALL`, checked with `visudo` and `sudo -l`). An
account that is there is opened again and keeps its other keys; root is refused
(NetOrk/netork#1817). With `uid=`, a new account gets that uid once no other account
has it; one that is there keeps its own. A driver supplies
`_account_create_with_uid`. An account that is there and cannot log in (`nologin`,
`false`) is another service's, such as a netOrk satellite's `netork`, and is refused
before anything changes.
- `LocalAccountsMixin` (`napalm_device_types.local_accounts`): personal login accounts
on a host. `get_local_accounts(names)` reads them back (groups, lock, sudo, key
files), `add_local_account(name, public_key, password_hash=, admin=)` creates or
reopens one, `lock_local_account(name)` takes its keys and sudo away and refuses
every login while keeping its home. Every command root runs is one fixed command,
the key files are read and written as the account itself, and the sudo rule is a
drop-in of its own, checked with `visudo` and `sudo -l`. A driver supplies the
commands that create, open and lock an account and take it out of a group, and may
replace where the keys are kept (`_account_key_files`, `_account_read_keys`,
`_account_add_key`, `_account_remove_keys`; default `~/.ssh/authorized_keys`)
(NetOrk/netork#1793).
- `OpenSshServerMixin.exempt_from_ssh_penalties`: puts the address the host sees
the managing client at (`$SSH_CONNECTION`) into `PerSourcePenaltyExemptList`,
in `/etc/ssh/sshd_config.d/00-netork-penalty-exempt.conf`, next to the host's
own exemptions. Only where sshd has per-source penalties (OpenSSH 9.8+; an older
one refuses the directive) and sshd_config includes the drop-in directory;
checked with `sshd -t` and `sshd -T`, then a new session, and taken back on any
failure. Helpers: `parse_penalty_exemptions`, `client_address`, `is_exempt`
(NetOrk/netork#1800).
- `retry_refused` and `refused_before_banner` (`napalm_device_types.ssh_host_keys`):
a connection sshd closes before its banner -- what OpenSSH 9.8+ does to a source
it has just penalised -- is tried again after 1, 3, 7 and 15 s.
`HostKeyCheck.connect` does so itself (NetOrk/netork#1800).
- `ARP_AGE_UNKNOWN` (`napalm_device_types.arp`, `-1.0`): the `age` a driver reports
in `get_arp_table` and `get_ipv6_neighbors_table` when the device does not say how
old an entry is. `0.0` means "confirmed just now" and is no stand-in for it
(NetOrk/netork#1327).
- `NetworkTargetDict.uplinks` (optional): the physical NICs a bridge leaves the
host through, one group per way out -- a bond's slaves together, a single NIC
alone (`[["eno1", "eno2"]]`). A missing key means unknown, an empty list a
bridge without uplink. NetOrk follows their LLDP neighbours to the switch port
and its untagged VLAN; one slave's neighbour speaks for its bond
(NetOrk/netork#1325).
- `HostShutdownMixin.shutdown_host()`, on `DeviceTypeDriver`: shut the device itself
down cleanly through its own means; declared, so `hasattr` says whether a driver
can. No forced variant (NetOrk/netork#1636).
- `VmStopRequestMixin.begin_stop_vm(name)`: ask for a VM's clean shutdown and return
at once, for callers that wait on a timer instead of in `stop_vm`. No role demands
it (NetOrk/netork#1636).
- `SudoHistoryMixin` (`napalm_device_types.sudo_history`): the sudo password an earlier
driver typed into an interactive shell (`echo '<password>' | sudo -S -p "" …`) and
left in the login user's shell history, for a driver to mix in (NetOrk/netork#1090).
The driver says what its lines carry (`SUDO_HISTORY_MARKER`) and which files its
partial sudo left to the login user (`SUDO_ROOT_FILES`), and supplies `run_command`.
- `get_history_sudo_lines()` counts the lines per history file (`~/.bash_history`,
`~/.zsh_history`, `~/.ash_history`, `~/.sh_history`) with `grep -c` as the login
user. No line is read back; a file it cannot read raises.
- `remove_history_sudo_lines()` deletes exactly those lines as the login user (`sed
-i`, its backup overwritten with `shred -u`), and as root gives each of
`SUDO_ROOT_FILES` back to `root:root` where it is a regular file, not a symbolic
link, owned by the login user. It reports every file's lines before and after,
how many were removed and which files are root's again. Nothing goes to stdin.
### Fixed
- `AptProxyMixin` knows that apt's https sources take the http proxy too: apt's https
method asks `Acquire::https::Proxy` and then `Acquire::http::Proxy`, for the host and
then for all. An `00proxy` with the http line alone sent download.docker.com and
every other https source to apt-cacher-ng, which answers `403 CONNECT denied` without
a `PassThroughPattern`, and the whole upgrade failed (NetOrk/netork#1811).
- The report carries `https_proxy`, `https_per_host` and `https_settings`;
`https_proxy_for(report, host)` is the proxy apt takes for one https host.
- `check_https_source()` fetches one https source's `InRelease` the way apt does;
None where no https source goes through a proxy.
- `configure_package_proxy` gains an `https` step: one https source through the new
proxy. Where the proxy refuses it, `00proxy` also says `Acquire::https::Proxy
"DIRECT";`, later https lines to the same proxy are commented out, and the source
is fetched direct. An HTTP error from the mirror counts as let through.
- `apt_fetch_command` takes https URLs; `direct` sets the host's https setting too.
- `comment_out_command(..., scheme="https")`.
## [4.22.0] - 2026-10-23
### Added
+64 -6
View File
@@ -85,20 +85,34 @@ base; a mixin listed after it loses the MRO to NAPALM's stub. For the same reaso
`hasattr` says nothing about NAPALM's getters, and the OS role's account listing is
`get_os_users`, not `get_users` -- NAPALM's `get_users` has a different shape.
### NAPALM's getters: an unknown ARP age is `-1.0`
NAPALM requires an `age` on every entry of `get_arp_table` and
`get_ipv6_neighbors_table`: seconds since the device last confirmed that neighbour.
Many devices do not report one (`ip neigh`, `/proc/net/arp`, a router's host list).
A driver that cannot read the age reports `ARP_AGE_UNKNOWN` (`-1.0`, as NAPALM's IOS
driver does), never `0.0`: `0.0` means "confirmed just now", and an application that
ranks sightings by freshness takes it at its word (NetOrk/netork#1327). An expiry
countdown (FreeBSD's `expires in 1180 seconds`) is not an age either. A driver whose
device does report the age keeps reporting it.
## Function classes: what a device *can do*
Behaviour shared across roles lives in a function class, exactly once, and a role base
is a thin bundle over them — `PackageManagementMixin`, `HealthMetricsMixin`,
`ServiceControlMixin`, `UpdateMixin`, `NatVpnMixin`, `MacAclMixin`, `FirewallRuleMixin`,
`DhcpServerMixin`, `PingSweepMixin`, `ConfigLifecycleMixin`, `InterfaceFilterMixin`.
`HostRebootMixin` (`reboot_host`) is mixed into `DeviceTypeDriver` itself, since any
device may be restartable; like the others it only declares.
`HostRebootMixin` (`reboot_host`) and `HostShutdownMixin` (`shutdown_host`) are mixed
into `DeviceTypeDriver` itself, since any device may be restartable or able to shut itself
down; like the others they only declare. `VmStopRequestMixin` (`begin_stop_vm`) asks for a
VM's clean shutdown without waiting for it — a hypervisor driver mixes it in when it can.
**Fixes are templates** (`napalm_device_types.fixes`, since 4.13). A change the
application wants made on a device runs as fixed steps the driver implements, with values
the caller supplies: `SnmpAgentMixin.configure_snmp_agent(settings)` (install, configure,
firewall, start), `EnsurePackageMixin.ensure_package(name)`,
`PackageProxyMixin.configure_package_proxy(url)` and
firewall, start), `LldpAgentMixin.configure_lldp_agent(interfaces)` (install, write the
interfaces, enable and restart; the driver writes only the interfaces),
`EnsurePackageMixin.ensure_package(name)`, `PackageProxyMixin.configure_package_proxy(url)` and
`ContainerEngineMixin.grant_container_engine_access(user)`. A driver whose device takes the
change in one call overrides the template method. A driver mixes in the fixes its device
can take; none is on a role base. They replace `run_device_action`, which is deprecated.
@@ -119,6 +133,9 @@ class MySwitchDriver(SwitchDriver):
A value is `oids` (the first that answers wins) or a `table` folded by `first`, `average`
or `max`, with an optional `pattern` for an agent that answers with text. What the profile
does not name, the application reads from the standard UCD-MIB and IF-MIB.
A value the agent gets wrong is `{"unknown": True}`: the application then reports no value
for it, rather than the standard reading or a default (OpenBSD's net-snmp reports its CPU
as never idle).
`KernelFactsMixin` (`get_kernel_facts`) is the exception that is mixed in by a driver
rather than by a role base: what a Linux kernel has built and loaded is read the same way
@@ -239,6 +256,16 @@ A driver mixes it in and supplies `run_command`; a platform that regenerates ssh
itself (an appliance with its own database) must not, or opts out with
`harden_ssh_server = None`.
`SudoHistoryMixin` (`get_history_sudo_lines`, `remove_history_sudo_lines`) deals with what
a driver left behind when it typed `echo '<password>' | sudo -S -p "" …` into an interactive
shell: that line, password and all, in the login user's shell history, and files its partial
sudo wrote as the login user. The driver names the marker its lines carry
(`SUDO_HISTORY_MARKER`) and those files (`SUDO_ROOT_FILES`). The getter counts the lines per
history file with `grep -c`, as the login user; the fix deletes them with `sed -i` and
overwrites the backup with `shred -u`, then gives the files back to root (regular files the
login user owns, never a symbolic link). The password is never read, sent or written. A
driver mixes it in and supplies `run_command`.
**Update readers raise when they cannot read.** `get_available_updates` returns an empty
list only when nothing is pending; netOrk keeps "pending since" per package, and an empty
list for "don't know" would reset it.
@@ -305,6 +332,26 @@ caller's; `ssh_host_keys.HostKeyCheck` is how every SSH driver verifies against
A caller that passes neither key gets first use on every connection, as before; a driver
that ignores both still connects. Old and new versions on either side keep working.
### TLS certificates: trust on first use
The same for a driver's HTTPS API. Devices ship self-signed certificates, so a CA check
fails on all of them and turning verification off trusts whatever answers;
`tls_certificates.CertificateCheck` pins the certificate by its fingerprint instead
(NAPALM/napalm-hpe-aruba-procurve#17). No CA is consulted.
- **In:** `optional_args["tls_fingerprints"]` = `{"443": "SHA256:AB:CD:…"}`, by port: the
SHA-256 of the DER certificate as `openssl x509 -fingerprint -sha256` prints it (prefix,
colons and case optional). With one, any other certificate is refused *before* a request
carries credentials and the connection raises `TlsCertificateMismatch` (a
`ConnectionException` naming both fingerprints). Without one, the first certificate is
accepted. A driver does not fall back to another transport on a mismatch.
- **Out:** `optional_args["tls_certificate_observer"]`, a callable, gets one dict per
connection: `{"host", "port", "seen": {"fingerprint"}, "expected", "accepted"}`.
- **requests:** `check = CertificateCheck(optional_args, host, port)`;
`check.verify_server(timeout)` before the login, `check.pin(session)` so every later
connection of the session must present that certificate, the login inside
`with check.refusing():`, then `check.observe()`.
## Design principle: generic vs. device-specific logic
When adding behavior to a device-type base class, split it along one line: **would
@@ -388,7 +435,7 @@ detection vs. presentation there.
pip install napalm-device-types
```
Requires Python ≥ 3.10 (tested up to 3.13) and NAPALM ≥ 4.0.
Requires Python ≥ 3.12 (tested up to 3.13) and NAPALM ≥ 4.0.
## Available base classes
@@ -501,6 +548,14 @@ class ProxmoxDriver(HypervisorDriver):
def create_vm_snapshot(self, name, snapshot, description="", include_memory=False): ...
def get_network_targets(self):
# return List[NetworkTargetDict]: bridges and vnets a new VM's NIC may
# attach to; a bridge may name its physical NICs in `uplinks`, one group
# per way out: [["eno1", "eno2"]] for a bond, [["eno1"]] for a single NIC,
# no internal or VLAN ports -- leave the key out when they cannot be
# determined
...
def get_vm_cpu_types(self):
# optional — return List[VMCpuTypeDict]: the CPU models a new VM may get
# on this node, each with its cpuinfo flags and whether the node can run
@@ -516,7 +571,10 @@ any other. A QEMU-based driver takes `provisioning.QEMU_GUEST_AGENTS`
(Linux, FreeBSD, OpenBSD). `provisioning.network_config()` writes the
network-config v2 every cloud-init flavour reads (FreeBSD's nuageinit reads
no other), and `provisioning.split_compression()` says how a packed image is
unpacked; both are the same for every hypervisor.
unpacked; both are the same for every hypervisor. A creation that fails once
the VM exists raises `VMProvisionIncomplete`, a `RuntimeError` that names the
half-built VM (`vmid`, `node`) for the caller to `destroy_vm`, with the failure
as its cause.
### OS / Linux
+58 -4
View File
@@ -44,9 +44,11 @@ instead of being restated on every role that happens to need it:
* :class:`~napalm_device_types.health_metrics.HealthMetricsMixin`
* :class:`~napalm_device_types.host_status.HostStatusMixin`
* :class:`~napalm_device_types.host_reboot.HostRebootMixin`
* :class:`~napalm_device_types.host_shutdown.HostShutdownMixin`
* :class:`~napalm_device_types.interface_filter.InterfaceFilterMixin`
* :class:`~napalm_device_types.kernel.KernelFactsMixin`
* :class:`~napalm_device_types.listening.ListeningSocketsMixin`
* :class:`~napalm_device_types.local_accounts.LocalAccountsMixin`
* :class:`~napalm_device_types.host_firewall.HostFirewallMixin`
* :class:`~napalm_device_types.mac_acl.MacAclMixin`
* :class:`~napalm_device_types.nat_vpn.NatVpnMixin`
@@ -54,10 +56,12 @@ instead of being restated on every role that happens to need it:
* :class:`~napalm_device_types.ping_sweep.PingSweepMixin`
* :class:`~napalm_device_types.service_clusters.ServiceClustersMixin`
* :class:`~napalm_device_types.service_packages.ServicePackagesMixin`
* :class:`~napalm_device_types.sudo_history.SudoHistoryMixin`
* :class:`~napalm_device_types.services.ServiceControlMixin`
* :class:`~napalm_device_types.systemd.SystemdServicesMixin`
* :class:`~napalm_device_types.updates.UpdateMixin`
* :class:`~napalm_device_types.virtual_ips.VirtualIpsMixin`
* :class:`~napalm_device_types.vm_stop_request.VmStopRequestMixin`
Introspection -- :func:`~napalm_device_types.roles.roles_of`,
:func:`~napalm_device_types.roles.role_keys_of` and
@@ -68,10 +72,12 @@ from napalm_device_types.access_point import AccessPointDriver
from napalm_device_types.apt_proxy import (
APT_PROXY_COMMAND,
AptProxyMixin,
https_proxy_for,
parse_apt_fetch,
parse_apt_proxy,
same_proxy_url,
)
from napalm_device_types.arp import ARP_AGE_UNKNOWN
from napalm_device_types.base import DeviceTypeDriver, FingerprintRule, PortSpec, rules_max_weight
from napalm_device_types.channel import (
ByteStream,
@@ -81,6 +87,15 @@ from napalm_device_types.channel import (
open_stream_on_transport,
run_on_transport,
)
from napalm_device_types.clusters import (
BackupDict,
BackupJobDict,
ClusterDict,
ClusterNetworkDict,
ClusterNodeDict,
ClusterStorageDict,
HAResourceDict,
)
from napalm_device_types.config_lifecycle import ConfigLifecycleMixin
from napalm_device_types.config_sync import ConfigSyncMixin
from napalm_device_types.container_engine import (
@@ -95,6 +110,7 @@ from napalm_device_types.firewall_rules import FirewallRuleMixin
from napalm_device_types.firmware import FirmwareOfferMixin
from napalm_device_types.fixes import (
EnsurePackageMixin,
LldpAgentMixin,
PackageProxyMixin,
SnmpAgentMixin,
run_fix_steps,
@@ -106,6 +122,7 @@ from napalm_device_types.host_firewall import (
parse_host_firewall,
)
from napalm_device_types.host_reboot import HostRebootMixin
from napalm_device_types.host_shutdown import HostShutdownMixin
from napalm_device_types.host_status import HOST_STATUS_COMMAND, HostStatusMixin, parse_host_status
from napalm_device_types.hypervisor import HypervisorDriver
from napalm_device_types.interface_filter import InterfaceFilterMixin
@@ -118,6 +135,11 @@ from napalm_device_types.listening import (
ListeningSocketsUnavailable,
parse_listening_sockets,
)
from napalm_device_types.local_accounts import (
LocalAccountChangeDict,
LocalAccountDict,
LocalAccountsMixin,
)
from napalm_device_types.mac_acl import MacAclMixin
from napalm_device_types.management_controller import ManagementControllerDriver
from napalm_device_types.media import MediaDriver
@@ -142,10 +164,6 @@ from napalm_device_types.models import (
PowerAction,
PowerStateDict,
SnmpAgentSettingsDict,
SnmpHealthProfile,
SnmpInterfacesSpec,
SnmpMemorySpec,
SnmpValueSpec,
UpdateProgressDict,
)
from napalm_device_types.nat_vpn import NatVpnMixin
@@ -160,6 +178,7 @@ from napalm_device_types.package_updates import (
from napalm_device_types.packages import PackageManagementMixin
from napalm_device_types.phone import PhoneDriver
from napalm_device_types.ping_sweep import PingSweepMixin, driver_supports_ping
from napalm_device_types.provisioning import VMProvisionIncomplete
from napalm_device_types.residential_gateway import ResidentialGatewayDriver
from napalm_device_types.roles import primary_role_of, role_keys_of, roles_of
from napalm_device_types.service_clusters import (
@@ -174,6 +193,12 @@ from napalm_device_types.service_packages import (
parse_service_packages,
)
from napalm_device_types.services import ServiceControlMixin
from napalm_device_types.snmp_health_profile import (
SnmpHealthProfile,
SnmpInterfacesSpec,
SnmpMemorySpec,
SnmpValueSpec,
)
from napalm_device_types.ssh_host_keys import (
KNOWN_KEYS_ARG as SSH_HOST_KEYS_ARG,
)
@@ -194,6 +219,12 @@ from napalm_device_types.ssh_server import (
plan_ssh_hardening,
)
from napalm_device_types.storage import StorageDriver
from napalm_device_types.sudo_history import (
HistoryCleanupDict,
HistorySudoLinesDict,
SudoHistoryCleanupDict,
SudoHistoryMixin,
)
from napalm_device_types.switch import SwitchDriver
from napalm_device_types.system_identity import (
SYSTEM_IDENTITY_COMMAND,
@@ -214,8 +245,18 @@ from napalm_device_types.virtual_ips import (
VirtualIpsMixin,
parse_virtual_ips,
)
from napalm_device_types.vm_migration import MigrationPreflightDict
from napalm_device_types.vm_stop_request import VmStopRequestMixin
__all__ = [
"BackupDict",
"BackupJobDict",
"ClusterDict",
"ClusterNetworkDict",
"ClusterNodeDict",
"ClusterStorageDict",
"HAResourceDict",
"MigrationPreflightDict",
"HypervisorHostDict",
"DnsForwardDict",
"DnsForwardMixin",
@@ -231,9 +272,11 @@ __all__ = [
"FixResultDict",
"FixStepDict",
"FixStepOutcomeDict",
"LldpAgentMixin",
"PackageProxyMixin",
"APT_PROXY_COMMAND",
"AptProxyMixin",
"https_proxy_for",
"parse_apt_fetch",
"parse_apt_proxy",
"same_proxy_url",
@@ -269,10 +312,12 @@ __all__ = [
"HOST_STATUS_COMMAND",
"HostStatusMixin",
"HostRebootMixin",
"HostShutdownMixin",
"HypervisorDriver",
"InterfaceFilterMixin",
"MacAclMixin",
"ManagementControllerDriver",
"ARP_AGE_UNKNOWN",
"POWER_ACTIONS",
"PowerAction",
"PowerStateDict",
@@ -303,6 +348,9 @@ __all__ = [
"ListeningSocketsMixin",
"ListeningSocketsUnavailable",
"parse_listening_sockets",
"LocalAccountsMixin",
"LocalAccountDict",
"LocalAccountChangeDict",
"PhoneDriver",
"PingSweepMixin",
"PortSpec",
@@ -317,6 +365,8 @@ __all__ = [
"UpdateMixin",
"VIRTUAL_IPS_COMMAND",
"VirtualIpsMixin",
"VmStopRequestMixin",
"VMProvisionIncomplete",
"SERVICE_CLUSTERS_COMMAND",
"ServiceClustersMixin",
"parse_service_clusters",
@@ -330,6 +380,10 @@ __all__ = [
"SshHardeningPlan",
"SshHardeningResult",
"plan_ssh_hardening",
"SudoHistoryMixin",
"HistorySudoLinesDict",
"HistoryCleanupDict",
"SudoHistoryCleanupDict",
"SSH_HOST_KEYS_ARG",
"SSH_HOST_KEY_OBSERVER_ARG",
"HostKeyCheck",
+4 -5
View File
@@ -8,12 +8,9 @@ name a single entry, and guessing would overwrite one the caller never meant.
from __future__ import annotations
from collections.abc import Callable, Iterable
from typing import TypeVar
T = TypeVar("T")
def index_live(entries: Iterable[T], key: Callable[[T], str]) -> tuple[dict[str, T], set[str]]:
def index_live[T](entries: Iterable[T], key: Callable[[T], str]) -> tuple[dict[str, T], set[str]]:
"""Live entries by key, and the keys more than one of them holds.
Entries without a key are left out: they are not the caller's to match.
@@ -32,7 +29,9 @@ def index_live(entries: Iterable[T], key: Callable[[T], str]) -> tuple[dict[str,
return by_key, shared
def check_desired(entries: list[T], key: Callable[[T], str], shared: set[str], what: str) -> None:
def check_desired[T](
entries: list[T], key: Callable[[T], str], shared: set[str], what: str
) -> None:
"""``ValueError`` for a desired entry with no key, a key given twice, or a
key that names more than one live entry."""
seen: set[str] = set()
+254 -47
View File
@@ -12,12 +12,18 @@ Debian installer writes its proxy into ``apt.conf``, Ubuntu's into
(NetOrk/netork#1397). ``DIRECT`` for the proxy, or for one host
(``Acquire::http::Proxy::<host>``), means no proxy there.
**https** goes through the http proxy too. apt's https method asks
``Acquire::https::Proxy`` first and ``Acquire::http::Proxy`` after it, for one
host and then for all (NetOrk/netork#1811). An http proxy alone therefore takes
every https source as well, and a cache that does not tunnel them answers 403.
**Where it is set** is every line of apt's files that sets the http proxy, in the
order apt reads them. A line of the form ``Acquire::http::Proxy "…";`` can be
commented out; a setting inside a block (``Acquire::http { Proxy "…"; };``) is
named but left alone. apt skips a file in ``apt.conf.d`` whose name has other
characters than letters, digits, ``_``, ``-`` and ``.``, or an extension other
than ``.conf`` (``00proxy.bak``, ``*.dpkg-old``); so is it here.
order apt reads them; the https proxy's lines are named apart. A line of the form
``Acquire::http::Proxy "…";`` can be commented out; a setting inside a block
(``Acquire::http { Proxy "…"; };``) is named but left alone. apt skips a file in
``apt.conf.d`` whose name has other characters than letters, digits, ``_``, ``-``
and ``.``, or an extension other than ``.conf`` (``00proxy.bak``, ``*.dpkg-old``);
so is it here.
**Whether a proxy answers** is a fetch with apt's own downloader,
``apt-helper download-file``, which is on every apt host and needs no root:
@@ -29,13 +35,15 @@ page is the right one, the caller says (``expect``).
out every other line that sets a different one, reads the result back from
``apt-config``, and fetches the ``InRelease`` of one http source through it. The
last step is what proves it: apt reached a mirror through the proxy it is now set
to. The source chosen is one apt does not fetch ``DIRECT``.
to. The source chosen is one apt does not fetch ``DIRECT``. Then it fetches one
https source the same way; where the proxy refuses it, https goes ``DIRECT``.
"""
from __future__ import annotations
import re
import shlex
from collections.abc import Callable
from typing import TYPE_CHECKING
from urllib.parse import urlsplit
@@ -47,7 +55,7 @@ from napalm_device_types.terminal import strip_terminal_codes
class AptProxySettingDict(TypedDict):
"""One line of apt's configuration that sets the http proxy (``AptProxyMixin``)."""
"""One line of apt's configuration that sets a proxy (``AptProxyMixin``)."""
file: str
line: int # 1-based
@@ -68,6 +76,14 @@ class AptProxyReportDict(TypedDict):
auto_detect: str | None
#: Every line that sets it, in the order apt reads them; the last one wins.
settings: list[AptProxySettingDict]
#: The proxy apt takes for https: ``Acquire::https::Proxy``, else the http one;
#: None for none or ``DIRECT``.
https_proxy: str | None
#: The per-host proxies https takes: ``Acquire::https::Proxy::<host>``, else
#: ``Acquire::http::Proxy::<host>``.
https_per_host: dict[str, str]
#: Every line that sets ``Acquire::https::Proxy``, in the order apt reads them.
https_settings: list[AptProxySettingDict]
#: The base URIs of the host's sources over http and over https.
http_sources: list[str]
https_sources: list[str]
@@ -85,6 +101,15 @@ class ProxyFetchDict(TypedDict):
error: str
class HttpsSourceFetchDict(ProxyFetchDict):
"""One https source fetched the way apt fetches it (``check_https_source``)."""
#: What was fetched: the source's ``InRelease``.
url: str
#: The proxy apt took for it.
proxy: str
_BEGIN = "APROXY_BEGIN"
_END = "APROXY_END"
_APT_HELPER = "/usr/lib/apt/apt-helper"
@@ -98,7 +123,7 @@ PROXY_URL = re.compile(r"^https?://[A-Za-z0-9.\-\[\]:@%_~/]+$")
#: them early.
APT_PROXY_COMMAND = (
"( export LC_ALL=C; printf '%s%s\\n' APROXY_ BEGIN; "
"echo '[dump]'; apt-config dump 2>/dev/null | grep -i '^Acquire::http::Proxy'; "
"echo '[dump]'; apt-config dump 2>/dev/null | grep -i -E '^Acquire::https?::Proxy'; "
'eval "$(apt-config shell ap_parts Dir::Etc::parts/d ap_main Dir::Etc::main/f 2>/dev/null)"; '
"echo '[parts]'; for f in \"${ap_parts:-/etc/apt/apt.conf.d/}\"*; do "
'[ -f "$f" ] && grep -Hin proxy "$f"; done; '
@@ -108,12 +133,15 @@ APT_PROXY_COMMAND = (
"printf '%s%s\\n' APROXY_ END ) 2>/dev/null | cat"
)
_DUMP = re.compile(r'^Acquire::http::Proxy(::\S+|-Auto-Detect)?\s+"([^"]*)"', re.I)
_DUMP = re.compile(r'^Acquire::(https?)::Proxy(::\S+|-Auto-Detect)?\s+"([^"]*)"', re.I)
_GREP = re.compile(r"^(/[^:]+):(\d+):(.*)$")
_FLAT = re.compile(r'^\s*Acquire::http::Proxy\s+"([^"]*)"\s*;', re.I)
_NOT_HTTP_PROXY = re.compile(
r"^\s*(//|#)|Acquire::(https|ftp|socks)\b|::Proxy::|Proxy-Auto-Detect", re.I
)
_FLAT_HTTPS = re.compile(r'^\s*Acquire::https::Proxy\s+"([^"]*)"\s*;', re.I)
#: A line that sets the https proxy inside a block, all on the line itself.
_HTTPS_BLOCK = re.compile(r'^\s*Acquire::https\s*\{[^}]*\bProxy\s+"([^"]*)"', re.I)
_IN_BLOCK = re.compile(r'\bProxy\s+"([^"]*)"', re.I)
#: The names apt reads in ``apt.conf.d``: no extension, or ``.conf``.
_PART_NAME = re.compile(r"^(?:[A-Za-z0-9_\-]+|[A-Za-z0-9_.\-]+\.conf)$")
@@ -138,13 +166,22 @@ def _sections(output: str) -> dict[str, list[str]]:
return sections
def _setting(line: str, main: bool) -> AptProxySettingDict | None:
def _grep_line(line: str, main: bool) -> tuple[str, int, str] | None:
"""File, line number and text of one line grep printed; None for a file apt skips."""
found = _GREP.match(line)
if not found:
return None
path, number, text = found.groups()
if not main and not _PART_NAME.match(path.rsplit("/", 1)[-1]):
return None
return path, int(number), text
def _setting(line: str, main: bool) -> AptProxySettingDict | None:
found = _grep_line(line, main)
if not found:
return None
path, number, text = found
flat = _FLAT.match(text)
if flat:
return {"file": path, "line": int(number), "value": flat.group(1), "editable": True}
@@ -156,22 +193,81 @@ def _setting(line: str, main: bool) -> AptProxySettingDict | None:
return {"file": path, "line": int(number), "value": nested.group(1), "editable": False}
def _dump(lines: list[str]) -> tuple[str | None, dict[str, str], str | None]:
"""The proxy, the per-host proxies and the auto-detect program in ``apt-config dump``."""
proxy: str | None = None
per_host: dict[str, str] = {}
auto_detect: str | None = None
def _https_setting(line: str, main: bool) -> AptProxySettingDict | None:
found = _grep_line(line, main)
if not found:
return None
path, number, text = found
flat = _FLAT_HTTPS.match(text)
if flat:
return {"file": path, "line": number, "value": flat.group(1), "editable": True}
block = _HTTPS_BLOCK.match(text)
if block:
return {"file": path, "line": number, "value": block.group(1), "editable": False}
return None
class _Dump(TypedDict):
proxy: str | None
per_host: dict[str, str]
auto_detect: str | None
#: ``Acquire::https::Proxy`` as written; None when unset or empty.
https: str | None
https_per_host: dict[str, str]
def _dump(lines: list[str]) -> _Dump:
"""The proxies, the per-host proxies and the auto-detect program in ``apt-config dump``."""
dump: _Dump = {
"proxy": None,
"per_host": {},
"auto_detect": None,
"https": None,
"https_per_host": {},
}
for found in (_DUMP.match(line.strip()) for line in lines):
if not found:
continue
suffix, value = found.groups()
if not suffix:
proxy = None if value.lower() in _NO_PROXY else value
elif suffix.lower() == "-auto-detect":
auto_detect = value or None
else:
per_host[suffix[2:]] = value
return proxy, per_host, auto_detect
if found:
_take(dump, *found.groups())
return dump
def _take(dump: _Dump, scheme: str, suffix: str | None, value: str) -> None:
"""One line of the dump into *dump*; a later one of the same key wins, as in apt."""
https = scheme.lower() == "https"
if not suffix:
_take_for_all(dump, https, value)
elif suffix.lower() == "-auto-detect":
if not https:
dump["auto_detect"] = value or None
elif value:
dump["https_per_host" if https else "per_host"][suffix[2:]] = value
def _take_for_all(dump: _Dump, https: bool, value: str) -> None:
"""The proxy for every host: https keeps an empty value as unset, apt's fallback."""
if https:
dump["https"] = value or None
else:
dump["proxy"] = None if value.lower() in _NO_PROXY else value
def _https_proxy(dump: _Dump) -> str | None:
"""apt's https method takes its own setting and falls back to the http one."""
if dump["https"] is None:
return dump["proxy"]
return None if dump["https"].lower() in _NO_PROXY else dump["https"]
def https_proxy_for(report: AptProxyReportDict, host: str) -> str | None:
"""The proxy apt takes for an https source on *host*; None for ``DIRECT`` or none.
The host's own setting first, https before http, then the one for all hosts.
"""
for per_host in (report["https_per_host"], report["per_host"]):
value = next((v for h, v in per_host.items() if h.lower() == host.lower()), None)
if value:
return None if value.lower() in _NO_PROXY else value
return report["https_proxy"]
def parse_apt_proxy(output: str) -> AptProxyReportDict:
@@ -180,24 +276,36 @@ def parse_apt_proxy(output: str) -> AptProxyReportDict:
:raises ValueError: if the output carried no intact report.
"""
sections = _sections(output)
proxy, per_host, auto_detect = _dump(sections.get("dump", []))
settings = [
s
dump = _dump(sections.get("dump", []))
lines = [
(main, line)
for main, key in ((False, "parts"), (True, "main"))
for s in (_setting(line, main) for line in sections.get(key, []))
if s
for line in sections.get(key, [])
]
sources = [line.strip() for line in sections.get("sources", [])]
return {
"proxy": proxy,
"per_host": per_host,
"auto_detect": auto_detect,
"settings": settings,
"http_sources": [s for s in sources if s.startswith("http://")],
"https_sources": [s for s in sources if s.startswith("https://")],
"proxy": dump["proxy"],
"per_host": dump["per_host"],
"auto_detect": dump["auto_detect"],
"settings": _settings(lines, _setting),
"https_proxy": _https_proxy(dump),
"https_per_host": {**dump["per_host"], **dump["https_per_host"]},
"https_settings": _settings(lines, _https_setting),
"http_sources": _of_scheme(sources, "http"),
"https_sources": _of_scheme(sources, "https"),
}
def _settings(
lines: list[tuple[bool, str]], parse: Callable[[str, bool], AptProxySettingDict | None]
) -> list[AptProxySettingDict]:
return [s for s in (parse(line, main) for main, line in lines) if s]
def _of_scheme(sources: list[str], scheme: str) -> list[str]:
return [s for s in sources if s.startswith(f"{scheme}://")]
_DEFAULT_PORT = {"http": 80, "https": 443}
@@ -225,14 +333,17 @@ def apt_fetch_command(url: str, *, direct: bool = False, expect: str = "") -> st
*direct* goes past any proxy apt is set to, to *url*'s host itself; without it
the fetch takes apt's own way. *expect* is text the page should contain.
:raises ValueError: if *url* is not an http URL a shell word can carry.
:raises ValueError: if *url* is not an http(s) URL a shell word can carry.
"""
if not PROXY_URL.match(url) or not url.startswith("http://"):
if not PROXY_URL.match(url):
raise ValueError(f"not an http URL: {url!r}")
options = ["-o Acquire::Retries=0", "-o Acquire::http::Timeout=10"]
host = urlsplit(url).hostname or ""
if direct and host and ":" not in host:
# *host* comes from a URL PROXY_URL let through: no quote in it.
# *host* comes from a URL PROXY_URL let through: no quote in it. For https
# the host's https setting comes first, so both are set.
if url.startswith("https://"):
options.append(f"-o 'Acquire::https::Proxy::{host}=DIRECT'")
options.append(f"-o 'Acquire::http::Proxy::{host}=DIRECT'")
match = f"echo '[match]'; grep -c -F -- {shlex.quote(expect)} \"$t\"; " if expect else ""
return (
@@ -272,13 +383,16 @@ def parse_apt_fetch(output: str) -> ProxyFetchDict:
return {"reachable": answered, "expected": False if answered else None, "error": error}
#: sed's edit of one line: comment it out, if it still sets the http proxy.
_COMMENT_OUT = "-e '{line}{{/^[[:space:]]*Acquire::http::Proxy[[:space:]]/Is|^|// |}}'"
#: sed's edit of one line: comment it out, if it still sets the proxy of *scheme*.
_COMMENT_OUT = "-e '{line}{{/^[[:space:]]*Acquire::{scheme}::Proxy[[:space:]]/Is|^|// |}}'"
_SCHEMES = ("http", "https")
def comment_out_command(path: str, lines: list[int]) -> str:
"""The root command that comments out *lines* of *path* where they set the http proxy."""
edits = " ".join(_COMMENT_OUT.format(line=int(n)) for n in lines)
def comment_out_command(path: str, lines: list[int], scheme: str = "http") -> str:
"""The root command that comments out *lines* of *path* where they set *scheme*'s proxy."""
if scheme not in _SCHEMES:
raise ValueError(f"not a scheme apt proxies: {scheme!r}")
edits = " ".join(_COMMENT_OUT.format(line=int(n), scheme=scheme) for n in lines)
return f"sed -i {edits} {shlex.quote(path)}"
@@ -333,6 +447,8 @@ class AptProxyMixin(PackageProxyMixin):
* per_host ({host: proxy or "DIRECT"})
* auto_detect (string or None): ``Acquire::http::Proxy-Auto-Detect``
* settings (list of {file, line, value, editable}), in apt's reading order
* https_proxy (string or None): the proxy https takes, the http one if unset
* https_per_host ({host: proxy or "DIRECT"}), https_settings (as settings)
* http_sources, https_sources (lists of the sources' base URIs)
:raises ValueError: if the host's output carried no intact report.
@@ -360,8 +476,30 @@ class AptProxyMixin(PackageProxyMixin):
self._run_apt_command(apt_fetch_command(url, direct=True, expect=expect))
)
def check_https_source(self) -> HttpsSourceFetchDict | None:
"""Fetch one https source's ``InRelease`` the way apt does, through its proxy.
None when no https source goes through a proxy: there are none, or https
goes ``DIRECT``.
:raises ValueError: if the host's output carried no intact report.
:raises NotImplementedError: on a host without apt.
"""
if not self._is_apt_host():
raise NotImplementedError("not an apt host")
report = self._report()
found = _https_source(report, lambda proxy: proxy is not None)
if found is None:
return None
target, proxy = found
fetched = parse_apt_fetch(self._run_apt_command(apt_fetch_command(target)))
return {**fetched, "url": target, "proxy": proxy}
def configure_package_proxy(self, url: str) -> FixResultDict:
"""apt fetches through *url*, and is shown to; ``""`` removes :attr:`APT_PROXY_FILE`."""
"""apt fetches through *url*, and is shown to; ``""`` removes :attr:`APT_PROXY_FILE`.
https goes through *url* too, unless *url* refuses it: then ``DIRECT``.
"""
if not url:
return run_fix_steps([("configure", self._remove_package_proxy)])
return run_fix_steps(
@@ -370,6 +508,7 @@ class AptProxyMixin(PackageProxyMixin):
("conflicts", lambda: self._comment_out_other_proxies(url)),
("verify", lambda: self._verify_package_proxy(url)),
("fetch", lambda: self._fetch_through_package_proxy(url)),
("https", lambda: self._https_through_package_proxy(url)),
]
)
@@ -383,12 +522,15 @@ class AptProxyMixin(PackageProxyMixin):
f"rm -f {shlex.quote(self.APT_PROXY_FILE)}", "proxy removed"
)
def _write_package_proxy(self, url: str) -> FixStepOutcomeDict:
def _write_package_proxy(self, url: str, https_direct: bool = False) -> FixStepOutcomeDict:
if not self._is_apt_host():
return self._not_apt()
if not PROXY_URL.match(url):
raise ValueError(f"not a proxy URL: {url!r}")
return self._write_apt_file(self.APT_PROXY_FILE, f'Acquire::http::Proxy "{url}";\n')
content = f'Acquire::http::Proxy "{url}";\n'
if https_direct:
content += 'Acquire::https::Proxy "DIRECT";\n'
return self._write_apt_file(self.APT_PROXY_FILE, content)
def _report(self) -> AptProxyReportDict:
# The steps after "configure" run only once it passed, on an apt host.
@@ -442,16 +584,81 @@ class AptProxyMixin(PackageProxyMixin):
return {"ok": True, "output": f"fetched {target} through {url}"}
return {"ok": False, "output": f"{target}: {fetched['error']}"}
def _https_through_package_proxy(self, url: str) -> FixStepOutcomeDict:
"""An https source through *url*; where *url* refuses it, https goes ``DIRECT``."""
found = _https_source(self._report(), lambda proxy: same_proxy_url(proxy, url))
if found is None:
return {"ok": True, "output": f"no https source goes through {url}; not tried"}
target = found[0]
fetched = parse_apt_fetch(self._run_apt_command(apt_fetch_command(target)))
# An HTTP error came from the mirror, through the tunnel: the cache let it by.
if fetched["reachable"]:
return {"ok": True, "output": f"fetched {target} through {url}"}
refused = f"{url} refused https ({fetched['error']})"
outcome = self._https_direct(url)
if not outcome["ok"]:
return {"ok": False, "output": f"{refused}; {outcome['output']}"}
direct = parse_apt_fetch(self._run_apt_command(apt_fetch_command(target)))
if not direct["reachable"]:
return {"ok": False, "output": f"{refused}; direct too: {target}: {direct['error']}"}
named = f"; {outcome['output']}" if outcome["output"] else ""
return {
"ok": True,
"output": f"{refused}{named}; https sources go direct now: fetched {target}",
}
def _https_direct(self, url: str) -> FixStepOutcomeDict:
"""Write https ``DIRECT`` beside *url*, and comment out the later lines to *url*."""
written = self._write_package_proxy(url, https_direct=True)
if not written["ok"]:
return written
later = self._later_https_lines(self._report(), url)
for path, numbers in _editable_lines(later).items():
outcome = self._run_apt_root_command(
comment_out_command(path, numbers, scheme="https"), f"commented out in {path}"
)
if not outcome["ok"]:
return outcome
report = self._report()
if same_proxy_url(report["https_proxy"], url):
still = [f"{s['file']}:{s['line']}" for s in self._later_https_lines(report, url)]
where = f", set in {', '.join(still)}" if still else ""
return {"ok": False, "output": f"https still goes through {url}{where}"}
return {"ok": True, "output": "; ".join(map(_described, later))}
def _later_https_lines(self, report: AptProxyReportDict, url: str) -> list[AptProxySettingDict]:
"""The lines outside :attr:`APT_PROXY_FILE` that send https to *url*."""
return [
s
for s in report["https_settings"]
if s["file"] != self.APT_PROXY_FILE and same_proxy_url(s["value"], url)
]
def _https_source(
report: AptProxyReportDict, through: Callable[[str | None], bool]
) -> tuple[str, str] | None:
"""The ``InRelease`` of the first https source whose proxy *through* accepts, and that proxy."""
for source in report["https_sources"]:
if "/dists/" not in source:
continue
proxy = https_proxy_for(report, urlsplit(source).hostname or "")
if through(proxy):
return source.rstrip("/") + "/InRelease", proxy or ""
return None
__all__ = [
"APT_PROXY_COMMAND",
"AptProxyMixin",
"AptProxyReportDict",
"AptProxySettingDict",
"HttpsSourceFetchDict",
"ProxyFetchDict",
"PROXY_URL",
"apt_fetch_command",
"comment_out_command",
"https_proxy_for",
"parse_apt_fetch",
"parse_apt_proxy",
"same_proxy_url",
+13
View File
@@ -0,0 +1,13 @@
"""The ``age`` of an entry in NAPALM's neighbour tables when the device does not say.
NAPALM requires an ``age`` on every entry of ``get_arp_table`` and
``get_ipv6_neighbors_table``: seconds since the device last confirmed that
neighbour. Many devices do not report one. ``0.0`` means "confirmed just now"
and must never stand in for "not reported": a caller ranking sightings by
freshness takes it at its word (NetOrk/netork#1327). An expiry countdown is not
an age either. The value is NAPALM's own; its IOS driver reports ``-1.0``.
"""
from typing import Final
ARP_AGE_UNKNOWN: Final[float] = -1.0
+8 -1
View File
@@ -15,6 +15,7 @@ from napalm.base import NetworkDriver
from napalm_device_types.channel import CommandChannelMixin
from napalm_device_types.host_reboot import HostRebootMixin
from napalm_device_types.host_shutdown import HostShutdownMixin
from napalm_device_types.ping_sweep import PingSweepMixin
@@ -74,7 +75,13 @@ class PortSpec(NamedTuple):
mandatory: bool = False
class DeviceTypeDriver(PingSweepMixin, HostRebootMixin, CommandChannelMixin, NetworkDriver): # type: ignore[misc]
class DeviceTypeDriver(
PingSweepMixin,
HostRebootMixin,
HostShutdownMixin,
CommandChannelMixin,
NetworkDriver, # type: ignore[misc]
):
"""Common base for all netOrk device-type drivers.
Sits between napalm.base.NetworkDriver and the type-specific abstract
+99
View File
@@ -0,0 +1,99 @@
"""The shapes of a hypervisor's clusters (``HypervisorDriver.get_clusters()``).
Apart from ``models.py``, which is over the file size limit.
"""
from typing import NotRequired, TypedDict
class ClusterNodeDict(TypedDict):
"""One host of a cluster, as its cluster sees it. A value the platform does not
report is None; it is never guessed."""
name: str # the node's name in its cluster, as get_vms() reports it in ``node``
status: str # "online", "offline" or "unknown"
local: bool # the node this connection talks to; False on a manager (vCenter)
maintenance: bool | None # in maintenance mode
cpu_threads: int | None # logical CPUs
cpu_cores: int | None # physical cores
cpu_model: str | None
cpu_flags: list[str] | None # cpuinfo flags; None where the platform does not say
cpu_usage: float | None # 0.0–1.0
memory_total: int | None # bytes
memory_used: int | None # bytes
votes: NotRequired[int | None] # quorum votes of this node; None standalone/unreadable
class ClusterStorageDict(TypedDict):
"""A storage of a cluster and the nodes that see it."""
name: str # usable as create_vm_from_cloud_init(storage=...) on each of ``nodes``
type: str # backend type: "dir", "lvmthin", "zfspool", "nfs", "rbd", "vmfs", ...
shared: bool # the same storage on every node in ``nodes``, so a VM on it can move
content: list[str] # what it holds: "images", "iso", "snippets", ...
nodes: list[str] # the nodes it is available on
total: int | None # bytes; None where the platform does not report it
used: int | None # bytes
class ClusterNetworkDict(TypedDict):
"""A network a VM's NIC may attach to, and the nodes that carry it."""
name: str # usable as NICConfigDict.bridge on each of ``nodes``
kind: str # "bridge", "vnet" or "portgroup", as in NetworkTargetDict
vlan_aware: bool # True if a NICConfigDict.vlan_tag may be set on top of it
fixed_vlan_tag: NotRequired[int | None] # vnet/portgroup: the VLAN baked into it
nodes: list[str]
class HAResourceDict(TypedDict):
"""A guest the cluster's HA manager looks after."""
vmid: int
kind: str # "qemu" or "lxc"
requested: str # state HA is asked to keep: "started", "stopped", "disabled", "ignored"
status: str | None # what the HA manager reports now: "started", "fence", "migrate", ...
node: str | None # the node the HA manager places it on now
group: NotRequired[str | None] # HA group (Proxmox VE 8)
rules: list[str] # ids of the HA rules that name it (Proxmox VE 9); [] where there are none
class BackupJobDict(TypedDict):
"""A scheduled backup job of the cluster."""
id: str
enabled: bool
schedule: str | None # as the platform writes it: "sat 02:00", "daily", "mon..fri 21:00"
storage: str | None
mode: str | None # "snapshot", "suspend", "stop"
all: bool # every guest, minus ``exclude``
vmids: list[int] # the guests it names; a pool job: the pool's members now; [] with all
exclude: list[int]
pool: str | None
node: str | None # the one node it runs on; None for every node
class BackupDict(TypedDict):
"""The backups of one guest on one storage."""
vmid: int
storage: str
time: int # unix seconds of the newest backup of this guest on this storage
size: int | None # bytes of that newest backup
count: int # backups of this guest on this storage
class ClusterDict(TypedDict):
"""A cluster the connection can see. A standalone host is a cluster of one,
with no name."""
id: str | None # stable across connections where the platform has one; None otherwise
name: str | None # the cluster's name; None for a standalone host
quorate: bool | None # None where the platform has no quorum (standalone, vSphere)
nodes: list[ClusterNodeDict]
storages: list[ClusterStorageDict]
networks: list[ClusterNetworkDict]
# None: could not be read; []: read, nothing there.
ha_resources: NotRequired[list[HAResourceDict] | None]
backup_jobs: NotRequired[list[BackupJobDict] | None]
backups: NotRequired[list[BackupDict] | None]
+62 -1
View File
@@ -22,7 +22,7 @@ They replace ``run_device_action`` (docs/STRUCTURE.md):
from __future__ import annotations
from collections.abc import Callable, Sequence
from typing import TYPE_CHECKING, Any
from typing import TYPE_CHECKING, Any, ClassVar
from napalm_device_types.models import (
FixResultDict,
@@ -133,6 +133,66 @@ class EnsurePackageMixin:
def install_package(self, name: str, version: str = "") -> None: ...
class LldpAgentMixin:
"""Run an LLDP agent on the device, on the interfaces the caller names.
The steps: install the agent, write the interfaces it runs on, enable it at boot
and restart it so it reads them. Installing and starting are generic, over
``ensure_package`` and ``manage_service`` with ``LLDP_AGENT_PACKAGE`` and
``LLDP_AGENT_SERVICE``; the driver writes the configuration. Which interfaces is
the caller's choice -- an agent on every interface also talks to the uplink of a
WAN port. Whether neighbours appear is the caller's to read with
``get_lldp_neighbors``, after the agent's first advertisement interval.
"""
#: The package that brings the agent, as ``ensure_package`` takes it.
LLDP_AGENT_PACKAGE: ClassVar[str] = "lldpd"
#: The agent's service, as ``manage_service`` takes it.
LLDP_AGENT_SERVICE: ClassVar[str] = "lldpd"
def configure_lldp_agent(self, interfaces: Sequence[str]) -> FixResultDict:
"""Install, configure for *interfaces*, enable and restart the agent.
:raises ValueError: for no interface, before anything changes.
"""
names = [str(name) for name in interfaces]
if not names:
raise ValueError("name at least one interface for the LLDP agent")
return run_fix_steps(
[
("install", self._install_lldp_agent),
("configure", lambda: self._write_lldp_agent_config(names)),
("start", self._start_lldp_agent),
]
)
def _install_lldp_agent(self) -> FixStepOutcomeDict:
result = self.ensure_package(self.LLDP_AGENT_PACKAGE)
return {"ok": bool(result["success"]), "output": str(result.get("output", ""))}
def _start_lldp_agent(self) -> FixStepOutcomeDict:
service = self.LLDP_AGENT_SERVICE
for action in ("enable", "restart"):
result = self.manage_service(service, action)
if not result.get("success"):
output = str(result.get("output", "")).strip()
return {"ok": False, "output": f"{action} {service}: {output}".rstrip(": ")}
return {"ok": True, "output": f"{service} enabled and restarted"}
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
def ensure_package(self, name: str) -> FixResultDict: ...
def manage_service(self, name: str, action: str) -> dict[str, Any]: ...
# The step a driver implements:
def _write_lldp_agent_config(self, interfaces: list[str]) -> FixStepOutcomeDict:
"""Have the agent send and listen on exactly *interfaces* (names as the
device's interface getters report them), replacing the list it had."""
...
class PackageProxyMixin:
"""Point the device's package manager at a caching proxy."""
@@ -152,6 +212,7 @@ class PackageProxyMixin:
__all__ = [
"EnsurePackageMixin",
"FixStep",
"LldpAgentMixin",
"PackageProxyMixin",
"SnmpAgentMixin",
"run_fix_steps",
+2 -2
View File
@@ -2,8 +2,8 @@
**Deprecated** (docs/STRUCTURE.md): drivers do not collect over SNMP. The
application polls; a driver whose device publishes values under its own OIDs
declares them as ``SNMP_HEALTH_PROFILE`` (:class:`~napalm_device_types.models.SnmpHealthProfile`).
This mixin goes in 5.0.
declares them as ``SNMP_HEALTH_PROFILE``
(:class:`~napalm_device_types.snmp_health_profile.SnmpHealthProfile`). This mixin goes in 5.0.
The collection itself is generic: CPU, memory, uptime, load and per-interface
counters come from the same standard OIDs on any UCD-MIB/IF-MIB capable device.
+32
View File
@@ -0,0 +1,32 @@
"""Shutting the device itself down cleanly.
Declared under ``if TYPE_CHECKING``: a contract, not a placeholder. Only a
driver that can shut its device down through the device's own means defines
``shutdown_host``, so ``hasattr(driver, "shutdown_host")`` tells a caller
whether to offer it. There is no forced variant here, by design: cutting the
power is the BMC's business, not a driver's.
"""
from __future__ import annotations
from typing import TYPE_CHECKING
class HostShutdownMixin:
if TYPE_CHECKING:
def shutdown_host(self) -> None:
"""
Shuts down the device this driver is connected to, cleanly.
Returns once the device has accepted the request; the session is
usually gone right after. Whether the device really went off is
the caller's business, e.g. through its BMC.
A driver that manages other machines shuts down its *own* host,
never one of them: a hypervisor node shuts down itself, and its
guests are stopped the way the node stops them on shutdown.
:raises RuntimeError: If the device refuses the request.
"""
...
+76 -14
View File
@@ -13,6 +13,7 @@ Usage::
from typing import TYPE_CHECKING, Any
from napalm_device_types.base import DeviceTypeDriver
from napalm_device_types.clusters import ClusterDict
from napalm_device_types.health_metrics import HealthMetricsMixin
from napalm_device_types.models import (
HypervisorHostDict,
@@ -30,6 +31,7 @@ from napalm_device_types.models import (
)
from napalm_device_types.packages import PackageManagementMixin
from napalm_device_types.provisioning import QEMU_GUEST_AGENTS, GuestAgents
from napalm_device_types.vm_migration import MigrationPreflightDict
class HypervisorDriver(PackageManagementMixin, HealthMetricsMixin, DeviceTypeDriver):
@@ -464,21 +466,18 @@ class HypervisorDriver(PackageManagementMixin, HealthMetricsMixin, DeviceTypeDri
storage: str | None = None,
cpu_type: str | None = None,
guest_os: str = "linux",
node: str | None = None,
download_timeout: int = 300,
timeout: int = 180,
) -> VMProvisionResultDict:
"""
Create a new virtual machine from a cloud image via Cloud-Init.
Downloads the cloud image (qcow2/raw) directly on the hypervisor if not
already cached there, creates a new VM shell, imports the image as its
root disk, configures virtual network interfaces, and injects Cloud-Init
configuration via a storage snippet or similar mechanism. The resulting
VM is left in a running state.
Implementations should cache downloaded images by URL/filename on the
hypervisor so repeated provisioning from the same image does not
re-download it every time.
Downloads the cloud image (qcow2/raw) directly on the hypervisor unless it is
cached there already (implementations cache by URL/filename), creates a new VM
shell, imports the image as its root disk, configures virtual network interfaces,
and injects Cloud-Init configuration via a storage snippet or similar mechanism.
The resulting VM is left in a running state.
Args:
name (string) - new VM display name
@@ -497,10 +496,8 @@ class HypervisorDriver(PackageManagementMixin, HealthMetricsMixin, DeviceTypeDri
image_checksum (string | None) - expected checksum of the downloaded
image (e.g. "sha256:<hex>"). If given, verified after download;
mismatch raises RuntimeError. If None, no verification is performed.
ssh_public_keys (list[str] | None) - SSH public keys to inject into guest.
If None or empty, no SSH key injection is performed.
disk_resize_gb (int | None) - resize root disk to this size in GB.
If None, disk remains the downloaded image's native size. Default None.
ssh_public_keys (list[str] | None) - SSH public keys to inject (None: none).
disk_resize_gb (int | None) - resize root disk to this size in GB (None: keep).
storage (string | None) - name of the storage pool to place the root
disk on (a name returned by ``get_image_storages()``). If None,
the driver auto-detects the first enabled, node-available storage
@@ -519,6 +516,9 @@ class HypervisorDriver(PackageManagementMixin, HealthMetricsMixin, DeviceTypeDri
network-config itself (``provisioning.network_config``). A
guest_os the driver does not list raises ValueError before
anything is created. Default ``"linux"``.
node (string | None) - a node ``name`` from ``get_clusters()`` to create
the VM on; None leaves it to the driver. One that only creates on its
own node raises ValueError for another, before creating anything.
download_timeout (int) - maximum seconds to wait for the image download
(skipped entirely if already cached on the hypervisor). Default 300.
timeout (int) - maximum seconds to wait for the remaining provisioning
@@ -531,6 +531,8 @@ class HypervisorDriver(PackageManagementMixin, HealthMetricsMixin, DeviceTypeDri
Raises:
RuntimeError - if provisioning fails (download failure, checksum
mismatch, storage unavailable, invalid config, timeout, etc.)
VMProvisionIncomplete - a RuntimeError for a failure after the VM was created,
with that VM's ``vmid`` and ``node`` for ``destroy_vm`` and the failure as cause.
"""
...
@@ -613,7 +615,9 @@ class HypervisorDriver(PackageManagementMixin, HealthMetricsMixin, DeviceTypeDri
List[NetworkTargetDict] - each entry's ``vlan_aware`` flag tells the
caller whether a ``NICConfigDict.vlan_tag`` may additionally be set
for a NIC using that target (see ``NetworkTargetDict`` for the
per-kind rules).
per-kind rules). ``uplinks``, where the driver can tell, names
the physical NICs a bridge leaves the host through, grouped:
a bond's slaves in one group, a single NIC in its own.
"""
...
@@ -635,6 +639,64 @@ class HypervisorDriver(PackageManagementMixin, HealthMetricsMixin, DeviceTypeDri
"""
...
def get_clusters(self) -> list[ClusterDict]:
"""
List the clusters this connection can see, each with its nodes, its
storages and its networks.
A cluster member (a Proxmox node) reports its one cluster, the whole of
it: every node, offline ones included with ``status`` ``"offline"``, and
``local`` set on the node it is. A manager (vCenter) reports each
cluster it runs, and each standalone host it runs as a cluster of one.
A standalone host reports itself as a cluster of one, ``name`` None.
The VMs ``get_vms()`` lists name their node in ``node``; that name is
the node's ``name`` here. A storage or network lists the nodes it is
available on, so a caller can tell where a VM may go. A value the
platform does not report is None, never a guess.
Facts only: which node is a good place for a VM is the caller's to
decide.
Returns:
List[ClusterDict]
"""
...
def get_vm_migration_preflight(self, name: str) -> MigrationPreflightDict:
"""Which nodes of its cluster the guest could move to, and what blocks it
(see ``vm_migration``).
:raises ValueError: If no VM with the given name/ID exists.
"""
...
def migrate_vm(
self,
name: str,
target_node: str,
online: bool = True,
target_storage: str | None = None,
timeout: int = 3600,
) -> None:
"""Move the guest to another node of its cluster; block until the
platform's task ended. ``online`` keeps a running guest running.
:raises ValueError: Unknown VM, or a target that is not another
member of the cluster.
:raises RuntimeError: The migration failed, with the platform's cause.
"""
...
def set_node_maintenance(self, node: str, enabled: bool) -> None:
"""Switch HA maintenance mode of a cluster node; HA guests leave it
while it is on.
:raises ValueError: If ``node`` is not a member of the cluster.
:raises RuntimeError: If the platform refuses or cannot be asked.
"""
...
def get_hypervisor_hosts(self) -> list[HypervisorHostDict]:
"""The hosts this hypervisor or its manager runs, and how each stands:
connected, in maintenance, with configuration issues, under a central
+751
View File
@@ -0,0 +1,751 @@
"""Personal login accounts on a host: reading them, adding one, locking one again.
A person who is to work on a host gets an account of their own: they log in
with their public key and, where they are to administer the host, may use
sudo with their own password. When they leave, the account is locked, not
deleted: their key goes, sudo goes, the login is refused, and the home stays.
:class:`LocalAccountsMixin` holds what is the same on every POSIX host -- the
checks, reading an account back, the order of the steps, the key file and the
sudo rule. A driver supplies the commands that differ between operating
systems: creating, opening and locking an account, reading its lock, and
taking it out of a group.
**Every command root runs is one fixed command**, with no shell syntax of its
own -- ``useradd ...``, ``install -m 440 ...``, ``rm -f -- <path>`` -- so a
sudoers allowlist of commands can name each one. What goes into a file is
staged first, without privilege; root never reads stdin, where a sudo that
does not prompt would leave the password.
**Nothing root runs follows the person's files.** The key files live in a home
the person owns, where a symbolic link may point anywhere, so they are read and
written as that person (``sudo -n -u <name>``). Only the existence of a key
file is asked as root.
**The sudo rule is a file of its own** in the drop-in directory sudo includes,
``<name> ALL=(ALL) ALL``: the person's own password, as on a new VM. A group
alone is not enough, as some distributions grant ``%wheel`` and others
``%sudo``. ``visudo`` checks the rule before it is installed, and ``sudo -l``
afterwards shows whether it took effect. A rule elsewhere -- cloud-init's
``90-cloud-init-users`` -- is not edited; it is reported.
**The login user's own account and system accounts are never touched**:
locking the first would lock the caller out.
**netOrk's own account** (``add_service_account``) is made the same way,
through the account it logs in with today: a key and no password, and a rule
of its own -- ``<name> ALL=(ALL) NOPASSWD: ALL`` in ``85-service-<name>`` --
so it gains root without a password that could end up anywhere. Once netOrk
logs in with it, the person's account it used before is one like any other.
"""
from __future__ import annotations
import re
import shlex
from collections.abc import Iterable, Sequence
from typing import TYPE_CHECKING, ClassVar
from typing_extensions import TypedDict
from napalm_device_types.fixes import run_fix_steps
from napalm_device_types.models import FixResultDict, FixStepDict, FixStepOutcomeDict
if TYPE_CHECKING: # pragma: no cover - for type checkers only
from napalm_device_types.channel import CommandResult
#: A login name every POSIX host takes as it is: lowercase, a letter or an
#: underscore first, at most 32 characters. It is also safe in a root command.
ACCOUNT_NAME_RE = re.compile(r"[a-z_][a-z0-9_.-]{0,31}")
#: The crypt(3) prefixes of each password scheme a driver may take.
PASSWORD_SCHEMES: dict[str, tuple[str, ...]] = {
"sha512": ("$6$",),
"bcrypt": ("$2b$", "$2a$", "$2y$"),
}
_CRYPT_RE = re.compile(r"\$[0-9a-z]+\$[./0-9A-Za-z$]+")
#: The public key types OpenSSH takes in a key file.
_KEY_TYPE_RE = re.compile(
r"(ssh-ed25519|ssh-rsa|ecdsa-sha2-nistp(256|384|521)"
r"|sk-ssh-ed25519@openssh\.com|sk-ecdsa-sha2-nistp256@openssh\.com)"
)
_KEY_BLOB_RE = re.compile(r"[A-Za-z0-9+/]+={0,3}")
#: Before the rule's file name; after cloud-init's ``90-`` it would win, so before it.
SUDOERS_FILE_PREFIX = "80-account-"
#: Before the file name of netOrk's own account's rule; after a person's, before cloud-init's.
SERVICE_SUDOERS_FILE_PREFIX = "85-service-"
#: A line of ``sudo -l`` that lets the account run every command as root.
_RUN_ALL_RE = re.compile(r"^\s*\(ALL(\s*:\s*ALL)?\)\s+([A-Z_]+:\s*)*ALL\s*$", re.MULTILINE)
#: The same, without a password.
_RUN_ALL_NOPASSWD_RE = re.compile(
r"^\s*\(ALL(\s*:\s*ALL)?\)\s+([A-Z_]+:\s*)*NOPASSWD:\s*([A-Z_]+:\s*)*ALL\s*$",
re.MULTILINE,
)
#: Login shells that refuse every login: an account with one belongs to a service.
_NO_LOGIN_SHELLS = frozenset({"nologin", "false"})
#: Stages stdin as a file anyone may read -- the person reads the key file
#: from it -- and prints its path. The content is a public key or a sudo rule.
_STAGE = 't=$(mktemp) && chmod 644 "$t" && cat > "$t" && printf %s "$t"'
class LocalAccountDict(TypedDict):
"""A person's account as the host has it (``get_local_accounts``)."""
name: str
uid: int
home: str
shell: str
#: Primary group first.
groups: list[str]
#: No login, with a key or a password.
locked: bool
#: ``sudo -l`` lets it run every command as root.
admin: bool
#: The lines of its key files; None when they could not be read (a locked
#: account cannot run a command any more).
authorized_keys: list[str] | None
class LocalAccountChangeDict(FixResultDict):
"""The result of ``add_local_account`` and ``lock_local_account``."""
#: The account read back after the change; None when it is not there or
#: could not be read.
account: LocalAccountDict | None
def sudoers_rule(name: str) -> str:
"""The rule that lets *name* run everything as root, with its own password."""
return f"{name} ALL=(ALL) ALL\n"
def sudoers_rule_path(directory: str, name: str) -> str:
"""Where *name*'s rule lives in *directory*.
sudo skips a file whose name holds a dot, so a dot becomes ``_2e``, and an
underscore ``__``: no two names share a file.
"""
return f"{directory}/{SUDOERS_FILE_PREFIX}{_rule_file_name(name)}"
def service_sudoers_rule(name: str) -> str:
"""The rule that lets netOrk's own account *name* run everything as root, no password."""
return f"{name} ALL=(ALL) NOPASSWD: ALL\n"
def service_sudoers_rule_path(directory: str, name: str) -> str:
"""Where the rule of netOrk's own account *name* lives in *directory*."""
return f"{directory}/{SERVICE_SUDOERS_FILE_PREFIX}{_rule_file_name(name)}"
def _rule_file_name(name: str) -> str:
return name.replace("_", "__").replace(".", "_2e")
def authorized_keys_paths(home: str) -> tuple[str, str]:
"""The key files sshd reads in *home* unless configured otherwise."""
return f"{home}/.ssh/authorized_keys", f"{home}/.ssh/authorized_keys2"
def check_account_name(name: str) -> None:
""":raises ValueError: when *name* is no login name every host takes."""
if not ACCOUNT_NAME_RE.fullmatch(name):
raise ValueError(f"not an account name: {name!r}")
def _key_blob(line: str) -> str | None:
"""The base64 part of a key file line, options and comment aside."""
words = line.split()
for i, word in enumerate(words[:-1]):
if _KEY_TYPE_RE.fullmatch(word):
return words[i + 1]
return None
def check_public_key(key: str) -> None:
""":raises ValueError: when *key* is not one OpenSSH public key on one line."""
words = key.split()
if (
"\n" in key.strip()
or len(words) < 2
or not _KEY_TYPE_RE.fullmatch(words[0])
or not _KEY_BLOB_RE.fullmatch(words[1])
):
raise ValueError("not a public key: expected '<type> <base64> [comment]' on one line")
def _command(argv: Sequence[str]) -> str:
return shlex.join(argv)
def _as(name: str, *argv: str) -> str:
"""*argv* run as *name*, from root: nothing it does reaches beyond that account."""
return _command(["sudo", "-n", "-u", name, *argv])
def _said(result: CommandResult) -> str:
return (result.stderr or result.stdout).strip() or f"exit {result.exit_code}"
class LocalAccountsMixin:
"""Reads, adds and locks personal accounts; a driver supplies the OS's commands.
Mixed in by the drivers that can, so ``callable(getattr(driver,
"add_local_account", None))`` says whether a driver can.
"""
#: The login shell of a new account.
ACCOUNT_SHELL: ClassVar[str] = "/bin/sh"
#: The password scheme the host's accounts take, a key of :data:`PASSWORD_SCHEMES`.
ACCOUNT_PASSWORD_SCHEME: ClassVar[str] = "sha512"
#: Groups that make an account an administrator; a person without sudo leaves them.
ACCOUNT_ADMIN_GROUPS: ClassVar[tuple[str, ...]] = ("wheel",)
#: The lowest uid of a person's account; below are the system's.
ACCOUNT_MIN_UID: ClassVar[int] = 1000
#: The drop-in directory sudo includes.
SUDOERS_DIR: ClassVar[str] = "/etc/sudoers.d"
#: The group of root's files.
ROOT_GROUP: ClassVar[str] = "root"
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
def run_command(
self,
command: str,
*,
privileged: bool = False,
timeout: float = 60,
stdin: bytes | None = None,
) -> CommandResult: ...
# -- what a driver supplies ------------------------------------------------------
#
# ``_account_*`` are the driver's: the five below it must supply, the key
# store (``_account_key_files``, ``_account_read_keys``, ``_account_add_key``,
# ``_account_remove_keys``) it may replace. ``_accounts_*`` are this mixin's own.
def _account_create(self, name: str, password_hash: str | None) -> list[list[str]]:
"""The commands that create *name* with a home and :attr:`ACCOUNT_SHELL`.
Without *password_hash* the account takes no password, only a key.
"""
raise NotImplementedError
def _account_open(
self, name: str, password_hash: str | None, *, locked: bool
) -> list[list[str]]:
"""The commands that let *name* log in again, with *password_hash* if given."""
raise NotImplementedError
def _account_lock(self, name: str, *, locked: bool) -> list[list[str]]:
"""The commands that refuse *name* every login, a key's too; none when *locked*."""
raise NotImplementedError
def _account_locked(self, name: str, passwd: list[str]) -> bool:
"""Whether *name* is locked; *passwd* is its ``getent passwd`` line, split.
:raises RuntimeError: when that cannot be read.
"""
raise NotImplementedError
def _account_leave_group(self, name: str, group: str, groups: list[str]) -> list[str]:
"""The command that takes *name* out of *group*; *groups* are all of its own."""
raise NotImplementedError
def _account_create_with_uid(self, name: str, uid: int) -> list[list[str]]:
"""The commands that create *name* as :meth:`_account_create` does, without a
password, and with the uid *uid*.
Only netOrk's own account asks for one (``add_service_account(uid=)``).
"""
raise NotImplementedError
# -- reading ---------------------------------------------------------------------
def get_local_accounts(self, names: Iterable[str]) -> list[LocalAccountDict]:
"""The accounts of *names* the host has, in that order; a missing one is left out.
:raises ValueError: for a name that is no account name.
:raises RuntimeError: when an account cannot be read.
"""
wanted = list(names)
for name in wanted:
check_account_name(name)
found = (self._accounts_read(name) for name in wanted)
return [account for account in found if account is not None]
def _accounts_passwd(self, name: str) -> list[str] | None:
result = self.run_command(_command(["getent", "passwd", name]), timeout=30)
if result.exit_code == 2:
return None
fields = result.stdout.strip().split(":")
if result.exit_code != 0 or len(fields) < 7 or fields[0] != name:
raise RuntimeError(f"cannot read the account {name}: {_said(result)}")
return fields
def _accounts_read(self, name: str) -> LocalAccountDict | None:
passwd = self._accounts_passwd(name)
if passwd is None:
return None
groups = self.run_command(_command(["id", "-Gn", name]), timeout=30)
if groups.exit_code != 0:
raise RuntimeError(f"cannot read the groups of {name}: {_said(groups)}")
home = passwd[5]
return {
"name": name,
"uid": int(passwd[2]),
"home": home,
"shell": passwd[6],
"groups": groups.stdout.split(),
"locked": self._account_locked(name, passwd),
"admin": self._accounts_admin(name),
"authorized_keys": self._account_read_keys(name, home),
}
def _accounts_admin(self, name: str) -> bool:
result = self.run_command(_command(["sudo", "-l", "-U", name]), privileged=True, timeout=30)
if result.exit_code != 0:
raise RuntimeError(f"cannot ask sudo about {name}: {_said(result)}")
return bool(_RUN_ALL_RE.search(result.stdout))
def _account_key_files(self, name: str, home: str) -> tuple[str, ...]:
"""Every file sshd reads *name*'s keys from; a host that adds one of its own says so."""
return authorized_keys_paths(home)
def _account_read_keys(self, name: str, home: str) -> list[str] | None:
"""The lines of :meth:`_account_key_files`, read as *name*; None when it cannot."""
lines: list[str] = []
for path in self._account_key_files(name, home):
there = self.run_command(_command(["test", "-e", path]), privileged=True, timeout=30)
if there.exit_code != 0:
continue
result = self.run_command(_as(name, "cat", "--", path), privileged=True, timeout=30)
if result.exit_code != 0:
return None
lines += [ln for ln in result.stdout.splitlines() if ln.strip()]
return lines
# -- changing ----------------------------------------------------------------------
def add_local_account(
self,
name: str,
public_key: str,
*,
password_hash: str | None = None,
admin: bool = False,
) -> LocalAccountChangeDict:
"""Give *name* an account that logs in with *public_key*, or open it again.
An account that is there is opened again and keeps its other keys; a
new one gets a home. *password_hash* (crypt(3), in
:attr:`ACCOUNT_PASSWORD_SCHEME`) becomes its password; *admin* gives it
the sudo rule, which asks for that password -- so an administrator
needs one. Without *admin*, the rule and :attr:`ACCOUNT_ADMIN_GROUPS`
are taken away.
Steps: ``account``, then ``sudo`` and ``keys``, each of which runs when
the other failed.
:raises ValueError: for a name, key or hash no host takes, the login
user's own account, or a system account.
"""
check_account_name(name)
check_public_key(public_key)
if password_hash is not None:
self._accounts_check_hash(password_hash)
existing = self._accounts_changeable(name)
first = run_fix_steps(
[("account", lambda: self._accounts_open_or_create(name, existing, password_hash))]
)
parts = [first]
if first["success"]:
parts += [
run_fix_steps([("sudo", lambda: self._accounts_set_sudo(name, admin=admin))]),
run_fix_steps([("keys", lambda: self._accounts_add_key(name, public_key))]),
]
return self._accounts_result(name, parts)
def lock_local_account(self, name: str) -> LocalAccountChangeDict:
"""Lock *name*: its keys go, every login is refused, sudo goes; its home stays.
Steps ``keys``, ``lock`` and ``sudo``, each run when one before failed:
the lock matters most. A sudo rule outside the drop-in directory is
reported, not edited -- with the account locked it can no longer be used.
:raises ValueError: for a name no host takes, the login user's own
account, or a system account.
"""
check_account_name(name)
if self._accounts_changeable(name) is None:
step: FixStepDict = {"step": "account", "ok": False, "output": f"no account {name}"}
return {**_joined([step]), "account": None}
parts = [
run_fix_steps([("keys", lambda: self._accounts_remove_keys(name))]),
run_fix_steps([("lock", lambda: self._accounts_lock(name))]),
run_fix_steps(
[("sudo", lambda: self._accounts_set_sudo(name, admin=False, locked=True))]
),
]
return self._accounts_result(name, parts)
def add_service_account(
self, name: str, public_key: str, *, uid: int | None = None
) -> LocalAccountChangeDict:
"""Make *name* netOrk's own account: it logs in with *public_key*, and gains
root without a password.
Created through the account the driver logs in with now, or opened
again when it is there; its other keys stay. It takes no password. Run
again with the account the driver already logs in with, it only adds
what is missing.
A new account gets *uid* when one is given -- the same on every host --
once no other account has it; an account that is there keeps its own,
as a uid is never changed under files that carry it.
Steps as in :meth:`add_local_account`: ``account``, then ``sudo`` and
``keys``. The new login itself is for the caller to try.
An account that is there and cannot log in (``nologin``, ``false``) is
another service's -- a satellite's ``netork``, say -- and is left alone.
:raises ValueError: for a name or key no host takes, root, a uid of 0 or
less, or another service's account.
:raises RuntimeError: when the login user cannot be told.
"""
check_public_key(public_key)
login, existing = self._accounts_service_target(name, uid)
if name == login:
step: FixStepDict = {
"step": "account",
"ok": True,
"output": f"this driver logs in as {name} already",
}
first: FixResultDict = _joined([step])
else:
first = run_fix_steps(
[("account", lambda: self._accounts_service_account(name, existing, uid))]
)
parts = [first]
if first["success"]:
parts += [
run_fix_steps([("sudo", lambda: self._accounts_grant_service_sudo(name))]),
run_fix_steps([("keys", lambda: self._accounts_add_key(name, public_key))]),
]
return self._accounts_result(name, parts)
def _accounts_service_target(self, name: str, uid: int | None) -> tuple[str, list[str] | None]:
"""The login user and *name*'s passwd line, once *name* may become netOrk's.
:raises ValueError: as :meth:`add_service_account` says.
"""
check_account_name(name)
if uid is not None and uid <= 0:
raise ValueError(f"not a uid for netOrk's own account: {uid}")
login = self._accounts_login()
existing = self._accounts_passwd(name)
if existing is not None and int(existing[2]) == 0:
raise ValueError(f"{name} is root")
if existing is not None and existing[6].rsplit("/", 1)[-1] in _NO_LOGIN_SHELLS:
raise ValueError(
f"{name} is there and cannot log in ({existing[6]}): another service's account"
)
return login, existing
def _accounts_service_account(
self, name: str, passwd: list[str] | None, uid: int | None
) -> FixStepOutcomeDict:
"""netOrk's own account opened or made, a new one with *uid* when given."""
if uid is None:
return self._accounts_open_or_create(name, passwd, None)
if passwd is not None:
outcome = self._accounts_open_or_create(name, passwd, None)
if outcome["ok"] and int(passwd[2]) != uid:
outcome["output"] += f"; it keeps its uid {passwd[2]}, not {uid}"
return outcome
owner = self._accounts_passwd_of_uid(uid)
if owner is not None:
return {"ok": False, "output": f"uid {uid} is the account {owner}'s"}
try:
create = self._account_create_with_uid(name, uid)
except NotImplementedError:
return {"ok": False, "output": f"this driver cannot give an account the uid {uid}"}
failed = self._accounts_run(create)
if failed is not None:
return {"ok": False, "output": _said(failed)}
return {"ok": True, "output": f"account {name} created, uid {uid}"}
def _accounts_passwd_of_uid(self, uid: int) -> str | None:
"""The name of the account with *uid*, or None."""
result = self.run_command(_command(["getent", "passwd", str(uid)]), timeout=30)
if result.exit_code == 2:
return None
if result.exit_code != 0:
raise RuntimeError(f"cannot ask who has uid {uid}: {_said(result)}")
return result.stdout.split(":", 1)[0].strip() or None
def _accounts_check_hash(self, password_hash: str) -> None:
prefixes = PASSWORD_SCHEMES[self.ACCOUNT_PASSWORD_SCHEME]
if not _CRYPT_RE.fullmatch(password_hash) or not password_hash.startswith(prefixes):
raise ValueError(f"not a {self.ACCOUNT_PASSWORD_SCHEME} password hash")
def _accounts_changeable(self, name: str) -> list[str] | None:
"""*name*'s passwd line, None when there is none.
:raises ValueError: for the login user's own account or a system account.
:raises RuntimeError: when the login user cannot be told.
"""
if name == self._accounts_login():
raise ValueError(f"{name} is the account this driver logs in with")
passwd = self._accounts_passwd(name)
if passwd is not None and int(passwd[2]) < self.ACCOUNT_MIN_UID:
raise ValueError(f"{name} is a system account (uid {passwd[2]})")
return passwd
def _accounts_login(self) -> str:
""":raises RuntimeError: when the login user cannot be told."""
login = self.run_command("id -un", timeout=30).stdout.strip()
if not login:
raise RuntimeError("cannot tell which account this driver logs in with")
return login
def _accounts_result(self, name: str, parts: list[FixResultDict]) -> LocalAccountChangeDict:
steps = [s for part in parts for s in part["steps"]]
result = _joined(steps)
account: LocalAccountDict | None = None
try:
account = self._accounts_read(name)
except RuntimeError as exc:
result["output"] += f"\n[read back] {exc}"
return {**result, "account": account}
def _accounts_root(self, argv: Sequence[str]) -> CommandResult:
return self.run_command(_command(argv), privileged=True, timeout=60)
def _accounts_run(self, commands: list[list[str]]) -> CommandResult | None:
"""Run *commands* as root in order; the first that failed, or None."""
for argv in commands:
result = self._accounts_root(argv)
if result.exit_code != 0:
return result
return None
def _accounts_open_or_create(
self, name: str, passwd: list[str] | None, password_hash: str | None
) -> FixStepOutcomeDict:
if passwd is None:
failed = self._accounts_run(self._account_create(name, password_hash))
done = f"account {name} created"
else:
locked = self._account_locked(name, passwd)
failed = self._accounts_run(self._account_open(name, password_hash, locked=locked))
done = f"account {name} opened again" if locked else f"account {name} is there"
if failed is not None:
return {"ok": False, "output": _said(failed)}
return {"ok": True, "output": done + (", password set" if password_hash else "")}
def _accounts_lock(self, name: str) -> FixStepOutcomeDict:
passwd = self._accounts_passwd(name)
if passwd is None:
return {"ok": False, "output": f"no account {name}"}
locked = self._account_locked(name, passwd)
failed = self._accounts_run(self._account_lock(name, locked=locked))
if failed is not None:
return {"ok": False, "output": _said(failed)}
if not self._account_locked(name, passwd):
return {"ok": False, "output": f"{name} is still not locked"}
return {"ok": True, "output": f"{name} was locked already" if locked else f"{name} locked"}
# -- sudo --------------------------------------------------------------------------
def _accounts_set_sudo(
self, name: str, *, admin: bool, locked: bool = False
) -> FixStepOutcomeDict:
path = sudoers_rule_path(self.SUDOERS_DIR, name)
if admin:
return self._accounts_grant_sudo(name, path)
said: list[str] = []
removed = self._accounts_root(["rm", "-f", "--", path])
if removed.exit_code != 0:
return {"ok": False, "output": f"{path}: {_said(removed)}"}
groups = self.run_command(_command(["id", "-Gn", name]), timeout=30).stdout.split()
for group in self.ACCOUNT_ADMIN_GROUPS:
if group not in groups[1:]:
continue
left = self._accounts_root(self._account_leave_group(name, group, groups))
if left.exit_code != 0:
return {"ok": False, "output": f"cannot take {name} out of {group}: {_said(left)}"}
groups.remove(group)
said.append(f"{name} left {group}")
if self._accounts_admin(name):
rest = f"another rule than {path} still lets {name} run everything as root"
if locked:
return {"ok": True, "output": f"{rest}; the account is locked, so it cannot use it"}
return {"ok": False, "output": rest}
said.append(f"{name} has no sudo")
return {"ok": True, "output": "\n".join(said)}
def _accounts_grant_sudo(self, name: str, path: str) -> FixStepOutcomeDict:
failed = self._accounts_install_rule(sudoers_rule(name), path)
if failed is not None:
return failed
if not self._accounts_admin(name):
return {
"ok": False,
"output": f"{path} is written, but sudo does not read it: "
f"does sudoers include {self.SUDOERS_DIR}?",
}
return {"ok": True, "output": f"{name} may use sudo with its password ({path})"}
def _accounts_grant_service_sudo(self, name: str) -> FixStepOutcomeDict:
path = service_sudoers_rule_path(self.SUDOERS_DIR, name)
failed = self._accounts_install_rule(service_sudoers_rule(name), path)
if failed is not None:
return failed
result = self.run_command(_command(["sudo", "-l", "-U", name]), privileged=True, timeout=30)
if result.exit_code != 0:
return {"ok": False, "output": f"cannot ask sudo about {name}: {_said(result)}"}
if not _RUN_ALL_NOPASSWD_RE.search(result.stdout):
return {
"ok": False,
"output": f"{path} is written, but sudo does not read it: "
f"does sudoers include {self.SUDOERS_DIR}?",
}
return {"ok": True, "output": f"{name} may use sudo without a password ({path})"}
def _accounts_install_rule(self, rule: str, path: str) -> FixStepOutcomeDict | None:
"""Check *rule* with visudo and install it at *path*; the failure, or None."""
staged = self._accounts_stage(rule)
try:
checked = self._accounts_root(["visudo", "-cf", staged])
if checked.exit_code != 0:
return {"ok": False, "output": f"visudo refuses the rule: {_said(checked)}"}
installed = self._accounts_root(
["install", "-m", "440", "-o", "root", "-g", self.ROOT_GROUP, "--", staged, path]
)
if installed.exit_code != 0:
return {"ok": False, "output": f"{path}: {_said(installed)}"}
finally:
self._accounts_unstage(staged)
return None
# -- keys --------------------------------------------------------------------------
def _accounts_stage(self, content: str) -> str:
"""*content* in a fresh file of the login user's; its path.
:raises RuntimeError: when it cannot be staged.
"""
result = self.run_command(_STAGE, timeout=30, stdin=content.encode())
path = result.stdout.strip().splitlines()[-1:] or [""]
if result.exit_code != 0 or not path[0].startswith("/"):
raise RuntimeError(f"cannot stage a file: {_said(result)}")
return path[0]
def _accounts_unstage(self, path: str) -> None:
self.run_command(_command(["rm", "-f", "--", path]), timeout=30)
def _accounts_home(self, name: str) -> str | None:
passwd = self._accounts_passwd(name)
return passwd[5] if passwd is not None else None
def _accounts_add_key(self, name: str, public_key: str) -> FixStepOutcomeDict:
home = self._accounts_home(name)
if home is None:
return {"ok": False, "output": f"no account {name}"}
return self._account_add_key(name, home, public_key)
def _accounts_remove_keys(self, name: str) -> FixStepOutcomeDict:
home = self._accounts_home(name)
if home is None:
return {"ok": False, "output": f"no account {name}"}
return self._account_remove_keys(name, home)
def _account_add_key(self, name: str, home: str, public_key: str) -> FixStepOutcomeDict:
"""Add *public_key* to *name*'s first key file, as *name*; the others stay.
A driver whose host keeps keys elsewhere overrides this, with
:meth:`_account_read_keys` and :meth:`_account_remove_keys`.
"""
path = authorized_keys_paths(home)[0]
current = self._account_read_keys(name, home)
if current is None:
return {"ok": False, "output": f"cannot read the key files of {name}"}
blob = _key_blob(public_key)
if any(_key_blob(line) == blob for line in current):
return {"ok": True, "output": f"the key is already in {path}"}
mine = self._accounts_key_file(name, path)
if mine is None:
return {"ok": False, "output": f"cannot read {path}"}
staged = self._accounts_stage("".join(f"{line}\n" for line in [*mine, public_key.strip()]))
try:
failed = self._accounts_run_as(
name,
[
["mkdir", "-p", "-m", "700", "--", f"{home}/.ssh"],
["install", "-m", "600", "--", staged, path],
],
)
finally:
self._accounts_unstage(staged)
if failed is not None:
return {"ok": False, "output": f"{path}: {_said(failed)}"}
return {"ok": True, "output": f"key added to {path}"}
def _accounts_key_file(self, name: str, path: str) -> list[str] | None:
there = self.run_command(_command(["test", "-e", path]), privileged=True, timeout=30)
if there.exit_code != 0:
return []
result = self.run_command(_as(name, "cat", "--", path), privileged=True, timeout=30)
if result.exit_code != 0:
return None
return [line for line in result.stdout.splitlines() if line.strip()]
def _accounts_run_as(self, name: str, commands: list[list[str]]) -> CommandResult | None:
for argv in commands:
result = self.run_command(_as(name, *argv), privileged=True, timeout=60)
if result.exit_code != 0:
return result
return None
def _account_remove_keys(self, name: str, home: str) -> FixStepOutcomeDict:
"""Remove *name*'s key files in *home*, as *name*; then no key may be left."""
paths = authorized_keys_paths(home)
failed = self._accounts_run_as(name, [["rm", "-f", "--", *paths]])
if failed is not None:
return {"ok": False, "output": f"cannot remove the keys of {name}: {_said(failed)}"}
if self._account_read_keys(name, home):
return {"ok": False, "output": f"{name} still has keys"}
return {"ok": True, "output": f"keys of {name} removed"}
def _joined(steps: list[FixStepDict]) -> FixResultDict:
return {
"success": bool(steps) and all(s["ok"] for s in steps),
"steps": steps,
"output": "\n".join(f"[{s['step']}] {s['output']}".rstrip() for s in steps),
}
__all__ = [
"ACCOUNT_NAME_RE",
"PASSWORD_SCHEMES",
"SERVICE_SUDOERS_FILE_PREFIX",
"SUDOERS_FILE_PREFIX",
"LocalAccountChangeDict",
"LocalAccountDict",
"LocalAccountsMixin",
"authorized_keys_paths",
"check_account_name",
"check_public_key",
"service_sudoers_rule",
"service_sudoers_rule_path",
"sudoers_rule",
"sudoers_rule_path",
]
+13 -54
View File
@@ -5,9 +5,15 @@ These supplement the types defined in napalm.base.models and are used by the
abstract device-type driver classes in this package.
"""
from typing import Literal, get_args
from typing import Literal, NotRequired, get_args
from typing_extensions import NotRequired, TypedDict
from typing_extensions import TypedDict
# The SNMP health profile shapes live in snmp_health_profile.py; re-exported here, where they were.
from napalm_device_types.snmp_health_profile import SnmpHealthProfile as SnmpHealthProfile
from napalm_device_types.snmp_health_profile import SnmpInterfacesSpec as SnmpInterfacesSpec
from napalm_device_types.snmp_health_profile import SnmpMemorySpec as SnmpMemorySpec
from napalm_device_types.snmp_health_profile import SnmpValueSpec as SnmpValueSpec
# ---------------------------------------------------------------------------
# Common (shared across device types)
@@ -1230,53 +1236,6 @@ class SnmpAgentSettingsDict(TypedDict):
listen: NotRequired[str] # the address the agent binds to; absent: every address
class SnmpValueSpec(TypedDict, total=False):
"""Where an SNMP agent publishes one value, inside a ``SnmpHealthProfile``.
Either ``oids``, scalars asked in order (the first non-empty answer wins), or
``table``, a column walked and folded to one value by ``fold``. ``pattern`` is a
regular expression whose first group is the number, for an agent that answers
with text.
"""
oids: list[str]
table: str
fold: Literal["first", "average", "max"]
pattern: str
class SnmpMemorySpec(TypedDict, total=False):
"""Memory: ``total`` with ``used`` or ``free``, or ``percent`` alone."""
total: SnmpValueSpec
used: SnmpValueSpec
free: SnmpValueSpec
percent: SnmpValueSpec
unit: Literal["KiB", "bytes"] # of total, used and free; KiB, as UCD-MIB counts, by default
class SnmpInterfacesSpec(TypedDict, total=False):
"""How the IF-MIB counters of a device are to be read."""
skip: str # a pattern of interface names whose counters are noise, not signal
tx_errors_are_drops: bool # the device counts dropped packets as TX errors (OPNsense)
class SnmpHealthProfile(TypedDict, total=False):
"""Where a device's SNMP agent publishes its health values.
A driver declares it as ``SNMP_HEALTH_PROFILE`` when its device does not answer
the standard UCD-MIB and IF-MIB; the application polls and computes. What a
profile does not name is read the standard way. Uptime is in TimeTicks, CPU in
percent busy.
"""
uptime: SnmpValueSpec
cpu: SnmpValueSpec
memory: SnmpMemorySpec
interfaces: SnmpInterfacesSpec
class NICConfigDict(TypedDict):
"""Network interface configuration for VM provisioning."""
@@ -1310,17 +1269,17 @@ class NetworkTargetDict(TypedDict):
Distinguishes real bridges (Linux or OVS) from SDN network segments (vnets),
and tells the caller whether a separate ``vlan_tag`` may be applied on top:
- Linux bridge: vlan_aware reflects the bridge's own ``bridge_vlan_aware`` flag.
- OVS bridge: always vlan_aware (OVS bridges tag per-port regardless of a
dedicated "VLAN aware" setting).
- SDN vnet: never vlan_aware — the VLAN is already fixed by the vnet's zone/tag,
so a NIC attached to it must not also carry a ``vlan_tag``. That fixed VLAN
is surfaced via ``fixed_vlan_tag`` instead, for display purposes.
- OVS bridge: always vlan_aware (OVS tags per port, with no "VLAN aware" setting).
- SDN vnet: never vlan_aware — its zone/tag fixes the VLAN (``fixed_vlan_tag``).
``uplinks``: the physical NICs it leaves the host through, one group per way out
(a bond's slaves together, a single NIC alone); no internal, VLAN or virtual ports.
"""
name: str # Bridge, vnet or port group name, usable directly as NICConfigDict.bridge
kind: str # "bridge", "vnet" or "portgroup" (VMware: VLAN fixed like a vnet's)
vlan_aware: bool # True if a NICConfigDict.vlan_tag may be set on top of this target
fixed_vlan_tag: NotRequired[int | None] # vnet only: the VLAN ID already baked into it
uplinks: NotRequired[list[list[str]]] # NIC groups it leaves through; absent = unknown
class StorageTargetDict(TypedDict):
+27 -2
View File
@@ -3,8 +3,9 @@ What provisioning a VM from a cloud image needs to know about the guest.
The same for every hypervisor: which agent a guest runs so a QEMU-based
hypervisor can read its IP, the network-config cloud-init and FreeBSD's
nuageinit both read, and how a packed cloud image is unpacked. How a
hypervisor attaches, boots and talks to the VM stays in its driver.
nuageinit both read, how a packed cloud image is unpacked, and how a failure
names the VM it left behind. How a hypervisor attaches, boots and talks to
the VM stays in its driver.
"""
from __future__ import annotations
@@ -37,6 +38,30 @@ _DECOMPRESSORS = {
}
class VMProvisionIncomplete(RuntimeError):
"""``create_vm_from_cloud_init`` failed after the VM was created.
The VM is still on the hypervisor, half built: ``vmid`` and ``node`` name it
as ``VMProvisionResultDict`` would have, so the caller can remove it with
``destroy_vm``. The failure itself is the ``__cause__`` (``raise ... from``),
and the message is its own. A failure before the VM exists raises as before.
"""
def __init__(self, message: str, *, vmid: str, node: str) -> None:
super().__init__(message)
self.vmid = vmid
self.node = node
def __reduce__(self) -> tuple[Any, ...]:
# The keyword-only arguments are not in ``args``, which is all that
# pickle passes back by default.
return (_vm_provision_incomplete, (str(self), self.vmid, self.node))
def _vm_provision_incomplete(message: str, vmid: str, node: str) -> VMProvisionIncomplete:
return VMProvisionIncomplete(message, vmid=vmid, node=node)
def network_config(nics: list[tuple[str, bool]]) -> dict[str, Any] | None:
"""Network-config v2: DHCP on every ``(mac, dhcp)`` NIC that asks for it.
@@ -0,0 +1,60 @@
"""The shape a driver states its device's SNMP health OIDs in (``SNMP_HEALTH_PROFILE``).
Apart from ``models.py``, which is over the file size limit; ``models`` still exports them.
"""
from typing import Literal
from typing_extensions import TypedDict
class SnmpValueSpec(TypedDict, total=False):
"""Where an SNMP agent publishes one value, inside a ``SnmpHealthProfile``.
Either ``oids``, scalars asked in order (the first non-empty answer wins), or
``table``, a column walked and folded to one value by ``fold``. ``pattern`` is a
regular expression whose first group is the number, for an agent that answers
with text.
``unknown`` says the agent answers this value wrongly, or not at all, where it
should: the application reports no value -- neither the standard reading nor a
default. It stands alone, without ``oids`` or ``table``.
"""
oids: list[str]
table: str
fold: Literal["first", "average", "max"]
pattern: str
unknown: bool
class SnmpMemorySpec(TypedDict, total=False):
"""Memory: ``total`` with ``used`` or ``free``, or ``percent`` alone."""
total: SnmpValueSpec
used: SnmpValueSpec
free: SnmpValueSpec
percent: SnmpValueSpec
unit: Literal["KiB", "bytes"] # of total, used and free; KiB, as UCD-MIB counts, by default
class SnmpInterfacesSpec(TypedDict, total=False):
"""How the IF-MIB counters of a device are to be read."""
skip: str # a pattern of interface names whose counters are noise, not signal
tx_errors_are_drops: bool # the device counts dropped packets as TX errors (OPNsense)
class SnmpHealthProfile(TypedDict, total=False):
"""Where a device's SNMP agent publishes its health values.
A driver declares it as ``SNMP_HEALTH_PROFILE`` when its device does not answer
the standard UCD-MIB and IF-MIB; the application polls and computes. What a
profile does not name is read the standard way, and what it marks ``unknown`` is
not read at all. Uptime is in TimeTicks, CPU in percent busy.
"""
uptime: SnmpValueSpec
cpu: SnmpValueSpec
memory: SnmpMemorySpec
interfaces: SnmpInterfacesSpec
+57 -2
View File
@@ -27,6 +27,11 @@ A caller that passes neither keeps working as before (first use, every time),
and a driver that ignores both still connects for a caller that passes them:
old and new versions on either side do not break each other.
A connection sshd turns away before its banner -- OpenSSH 9.8+ does that to a
source it has just penalised -- is tried again after a pause
(:func:`retry_refused`, :data:`REFUSED_RETRY_WAITS`); :meth:`HostKeyCheck.connect`
does so itself, a netmiko driver wraps its ``ConnectHandler`` in it.
:class:`HostKeyCheck` is the one implementation, for a paramiko ``SSHClient``
(:meth:`~HostKeyCheck.connect`) and for netmiko (:meth:`~HostKeyCheck.netmiko`,
then :meth:`~HostKeyCheck.observe`). Like :mod:`napalm_device_types.channel` it
@@ -40,7 +45,8 @@ import hashlib
import logging
import os
import tempfile
from collections.abc import Iterator, Mapping
import time
from collections.abc import Callable, Iterator, Mapping
from contextlib import contextmanager
from dataclasses import dataclass
from typing import Any
@@ -54,6 +60,13 @@ KNOWN_KEYS_ARG = "ssh_host_keys"
#: The ``optional_args`` key that carries the callable every seen key goes to.
OBSERVER_ARG = "ssh_host_key_observer"
#: The waits before each new attempt when sshd turns a connection away before its
#: banner. OpenSSH 9.8+ does that to every connection from a source it has just
#: penalised (``PerSourcePenalties``) -- one whose connections did not
#: authenticate, a scanner's probes -- for seconds to minutes. 26 s in all
#: outlasts the 16 s penalty seen on a new Ubuntu 26.04 host (NetOrk/netork#1800).
REFUSED_RETRY_WAITS = (1.0, 3.0, 7.0, 15.0)
#: The netmiko arguments this module decides; a caller's own values give way.
_NETMIKO_HOST_KEY_ARGS = ("ssh_strict", "system_host_keys", "alt_host_keys", "alt_key_file")
@@ -185,6 +198,48 @@ def _refused_key(exc: BaseException) -> Any | None:
return None
def refused_before_banner(exc: BaseException) -> bool:
"""Whether *exc*, or what it wraps, is sshd closing the connection before its banner.
paramiko says "Error reading SSH protocol banner" (netmiko and drivers wrap
it), with the connection's EOF or reset underneath. A failed login, a
silent host and a closed port are something else: waiting does not end them.
"""
seen: set[int] = set()
stack: list[BaseException | None] = [exc]
while stack:
current = stack.pop()
if current is None or id(current) in seen:
continue
seen.add(id(current))
if "protocol banner" in str(current):
return True
stack.extend([current.__cause__, current.__context__])
return False
def retry_refused[T](
attempt: Callable[[], T],
*,
waits: tuple[float, ...] = REFUSED_RETRY_WAITS,
sleep: Callable[[float], None] | None = None,
) -> T:
"""*attempt*, again after each of *waits* while sshd turns it away before its banner.
Any other error, and the last refusal, go to the caller unchanged.
"""
pause = sleep or time.sleep
for wait in waits:
try:
return attempt()
except Exception as exc:
if not refused_before_banner(exc):
raise
logger.info("SSH connection turned away before its banner; again in %s s", wait)
pause(wait)
return attempt()
class HostKeyCheck:
"""How one driver connection verifies its host's key.
@@ -220,7 +275,7 @@ class HostKeyCheck:
"""``client.connect()`` to this host with *kwargs*, verified and reported."""
self.prepare(client)
with self.refusing():
client.connect(hostname=self.host, port=self.port, **kwargs)
retry_refused(lambda: client.connect(hostname=self.host, port=self.port, **kwargs))
return self.observe(client.get_transport())
# ------------------------------------------------------------------
+280 -47
View File
@@ -30,10 +30,19 @@ Only without such an ``Include`` is sshd_config itself rewritten, after a copy.
3. After the reload, *verify* -- the caller's -- opens a new session. The
running one survives a reload, so if the new one cannot get in, this one
puts the old state back and reloads again.
**The managing client is exempt from per-source penalties** where sshd has them
(OpenSSH 9.8+, :meth:`OpenSshServerMixin.exempt_from_ssh_penalties`). sshd
penalises a source whose connections do not authenticate -- a scanner's probes
-- and then drops every connection from it, the managing session's included.
The address is the one the host sees (``$SSH_CONNECTION``); it joins the
host's own ``PerSourcePenaltyExemptList`` in :data:`PENALTY_DROPIN_PATH`, with
the same three checks. An older sshd is not touched: it refuses the directive.
"""
from __future__ import annotations
import ipaddress
import time
from collections.abc import Callable, Iterable, Mapping, Sequence
from dataclasses import dataclass, field
@@ -57,6 +66,9 @@ MAIN_CONFIG = "/etc/ssh/sshd_config"
DROPIN_DIR = "/etc/ssh/sshd_config.d"
#: ``00-`` so it sorts before every other drop-in: sshd keeps the first value it reads.
DROPIN_PATH = f"{DROPIN_DIR}/00-netork-ssh-hardening.conf"
#: Where :meth:`OpenSshServerMixin.exempt_from_ssh_penalties` names the managing
#: client; ``00-`` like the hardening drop-in, since sshd keeps the first value.
PENALTY_DROPIN_PATH = f"{DROPIN_DIR}/00-netork-penalty-exempt.conf"
#: What marks a line this module disabled in sshd_config.
DISABLED_PREFIX = "# netork: "
@@ -65,6 +77,11 @@ _HEADER = (
"# taken out of what this server offers. Delete this file to undo it.\n"
)
_PENALTY_HEADER = (
"# Written by netOrk: the address netOrk manages this host from, exempt from\n"
"# sshd's per-source penalties (PerSourcePenalties). Delete this file to undo it.\n"
)
#: Host key type, as ``ssh-keygen -l`` prints it -> the algorithms that key serves.
#: Certificate algorithms are left out: they also need a HostCertificate.
_KEY_TYPE_ALGORITHMS: dict[str, tuple[str, ...]] = {
@@ -302,6 +319,56 @@ def render_main_config(text: str, keep: Mapping[str, Sequence[str]], kinds: Iter
return render_dropin(keep, wanted) + "\n".join(out) + ("\n" if body.endswith("\n") else "")
def parse_penalty_exemptions(text: str) -> list[str] | None:
"""The ``persourcepenaltyexemptlist`` from ``sshd -T`` output; None without penalties.
Per-source penalties came with OpenSSH 9.8. An older sshd prints neither
line and refuses ``PerSourcePenaltyExemptList`` outright -- it does not
start with it in its configuration -- so None means: write nothing.
"""
penalties = False
exempt: list[str] = []
for raw in text.splitlines():
parts = raw.strip().split(None, 1)
if not parts:
continue
word = parts[0].lower()
if word == "persourcepenalties":
penalties = True
elif word == "persourcepenaltyexemptlist" and len(parts) == 2:
value = parts[1].strip()
exempt = [] if value.lower() == "none" else [a for a in value.split(",") if a]
return exempt if penalties else None
def client_address(ssh_connection: str) -> str | None:
"""The client's address in ``$SSH_CONNECTION`` (``client port server port``), or None."""
parts = ssh_connection.split()
if len(parts) != 4:
return None
try:
return str(ipaddress.ip_address(parts[0]))
except ValueError:
return None
def is_exempt(address: str, exempt: Iterable[str]) -> bool:
"""Whether *address* is one of *exempt*'s addresses or inside one of its networks."""
ip = ipaddress.ip_address(address)
for entry in exempt:
try:
if ip in ipaddress.ip_network(entry, strict=False):
return True
except ValueError:
continue
return False
def render_penalty_dropin(exempt: Sequence[str]) -> str:
"""The drop-in stating the full exempt list."""
return _PENALTY_HEADER + f"PerSourcePenaltyExemptList {','.join(exempt)}\n"
@dataclass
class SshHardeningResult:
"""What :meth:`OpenSshServerMixin.harden_ssh_server` did.
@@ -319,6 +386,38 @@ class SshHardeningResult:
message: str = ""
@dataclass
class SshPenaltyExemption:
"""What :meth:`OpenSshServerMixin.exempt_from_ssh_penalties` did.
*address* is the one the host sees the managing client at. *applied* is
True only when sshd uses the exemption and a new session got in;
*rolled_back* says the host was changed and then put back.
"""
address: str | None = None
applied: bool = False
rolled_back: bool = False
path: str | None = None
message: str = ""
#: Why an algorithm stays, by :class:`Refusal` reason.
_WHYS = {
"last": "it is the last one",
"client": "netOrk could no longer connect",
"host_keys_unknown": "the host keys could not be read",
}
def _unchanged(
result: SshPenaltyExemption, message: str, say: Callable[[str], None]
) -> SshPenaltyExemption:
result.message = message
say(message)
return result
class OpenSshServerMixin:
"""Adds :meth:`harden_ssh_server` to a driver that can run commands as root on an OpenSSH host.
@@ -453,24 +552,7 @@ class OpenSshServerMixin:
:raises RuntimeError: if the current state cannot be read.
"""
say = log or (lambda _line: None)
say("Reading the algorithms sshd offers (sshd -T)")
sshd_t = self._sshd_effective_text()
host_keys: set[str] | None = None
if remove.get("hostkey"):
say("Reading which host keys the server has (ssh-keygen -l)")
host_keys = self._ssh_host_key_algorithms(sshd_t)
plan = plan_ssh_hardening(
parse_sshd_effective(sshd_t), remove, client=client, host_keys=host_keys
)
whys = {
"last": "it is the last one",
"client": "netOrk could no longer connect",
"host_keys_unknown": "the host keys could not be read",
}
for r in plan.refused:
last_key = r.reason == "last" and r.kind == "hostkey"
why = "it is the last one the host has a key for" if last_key else whys[r.reason]
say(f"Keeping {r.algorithm}: {why}")
plan = self._plan_hardening(remove, client, say)
kinds = plan.changed_classes
result = SshHardeningResult(plan=plan)
if not kinds:
@@ -484,7 +566,49 @@ class OpenSshServerMixin:
if main is None:
raise RuntimeError(f"cannot read {MAIN_CONFIG}")
restore = self._write_change(plan, main, result, say)
return self._apply_or_restore(plan, kinds, result, restore, verify, say)
return self._apply_or_restore(
result,
restore,
verify,
say,
ignored=lambda: self._algorithms_ignored(plan, result.path),
done="The server no longer offers the selected algorithms.",
)
def _plan_hardening(
self,
remove: Mapping[str, Iterable[str]],
client: Mapping[str, Iterable[str]] | None,
say: Callable[[str], None],
) -> SshHardeningPlan:
"""Read what sshd offers and which host keys it has; plan, and say what stays."""
say("Reading the algorithms sshd offers (sshd -T)")
sshd_t = self._sshd_effective_text()
host_keys: set[str] | None = None
if remove.get("hostkey"):
say("Reading which host keys the server has (ssh-keygen -l)")
host_keys = self._ssh_host_key_algorithms(sshd_t)
plan = plan_ssh_hardening(
parse_sshd_effective(sshd_t), remove, client=client, host_keys=host_keys
)
for r in plan.refused:
last_key = r.reason == "last" and r.kind == "hostkey"
why = "it is the last one the host has a key for" if last_key else _WHYS[r.reason]
say(f"Keeping {r.algorithm}: {why}")
return plan
def _algorithms_ignored(self, plan: SshHardeningPlan, path: str | None) -> str | None:
"""Why sshd does not use the new lists; None when it does."""
now = self.get_ssh_server_algorithms()
names = [
SSH_ALGORITHM_CLASSES[k][1] for k in plan.changed_classes if now[k] != plan.keep[k]
]
if not names:
return None
return (
f"sshd does not use the new {', '.join(names)}: a setting read before"
f" {path} decides them"
)
def _write_change(
self,
@@ -520,43 +644,33 @@ class OpenSshServerMixin:
return restore_main
def _apply_or_restore(
def _apply_or_restore[R: (SshHardeningResult, SshPenaltyExemption)](
self,
plan: SshHardeningPlan,
kinds: list[str],
result: SshHardeningResult,
result: R,
restore: Callable[[], None],
verify: Callable[[], bool] | None,
say: Callable[[str], None],
) -> SshHardeningResult:
def back(message: str, reload: bool) -> SshHardeningResult:
say(message)
say("Putting the previous configuration back")
try:
restore()
if reload:
self._reload_ssh_server()
except Exception as exc: # noqa: BLE001 - report it, the caller must look
result.message = f"{message}; putting the previous configuration back failed: {exc}"
say(result.message)
return result
result.rolled_back = True
result.message = message
return result
*,
ignored: Callable[[], str | None],
done: str,
) -> R:
"""Check what was written, reload, verify; put the old state back on any failure.
*ignored* reads ``sshd -T`` after ``sshd -t`` and says why sshd does not
use the change, or None when it does; *done* is the message of success.
"""
def back(message: str, reload: bool) -> R:
return self._put_back(result, restore, say, message, reload=reload)
try:
say("Checking the configuration (sshd -t)")
complaint = self._sshd_accepts()
if complaint:
return back(f"sshd rejected the new configuration: {complaint}", reload=False)
now = self.get_ssh_server_algorithms()
ignored = [SSH_ALGORITHM_CLASSES[k][1] for k in kinds if now[k] != plan.keep[k]]
if ignored:
return back(
f"sshd does not use the new {', '.join(ignored)}: a setting read before"
f" {result.path} decides them",
reload=False,
)
why = ignored()
if why:
return back(why, reload=False)
say("Reloading sshd")
self._reload_ssh_server()
if verify is not None:
@@ -566,6 +680,125 @@ class OpenSshServerMixin:
except Exception as exc: # noqa: BLE001 - whatever went wrong, the host goes back
return back(f"The change failed: {exc}", reload=True)
result.applied = True
result.message = "The server no longer offers the selected algorithms."
result.message = done
say(result.message)
return result
def _put_back[R: (SshHardeningResult, SshPenaltyExemption)](
self,
result: R,
restore: Callable[[], None],
say: Callable[[str], None],
message: str,
*,
reload: bool,
) -> R:
"""Say *message*, put the previous state back, and say whether that worked."""
say(message)
say("Putting the previous configuration back")
try:
restore()
if reload:
self._reload_ssh_server()
except Exception as exc: # noqa: BLE001 - report it, the caller must look
result.message = f"{message}; putting the previous configuration back failed: {exc}"
say(result.message)
return result
result.rolled_back = True
result.message = message
return result
# ── the penalty exemption ───────────────────────────────────────────
def exempt_from_ssh_penalties(
self,
*,
verify: Callable[[], bool] | None = None,
log: Callable[[str], None] | None = None,
) -> SshPenaltyExemption:
"""Exempt the address this session comes from from sshd's per-source penalties.
OpenSSH 9.8+ penalises a source whose connections do not authenticate --
a scanner's probes -- and then drops every connection from it, the
managing client's included (NetOrk/netork#1800). The address is the
one the host sees (``$SSH_CONNECTION``), so NAT and relays do not
matter. The host's own exemptions stay: the new list is the one sshd
uses now plus this address.
Only into the drop-in directory: a sshd_config that does not include it
is left alone, and the result says so. An sshd before 9.8 is not
touched -- it would not start with the directive. Checked like
:meth:`harden_ssh_server`: ``sshd -t``, ``sshd -T`` shows the address,
reload, then *verify*; on any failure the previous state goes back.
:raises RuntimeError: if the session names no client address, or
``sshd -T`` or sshd_config cannot be read.
"""
say = log or (lambda _line: None)
address = self._session_client_address()
result = SshPenaltyExemption(address=address)
say(f"netOrk connects from {address}")
say("Reading sshd's per-source penalties (sshd -T)")
exempt = parse_penalty_exemptions(self._sshd_effective_text())
reason = self._penalty_exemption_unchanged(address, exempt)
if reason is not None or exempt is None:
return _unchanged(result, reason or "", say)
restore = self._write_penalty_dropin([*exempt, address])
result.path = PENALTY_DROPIN_PATH
say(f"Wrote {PENALTY_DROPIN_PATH}")
return self._apply_or_restore(
result,
restore,
verify,
say,
ignored=lambda: self._exemption_ignored(address),
done=f"{address} is exempt from sshd's per-source penalties.",
)
def _session_client_address(self) -> str:
"""The address the host sees this session come from (``$SSH_CONNECTION``)."""
session = self.run_command('printf %s "$SSH_CONNECTION"', privileged=False, timeout=30)
address = client_address(session.stdout)
if address is None:
raise RuntimeError("the session names no client address ($SSH_CONNECTION)")
return address
def _penalty_exemption_unchanged(self, address: str, exempt: list[str] | None) -> str | None:
"""Why the exemption is left as it is; None when it is to be written."""
if exempt is None:
return "Nothing to change: this sshd has no per-source penalties (before OpenSSH 9.8)."
if is_exempt(address, exempt):
return f"Nothing to change: {address} is already exempt."
main = self._ssh_read(MAIN_CONFIG)
if main is None:
raise RuntimeError(f"cannot read {MAIN_CONFIG}")
if not has_dropin_include(main):
return (
f"Not changed: {MAIN_CONFIG} does not include {DROPIN_DIR},"
" where the exemption would go."
)
return None
def _exemption_ignored(self, address: str) -> str | None:
"""Why sshd does not use the new exemption; None when it does."""
if is_exempt(address, parse_penalty_exemptions(self._sshd_effective_text()) or []):
return None
return (
"sshd does not use the new PerSourcePenaltyExemptList: a setting read before"
f" {PENALTY_DROPIN_PATH} decides it"
)
def _write_penalty_dropin(self, exempt: Sequence[str]) -> Callable[[], None]:
"""Write the exemption list; return what puts the previous drop-in back."""
previous = self._ssh_read(PENALTY_DROPIN_PATH)
self._ssh_run(f"mkdir -p -- {quote(DROPIN_DIR)}")
self._ssh_write(PENALTY_DROPIN_PATH, render_penalty_dropin(exempt))
def restore() -> None:
if previous is None:
self._ssh_remove(PENALTY_DROPIN_PATH)
else:
self._ssh_write(PENALTY_DROPIN_PATH, previous)
return restore
+312
View File
@@ -0,0 +1,312 @@
"""The sudo password an earlier driver left in shell histories, and taking it back.
A driver that typed ``echo '<password>' | sudo -S -p "" <command>`` into an
interactive shell left that line, password and all, in the login user's shell
history, once per command -- and where only the part before the first ``&&``
ran as root, it left files root should own to the login user. A driver that has
stopped doing so still leaves both behind on every host it ever reached.
:class:`SudoHistoryMixin` reads the first as a fact and takes both back. The
commands are the same on every POSIX host, so they live here once; a driver
mixes it in, supplies ``run_command`` and says what it once left behind:
:attr:`~SudoHistoryMixin.SUDO_HISTORY_MARKER` and
:attr:`~SudoHistoryMixin.SUDO_ROOT_FILES`.
**The password never travels.** Lines are counted with ``grep -c`` and deleted
with ``sed``, both matching the marker only, as the login user who owns the
history. Nothing of a line's content is read back, and nothing goes to stdin.
**What is deleted is gone from the disk too, as far as a file can say.** ``sed
-i`` with a backup suffix renames the old file to the backup and writes a new
one; the backup -- the old file's blocks -- is overwritten with ``shred -u``,
or removed with ``rm`` where there is no ``shred``. Backups, snapshots and an
audit log of the host may still hold the password: only rotating it ends that.
**A file given back to root** must exist as a regular file, not be a symbolic
link, and be owned by the login user. ``chown -h`` never follows a link that
appears in between.
"""
from __future__ import annotations
from collections.abc import Callable
from shlex import quote
from typing import TYPE_CHECKING, ClassVar
from typing_extensions import TypedDict
from napalm_device_types.fixes import run_fix_steps
from napalm_device_types.models import FixResultDict, FixStepDict, FixStepOutcomeDict
if TYPE_CHECKING: # pragma: no cover - for type checkers only
from napalm_device_types.channel import CommandResult
#: The shell history files looked at, in the login user's home.
HISTORY_FILES = (".bash_history", ".zsh_history", ".ash_history", ".sh_history")
#: The suffix of the moment's backup ``sed -i`` makes; it carries no application's name.
HISTORY_BACKUP_SUFFIX = ".napalm-bak"
_COUNT = 'LC_ALL=C grep -c -F -e "$m" -- "$p"'
class HistorySudoLinesDict(TypedDict):
"""A history file holding lines with the sudo password (``get_history_sudo_lines``)."""
#: The file, as ``~/<name>``.
file: str
lines: int
class HistoryCleanupDict(TypedDict):
"""What one history file held before the cleanup and holds after it."""
file: str
before: int
after: int
class SudoHistoryCleanupDict(FixResultDict):
"""The result of ``remove_history_sudo_lines``: the fix, and what it changed."""
#: Every history file there is, with its lines before and after.
files: list[HistoryCleanupDict]
#: How many lines were deleted, over all files.
removed: int
#: The files given back to root.
owners_restored: list[str]
def _bre(text: str) -> str:
"""*text* as a POSIX basic regular expression matching it literally, for ``sed``."""
return "".join("\\" + c if c in "\\.[]*^$/" else c for c in text)
def _history_loop(body: str) -> str:
names = " ".join(HISTORY_FILES)
return f'for f in {names}; do p="$HOME/$f"; [ -f "$p" ] || continue; {body}; done'
def count_script(marker: str) -> str:
"""Print ``<count> <name>`` per history file there is; exit 2 when one cannot be read."""
return f"m={quote(marker)}; " + _history_loop(
f'n=$({_COUNT}); [ $? -le 1 ] || exit 2; printf "%s %s\\n" "$n" "$f"'
)
def cleanup_script(marker: str) -> str:
"""Delete the lines holding *marker* from every history file there is.
Prints per file: ``done <before> <after> <name>``, ``link <count> <name>``
(left alone), ``unreadable <name>`` or ``failed <before> <name>``.
"""
backup = f'"$p"{quote(HISTORY_BACKUP_SUFFIX)}'
return f"m={quote(marker)}; r={quote(_bre(marker))}; " + _history_loop(
f"b=$({_COUNT}); "
'[ $? -le 1 ] || { printf "unreadable %s\\n" "$f"; continue; }; '
'if [ -L "$p" ]; then printf "link %s %s\\n" "$b" "$f"; continue; fi; '
'if [ "$b" -gt 0 ]; then '
f'LC_ALL=C sed -i{quote(HISTORY_BACKUP_SUFFIX)} -e "/$r/d" "$p"; s=$?; '
# A sed that renamed the file away and wrote no new one: the backup is the history.
f'[ -e "$p" ] || mv -f {backup} "$p"; '
f"[ -e {backup} ] && {{ shred -u {backup} 2>/dev/null || rm -f {backup}; }}; "
'[ $s -eq 0 ] || { printf "failed %s %s\\n" "$b" "$f"; continue; }; '
"fi; "
f'a=$({_COUNT}); printf "done %s %s %s\\n" "$b" "$a" "$f"'
)
def owners_script(user: str, paths: tuple[str, ...]) -> str:
"""As root: give each of *paths* back to root:root where *user* owns it.
Prints ``fixed <path>`` or ``failed <path>`` per file it changed or tried to.
"""
return (
f"u={quote(user)}; for p in {' '.join(quote(p) for p in paths)}; do "
'[ -f "$p" ] && [ ! -L "$p" ] || continue; '
'[ -n "$(find "$p" -prune -user "$u" 2>/dev/null)" ] || continue; '
'if chown -h 0:0 "$p"; then printf "fixed %s\\n" "$p"; '
'else printf "failed %s\\n" "$p"; fi; done'
)
def _home(name: str) -> str:
return f"~/{name}"
def _lines(n: int) -> str:
return "1 line" if n == 1 else f"{n} lines"
def _done(rest: list[str]) -> tuple[bool, str, HistoryCleanupDict | None]:
before, after, name = int(rest[0]), int(rest[1]), _home(rest[2])
removed = before - after
text = (
f"{name}: {_lines(removed)} removed ({before} before, {after} after)"
if before
else f"{name}: no such line"
)
return after == 0, text, {"file": name, "before": before, "after": after}
def _link(rest: list[str]) -> tuple[bool, str, HistoryCleanupDict | None]:
count = int(rest[0])
return count == 0, f"{_home(rest[1])}: a symbolic link, left alone ({_lines(count)})", None
def _unreadable(rest: list[str]) -> tuple[bool, str, HistoryCleanupDict | None]:
return False, f"{_home(rest[0])}: cannot be read", None
def _failed(rest: list[str]) -> tuple[bool, str, HistoryCleanupDict | None]:
kept = _lines(int(rest[0]))
return False, f"{_home(rest[1])}: could not be rewritten ({kept} kept)", None
#: Per kind of line :func:`cleanup_script` prints: how many words follow, and what it says.
_CLEANUP_LINES: dict[
str, tuple[int, Callable[[list[str]], tuple[bool, str, HistoryCleanupDict | None]]]
] = {
"done": (3, _done),
"link": (2, _link),
"unreadable": (1, _unreadable),
"failed": (2, _failed),
}
class SudoHistoryMixin:
"""Reads and removes the sudo password an earlier driver left in shell histories.
Mixed in by the drivers that once typed it into a shell, not by a base
class, so ``callable(getattr(driver, "get_history_sudo_lines", None))``
stays a truthful answer.
"""
#: What every line carries that typed the sudo password into the shell, as a
#: fixed string -- ``"| sudo -S -p "`` for ``echo '<password>' | sudo -S -p ""
#: <command>``. None for a driver that never did: nothing is looked for.
SUDO_HISTORY_MARKER: ClassVar[str | None] = None
#: Files an earlier version wrote as the login user where root should have.
SUDO_ROOT_FILES: ClassVar[tuple[str, ...]] = ()
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
def run_command(
self,
command: str,
*,
privileged: bool = False,
timeout: float = 60,
stdin: bytes | None = None,
) -> CommandResult: ...
def get_history_sudo_lines(self) -> list[HistorySudoLinesDict]:
"""The login user's history files that hold lines with the sudo password, and how many.
A fact: counted with ``grep -c`` as the login user; no line is read.
An empty list when no file holds one, or the driver never typed it.
:raises RuntimeError: when a history file cannot be read.
"""
marker = self.SUDO_HISTORY_MARKER
if not marker:
return []
result = self.run_command(count_script(marker), timeout=30)
if result.exit_code != 0:
raise RuntimeError(
f"cannot count the shell history: {(result.stderr or result.stdout).strip()}"
)
found: list[HistorySudoLinesDict] = []
for line in result.stdout.splitlines():
count, _, name = line.strip().partition(" ")
if count.isdigit() and int(count) and name:
found.append({"file": _home(name), "lines": int(count)})
return found
def remove_history_sudo_lines(self) -> SudoHistoryCleanupDict:
"""Delete those lines, as the login user, and give :attr:`SUDO_ROOT_FILES` back to root.
Two steps, ``history`` and ``owners``; the second runs when the first
failed, too. Succeeds when no history file holds such a line any more
and every file the login user owned is root's again.
"""
files: list[HistoryCleanupDict] = []
restored: list[str] = []
parts = [
run_fix_steps([("history", lambda: self._clean_histories(files))]),
run_fix_steps([("owners", lambda: self._restore_owners(restored))]),
]
steps: list[FixStepDict] = [s for part in parts for s in part["steps"]]
return {
"success": all(part["success"] for part in parts),
"steps": steps,
"output": "\n".join(f"[{s['step']}] {s['output']}".rstrip() for s in steps),
"files": files,
"removed": sum(f["before"] - f["after"] for f in files),
"owners_restored": restored,
}
def _clean_histories(self, files: list[HistoryCleanupDict]) -> FixStepOutcomeDict:
marker = self.SUDO_HISTORY_MARKER
if not marker:
return {"ok": True, "output": "this driver never typed the sudo password"}
result = self.run_command(cleanup_script(marker), timeout=60)
said: list[str] = []
ok = result.exit_code == 0
for line in result.stdout.splitlines():
ok = self._history_line(line.split(), files, said) and ok
if result.exit_code != 0:
said.append((result.stderr or result.stdout).strip())
return {"ok": ok, "output": "\n".join(said) or "no shell history"}
@staticmethod
def _history_line(words: list[str], files: list[HistoryCleanupDict], said: list[str]) -> bool:
"""Take one line of :func:`cleanup_script` into *files* and *said*; whether it is clean."""
kind, rest = (words[0], words[1:]) if words else ("", [])
read = _CLEANUP_LINES.get(kind)
if read is None or len(rest) != read[0]:
return True
clean, text, entry = read[1](rest)
said.append(text)
if entry is not None:
files.append(entry)
return clean
def _restore_owners(self, restored: list[str]) -> FixStepOutcomeDict:
paths = tuple(self.SUDO_ROOT_FILES)
if not paths:
return {"ok": True, "output": "no file to check"}
who = self.run_command("id -u; id -un", timeout=30).stdout.split()
if len(who) < 2:
return {"ok": False, "output": "cannot tell who the login user is"}
uid, user = who[0], who[1]
if uid == "0":
return {"ok": True, "output": "logged in as root: every file it wrote is root's"}
result = self.run_command(owners_script(user, paths), privileged=True, timeout=30)
said: list[str] = []
ok = result.exit_code == 0
for line in result.stdout.splitlines():
kind, _, path = line.strip().partition(" ")
if kind == "fixed":
restored.append(path)
said.append(f"{path}: owned by root again")
elif kind == "failed":
ok = False
said.append(f"{path}: could not be given back to root")
if result.exit_code != 0:
said.append((result.stderr or result.stdout).strip())
return {"ok": ok, "output": "\n".join(said) or f"no file owned by {user}"}
__all__ = [
"HISTORY_BACKUP_SUFFIX",
"HISTORY_FILES",
"HistoryCleanupDict",
"HistorySudoLinesDict",
"SudoHistoryCleanupDict",
"SudoHistoryMixin",
"cleanup_script",
"count_script",
"owners_script",
]
+285
View File
@@ -0,0 +1,285 @@
"""TLS certificates: trust on first use by fingerprint, a changed one refused before any login.
Network devices ship self-signed certificates no CA vouches for. A driver that
then turns verification off logs in with the device's credentials at whatever
answers on its address (NetOrk/netork#1167, NAPALM/napalm-hpe-aruba-procurve#17).
The same answer as for SSH host keys (:mod:`napalm_device_types.ssh_host_keys`):
the layer above keeps the store of the certificates it trusts, a driver verifies.
* It receives the fingerprints to expect in ``optional_args["tls_fingerprints"]``
(:data:`KNOWN_FINGERPRINTS_ARG`), by the port they were seen on:
``{"443": "SHA256:AB:CD:…"}``, the SHA-256 of the DER certificate as
``openssl x509 -fingerprint -sha256`` prints it (the ``SHA256:`` prefix,
colons and case are optional, :func:`normalise_fingerprint`). With a
fingerprint, any other certificate is refused before a password leaves the
caller; no CA is consulted. Without one, the first certificate is accepted:
trust on first use.
* It reports every certificate it sees through
``optional_args["tls_certificate_observer"]`` (:data:`OBSERVER_ARG`), a
callable that receives one dict per connection::
{"host": str, "port": int,
"seen": {"fingerprint"},
"expected": {"fingerprint"} | None,
"accepted": bool}
``accepted`` is False only for a refused certificate; the connection then
raises :class:`TlsCertificateMismatch`. A failing observer is logged and
ignored.
A caller that passes neither keeps working as before (first use, every time),
and a driver that ignores both still connects for a caller that passes them.
:class:`CertificateCheck` is the one implementation. :meth:`~CertificateCheck.verify_server`
takes the certificate from a handshake of its own, before any request;
:meth:`~CertificateCheck.pin` makes every later connection of a ``requests``
session present that same certificate (urllib3's ``assert_fingerprint``), so
another host cannot step in between the check and the login;
:meth:`~CertificateCheck.refusing` turns such a refusal into
:class:`TlsCertificateMismatch`; :meth:`~CertificateCheck.observe` reports the
certificate once the login stands.
"""
from __future__ import annotations
import hashlib
import logging
import re
import socket
import ssl
from collections.abc import Callable, Iterator, Mapping
from contextlib import contextmanager
from dataclasses import dataclass
from typing import Any
from napalm.base.exceptions import ConnectionException
logger = logging.getLogger(__name__)
#: The ``optional_args`` key that carries the fingerprints to expect, by port.
KNOWN_FINGERPRINTS_ARG = "tls_fingerprints"
#: The ``optional_args`` key that carries the callable every seen certificate goes to.
OBSERVER_ARG = "tls_certificate_observer"
_PREFIX = "SHA256:"
_SHA256_HEX = re.compile(r"^[0-9A-F]{64}$")
#: What urllib3 says when a connection presents another certificate than the pinned one
#: (``got "…"``; 1.26 writes the digest as ``b'…'``, escaped once requests wraps it).
_URLLIB3_MISMATCH = re.compile(r"Fingerprints did not match.*?got \"(?:b\\?')?([0-9a-fA-F]{64})")
def fingerprint(der: bytes) -> str:
"""``SHA256:AB:CD:…``: the SHA-256 of a DER certificate, as openssl prints it."""
return _format(hashlib.sha256(der).hexdigest())
def _format(hex_digest: str) -> str:
upper = hex_digest.upper()
return _PREFIX + ":".join(upper[i : i + 2] for i in range(0, len(upper), 2))
def normalise_fingerprint(value: Any) -> str:
"""*value* as :func:`fingerprint` writes it; ``SHA256:``, colons, spaces and case optional."""
if not isinstance(value, str):
raise ValueError(f"not a SHA-256 certificate fingerprint: {value!r}")
text = value.strip()
if text[: len(_PREFIX)].upper() == _PREFIX:
text = text[len(_PREFIX) :]
digits = re.sub(r"[\s:]", "", text).upper()
if not _SHA256_HEX.match(digits):
raise ValueError(f"not a SHA-256 certificate fingerprint: {value!r}")
return _format(digits)
def _hex(fingerprint_: str) -> str:
"""The bare lower-case hex urllib3's ``assert_fingerprint`` takes."""
return fingerprint_.removeprefix(_PREFIX).replace(":", "").lower()
def fetch_certificate(
host: str,
port: int,
timeout: float,
*,
connect: Callable[..., socket.socket] = socket.create_connection,
) -> bytes:
"""The DER certificate *host* presents on *port*, from a handshake that sends nothing else.
No CA and no host name are checked: the certificate is judged by its fingerprint.
"""
context = ssl.create_default_context()
context.check_hostname = False
context.verify_mode = ssl.CERT_NONE
with connect((host, port), timeout=timeout) as sock:
with context.wrap_socket(sock, server_hostname=host) as tls:
der = tls.getpeercert(binary_form=True)
if not der: # pragma: no cover - a TLS server always presents one
raise ConnectionException(f"{host}:{port} presented no TLS certificate")
return der
@dataclass(frozen=True)
class CertificateObservation:
"""The certificate one connection saw, against the one it was to expect."""
host: str
port: int
#: The fingerprint of the certificate presented.
seen: str
#: The fingerprint it was to present; None on first use.
expected: str | None
@property
def accepted(self) -> bool:
"""True on first use and for the expected certificate; False for a refused one."""
return self.expected is None or self.expected == self.seen
def as_dict(self) -> dict[str, Any]:
return {
"host": self.host,
"port": self.port,
"seen": {"fingerprint": self.seen},
"expected": {"fingerprint": self.expected} if self.expected else None,
"accepted": self.accepted,
}
class TlsCertificateMismatch(ConnectionException): # type: ignore[misc]
"""The device presented another TLS certificate than the one it is known by.
Raised before any request that carries credentials: nothing was sent to it.
"""
def __init__(self, observation: CertificateObservation) -> None:
self.observation = observation
super().__init__(
f"TLS certificate mismatch at {observation.host}:{observation.port}: it presented "
f"{observation.seen}, the trusted certificate is {observation.expected or 'none'}. "
"Nothing was sent to it."
)
def _expected(known: Any, port: int) -> str | None:
"""The fingerprint in *known* (``{port: fingerprint}``, port as text or number) for *port*."""
if not known:
return None
if not isinstance(known, Mapping):
raise ValueError(f"TLS fingerprints must be given by port, not as {type(known).__name__}")
value = known.get(str(port)) or known.get(port)
return normalise_fingerprint(value) if value else None
def _refused_fingerprint(exc: BaseException) -> str | None:
"""The fingerprint urllib3 saw when it refused a pinned connection, in *exc*'s chain."""
seen: set[int] = set()
stack: list[BaseException | None] = [exc]
while stack:
current = stack.pop()
if current is None or id(current) in seen:
continue
seen.add(id(current))
match = _URLLIB3_MISMATCH.search(str(current))
if match:
return _format(match.group(1))
stack.extend([current.__cause__, current.__context__])
return None
class CertificateCheck:
"""How one driver connection verifies its host's TLS certificate.
*optional_args* are the driver's (both keys optional), *host* and *port* the
address the connection goes to.
"""
def __init__(self, optional_args: Mapping[str, Any] | None, host: str, port: int) -> None:
args = optional_args or {}
self.host = host
self.port = int(port)
self.expected = _expected(args.get(KNOWN_FINGERPRINTS_ARG), self.port)
self._observer = args.get(OBSERVER_ARG)
self._verified: CertificateObservation | None = None
def verify_server(
self,
timeout: float,
*,
connect: Callable[..., socket.socket] = socket.create_connection,
) -> CertificateObservation:
"""Take the certificate from a handshake of its own and :meth:`verify` it."""
return self.verify(fetch_certificate(self.host, self.port, timeout, connect=connect))
def verify(self, der: bytes) -> CertificateObservation:
"""Judge the DER certificate *der*: a refused one is reported and raises.
An accepted one is reported by :meth:`observe`, once the login stands.
"""
observation = CertificateObservation(self.host, self.port, fingerprint(der), self.expected)
if not observation.accepted:
self._report(observation)
raise TlsCertificateMismatch(observation)
self._verified = observation
return observation
def pin(self, session: Any) -> None:
"""Every HTTPS connection of the ``requests`` *session* must present the verified one."""
verified = self._require_verified()
session.mount("https://", _pinned_adapter(_hex(verified.seen)))
@contextmanager
def refusing(self) -> Iterator[None]:
"""Around requests on a pinned session: a refused certificate becomes a mismatch."""
try:
yield
except Exception as exc:
offered = _refused_fingerprint(exc)
if offered is None:
raise
trusted = self.expected or self._require_verified().seen
observation = CertificateObservation(self.host, self.port, offered, trusted)
self._report(observation)
raise TlsCertificateMismatch(observation) from exc
def observe(self) -> CertificateObservation:
"""After the login: the verified certificate, reported."""
verified = self._require_verified()
self._report(verified)
return verified
def _require_verified(self) -> CertificateObservation:
if self._verified is None:
raise RuntimeError("verify the certificate first (verify_server or verify)")
return self._verified
def _report(self, observation: CertificateObservation) -> None:
if not callable(self._observer):
return
try:
self._observer(observation.as_dict())
except Exception: # the store is the caller's; a connection does not hang on it
logger.warning(
"Could not report the TLS certificate of %s:%s",
self.host,
self.port,
exc_info=True,
)
def _pinned_adapter(hex_digest: str) -> Any:
"""A ``requests`` adapter whose connections, direct or through a proxy, must match *hex_digest*.
``requests`` is imported here: it comes with napalm, and a module that only
verifies a fingerprint does not need it loaded.
"""
from requests.adapters import HTTPAdapter
class _PinnedAdapter(HTTPAdapter): # type: ignore[misc,unused-ignore]
def init_poolmanager(self, *args: Any, **kwargs: Any) -> None:
kwargs["assert_fingerprint"] = hex_digest
super().init_poolmanager(*args, **kwargs)
def proxy_manager_for(self, proxy: str, **kwargs: Any) -> Any:
kwargs["assert_fingerprint"] = hex_digest
return super().proxy_manager_for(proxy, **kwargs)
return _PinnedAdapter()
+26
View File
@@ -0,0 +1,26 @@
"""Moving a guest to another node of its cluster (``HypervisorDriver.migrate_vm``).
``get_vm_migration_preflight`` reports what the platform says before a move:
which nodes would take the guest, which refuse it and why, and what the guest
holds on its current node. ``migrate_vm`` then moves it and blocks until the
platform's task ended; a failure raises with the platform's own cause.
``set_node_maintenance`` switches a node's HA maintenance mode, which makes the
HA manager move its HA guests away while it is on.
Facts only: which node to pick, and whether a local disk is worth copying, is
the caller's to decide.
"""
from typing import TypedDict
class MigrationPreflightDict(TypedDict):
"""Where a guest could move to, and what stands in the way."""
running: bool
allowed_nodes: list[str] # nodes the platform would move it to
refused_nodes: dict[str, str] # node -> the platform's reason, in plain words
# volume ids on storage the target does not share; they block nothing by
# themselves, the migration copies them
local_disks: list[str]
local_resources: list[str] # e.g. "hostpci0", "usb0": they block the migration
+29
View File
@@ -0,0 +1,29 @@
"""Asking for a VM's clean shutdown without waiting for it.
``HypervisorDriver.stop_vm`` blocks until the VM is stopped, within a limit
of the driver's own. A caller that keeps its waits as timers asks with
``begin_stop_vm`` and reads ``get_vm_status`` until the VM says it stopped.
Declared under ``if TYPE_CHECKING``: a driver mixes this in when its
hypervisor can take the request on its own; ``hasattr`` tells a caller
whether it did. No role base carries it.
"""
from __future__ import annotations
from typing import TYPE_CHECKING
class VmStopRequestMixin:
if TYPE_CHECKING:
def begin_stop_vm(self, name: str) -> None:
"""
Asks the hypervisor to shut a virtual machine down cleanly (ACPI)
and returns as soon as the request was accepted.
:param name: VM name or its ``vmid``.
:raises ValueError: If no VM with the given name/ID exists.
:raises RuntimeError: If the hypervisor refuses the request.
"""
...
+6 -7
View File
@@ -4,10 +4,10 @@ build-backend = "setuptools.build_meta"
[project]
name = "napalm-device-types"
version = "4.22.0"
version = "4.23.0"
description = "Abstract device-type base classes for NAPALM drivers"
readme = "README.md"
requires-python = ">=3.10"
requires-python = ">=3.12"
license = "Apache-2.0"
authors = [
{ name = "Christian Manivong", email = "christian@manivong.de" },
@@ -30,8 +30,6 @@ classifiers = [
"Intended Audience :: System Administrators",
"Operating System :: OS Independent",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Topic :: System :: Networking",
@@ -39,7 +37,8 @@ classifiers = [
"Typing :: Typed",
]
dependencies = [
"napalm>=4.0",
"napalm>=5.1", # older releases import pkg_resources, which a 3.12 venv lacks
"requests>=2.25.0",
"typing-extensions>=4.0",
]
@@ -65,7 +64,7 @@ napalm_device_types = ["py.typed"]
[tool.ruff]
line-length = 100
target-version = "py310"
target-version = "py312"
[tool.ruff.lint]
select = ["E", "F", "I", "UP", "B"]
@@ -75,7 +74,7 @@ select = ["E", "F", "I", "UP", "B"]
"tests/**" = ["E501"]
[tool.mypy]
python_version = "3.10"
python_version = "3.12"
strict = true
ignore_missing_imports = true
+1 -2
View File
@@ -25,7 +25,6 @@
"napalm_device_types/firewall_rules.py::FirewallRuleMixin.diff_firewall_rules": 56,
"napalm_device_types/hypervisor.py::HypervisorDriver.create_vm_from_cloud_init": 84,
"napalm_device_types/hypervisor.py::HypervisorDriver.get_vm_config": 77,
"napalm_device_types/ssh_server.py::OpenSshServerMixin.harden_ssh_server": 52,
"napalm_device_types/ssh_server.py::plan_ssh_hardening": 60,
"napalm_device_types/storage.py::StorageDriver.get_volumes": 52,
"napalm_device_types/switch.py::SwitchDriver.get_dot1x_ports": 51,
@@ -33,7 +32,7 @@
},
"mypy_relaxed": [],
"python_file_lines": {
"napalm_device_types/models.py": 1504
"napalm_device_types/models.py": 1463
},
"untyped_public": []
}
+306 -13
View File
@@ -20,6 +20,7 @@ from napalm_device_types.apt_proxy import (
AptProxyMixin,
apt_fetch_command,
comment_out_command,
https_proxy_for,
parse_apt_fetch,
parse_apt_proxy,
same_proxy_url,
@@ -141,7 +142,7 @@ class TestWhereItIsSet:
class TestSources:
def test_https_sources_bypass_an_http_proxy(self):
def test_sources_by_scheme(self):
report = parse_apt_proxy(
_report(
sources=[
@@ -155,6 +156,102 @@ class TestSources:
assert report["https_sources"] == ["https://download.docker.com/linux/ubuntu/dists/noble/"]
class TestHttpsProxy:
"""apt's https method asks ``Acquire::https::Proxy`` and then ``Acquire::http::Proxy``:
an http proxy alone takes the https sources too (NetOrk/netork#1811). Measured with
apt 2.8.3's apt-helper against a dead port."""
def test_the_http_proxy_alone_takes_https_too(self):
report = parse_apt_proxy(_report(dump=['Acquire::http::Proxy "http://c:3142";']))
assert report["https_proxy"] == "http://c:3142"
def test_an_https_proxy_of_its_own_wins(self):
report = parse_apt_proxy(
_report(
dump=[
'Acquire::http::Proxy "http://c:3142";',
'Acquire::https::Proxy "http://other:3128";',
]
)
)
assert (report["proxy"], report["https_proxy"]) == ("http://c:3142", "http://other:3128")
@pytest.mark.parametrize("value", ["DIRECT", "direct"])
def test_direct_https_is_none(self, value):
report = parse_apt_proxy(
_report(
dump=['Acquire::http::Proxy "http://c:3142";', f'Acquire::https::Proxy "{value}";']
)
)
assert (report["proxy"], report["https_proxy"]) == ("http://c:3142", None)
def test_an_empty_https_proxy_is_not_set(self):
"""apt takes an empty value for unset and goes on to the http one."""
report = parse_apt_proxy(
_report(dump=['Acquire::http::Proxy "http://c:3142";', 'Acquire::https::Proxy "";'])
)
assert report["https_proxy"] == "http://c:3142"
def test_per_host_https_then_http(self):
report = parse_apt_proxy(
_report(
dump=[
'Acquire::http::Proxy::a.example "DIRECT";',
'Acquire::http::Proxy::b.example "DIRECT";',
'Acquire::https::Proxy::b.example "http://c:3142";',
]
)
)
assert report["https_per_host"] == {"a.example": "DIRECT", "b.example": "http://c:3142"}
assert report["per_host"] == {"a.example": "DIRECT", "b.example": "DIRECT"}
def test_where_the_https_proxy_is_set(self):
report = parse_apt_proxy(
_report(
parts=[
f'{_D}00proxy:1:Acquire::http::Proxy "http://c:3142";',
f'{_D}90curtin-aptproxy:2:Acquire::https::Proxy "http://c:3142";',
f'{_D}91x:1:Acquire::https {{ Proxy "http://n:1"; }};',
f'{_D}92x:1:// Acquire::https::Proxy "http://c:3142";',
f'{_D}93x:1:Acquire::https::Proxy::h.example "DIRECT";',
]
)
)
assert report["https_settings"] == [
{
"file": f"{_D}90curtin-aptproxy",
"line": 2,
"value": "http://c:3142",
"editable": True,
},
{"file": f"{_D}91x", "line": 1, "value": "http://n:1", "editable": False},
]
assert [s["file"] for s in report["settings"]] == [f"{_D}00proxy"]
@pytest.mark.parametrize(
"host, proxy",
[
("deb.example", "http://c:3142"),
("direct.example", None),
("Other.Example", "http://o:1"),
],
)
def test_the_proxy_for_one_host(self, host, proxy):
report = parse_apt_proxy(
_report(
dump=[
'Acquire::http::Proxy "http://c:3142";',
'Acquire::https::Proxy::direct.example "DIRECT";',
'Acquire::http::Proxy::other.example "http://o:1";',
]
)
)
assert https_proxy_for(report, host) == proxy
class TestSameProxyUrl:
@pytest.mark.parametrize(
"a, b",
@@ -199,6 +296,12 @@ E: Download Failed"""
_FETCHED = """Get:1 http://172.22.8.50:3142/acng-report.html
Fetched 14.0 kB in 0s (0 B/s)"""
#: apt-cacher-ng without a PassThroughPattern, as apt 2.8.3 tells it (NetOrk/netork#1811).
_CONNECT_DENIED = """Err:1 https://download.docker.com/linux/ubuntu/dists/noble/InRelease
Invalid response from proxy: HTTP/1.0 403 CONNECT denied (ask the admin to allow HTTPS tunnels) [IP: 10.40.100.10 3142]
E: Failed to fetch https://download.docker.com/linux/ubuntu/dists/noble/InRelease Invalid response from proxy: HTTP/1.0 403 CONNECT denied (ask the admin to allow HTTPS tunnels) [IP: 10.40.100.10 3142]
E: Download Failed"""
def _fetch(output, rc, match=None):
lines = ["APROXY_BEGIN", "[fetch]", *output.splitlines(), "[rc]", str(rc)]
@@ -243,6 +346,12 @@ class TestFetch:
assert result["expected"] is False
assert result["error"] == "503 Host not found [IP: 172.22.8.50 3142]"
def test_a_refused_tunnel(self):
result = parse_apt_fetch(_fetch(_CONNECT_DENIED, 100))
assert result["reachable"] is False
assert "403 CONNECT denied" in result["error"]
def test_without_apt_helper_nothing_could_be_tried(self):
with pytest.raises(ValueError, match="apt-helper"):
parse_apt_fetch(_fetch("sh: 1: /usr/lib/apt/apt-helper: not found", 127))
@@ -265,6 +374,21 @@ class TestFetchCommand:
command = apt_fetch_command("http://archive.ubuntu.com/ubuntu/dists/noble/InRelease")
assert "DIRECT" not in command
def test_an_https_source_through_apt_s_own_proxy(self):
command = apt_fetch_command(
"https://download.docker.com/linux/ubuntu/dists/noble/InRelease"
)
assert "DIRECT" not in command
assert "download-file https://download.docker.com/" in command
def test_past_the_proxy_to_an_https_host(self):
"""For https the host's own https setting comes first; the http one alone
would lose to an https proxy for that host."""
command = apt_fetch_command("https://h.example/x", direct=True)
assert "'Acquire::https::Proxy::h.example=DIRECT'" in command
assert "'Acquire::http::Proxy::h.example=DIRECT'" in command
@pytest.mark.parametrize("url", ["http://c:3142/x'; rm -rf /", "ftp://c/x", "c:3142"])
def test_a_url_that_would_break_the_command_is_refused(self, url):
with pytest.raises(ValueError):
@@ -275,6 +399,7 @@ def test_the_commands_are_valid_posix_sh():
for command in (
APT_PROXY_COMMAND,
apt_fetch_command("http://c:3142/x", direct=True, expect="y"),
apt_fetch_command("https://h.example/dists/x/InRelease"),
):
subprocess.run(["sh", "-n", "-c", command], check=True)
@@ -283,16 +408,28 @@ def test_the_commands_are_valid_posix_sh():
def test_the_report_command_on_this_host():
output = subprocess.run(["sh", "-c", APT_PROXY_COMMAND], capture_output=True, text=True).stdout
report = parse_apt_proxy(output)
assert set(report) >= {"proxy", "settings", "https_sources"}
assert set(report) >= {"proxy", "https_proxy", "settings", "https_settings", "https_sources"}
_DOCKER = "https://download.docker.com/linux/ubuntu/dists/noble/"
class _Host(AptProxyMixin):
"""A Debian host: its apt files, and what apt-config would make of them."""
"""A Debian host: its apt files, and what apt-config would make of them.
def __init__(self, files=None, fetch_rc=0, apt=True):
*https_sources* are its sources over https. *tunnel* is whether the proxy lets
them through; *direct_rc* is how a fetch of one goes that takes no proxy.
"""
def __init__(
self, files=None, fetch_rc=0, apt=True, https_sources=(), tunnel=True, direct_rc=0
):
self.files = dict(files or {})
self.fetch_rc = fetch_rc
self.apt = apt
self.https_sources = list(https_sources)
self.tunnel = tunnel
self.direct_rc = direct_rc
self.root = []
self.fetched = []
@@ -309,26 +446,36 @@ class _Host(AptProxyMixin):
out.append((path, n, text))
return out
def _effective(self):
def _effective(self, scheme="http"):
value = None
for _path, _n, text in self._settings():
m = re.match(r'\s*Acquire::http::Proxy\s+"([^"]*)"', text, re.I)
m = re.match(rf'\s*Acquire::{scheme}::Proxy\s+"([^"]*)"', text, re.I)
if m:
value = m.group(1)
return value
def _run_apt_command(self, command):
if command == APT_PROXY_COMMAND:
value = self._effective()
value, https = self._effective(), self._effective("https")
dump = [f'Acquire::http::Proxy "{value}";'] if value else []
dump += [f'Acquire::https::Proxy "{https}";'] if https else []
return _report(
dump=[f'Acquire::http::Proxy "{value}";'] if value else [],
dump=dump,
parts=[f"{p}:{n}:{t}" for p, n, t in self._settings() if p.startswith(_D)],
main=[f"{p}:{n}:{t}" for p, n, t in self._settings() if not p.startswith(_D)],
sources=["http://deb.debian.org/debian/dists/trixie/"],
sources=["http://deb.debian.org/debian/dists/trixie/", *self.https_sources],
)
self.fetched.append(command)
if re.search(r"download-file '?https://", command):
return self._fetch_https(command)
return _fetch(_FETCHED if self.fetch_rc == 0 else _REFUSED, self.fetch_rc, match=1)
def _fetch_https(self, command):
https = self._effective("https") or self._effective()
if "DIRECT" in command or not https or https.upper() == "DIRECT":
return _fetch(_FETCHED if self.direct_rc == 0 else _UNRESOLVED, self.direct_rc)
return _fetch(_FETCHED, 0) if self.tunnel else _fetch(_CONNECT_DENIED, 100)
def _write_apt_file(self, path, content):
self.files[path] = content
return {"ok": True, "output": f"wrote {path}"}
@@ -346,6 +493,41 @@ class _Host(AptProxyMixin):
return {"ok": True, "output": done}
class TestCheckHttpsSource:
def test_one_source_the_way_apt_fetches_it(self):
host = _Host(
{f"{_D}00proxy": 'Acquire::http::Proxy "http://c:3142";\n'},
https_sources=[_DOCKER],
tunnel=False,
)
result = host.check_https_source()
assert result["url"] == f"{_DOCKER}InRelease"
assert result["proxy"] == "http://c:3142"
assert result["reachable"] is False and "403 CONNECT denied" in result["error"]
assert "DIRECT" not in host.fetched[0]
def test_none_without_an_https_source(self):
host = _Host({f"{_D}00proxy": 'Acquire::http::Proxy "http://c:3142";\n'})
assert host.check_https_source() is None
def test_none_where_https_goes_direct(self):
host = _Host(
{
f"{_D}00proxy": (
'Acquire::http::Proxy "http://c:3142";\nAcquire::https::Proxy "DIRECT";\n'
)
},
https_sources=[_DOCKER],
)
assert host.check_https_source() is None
def test_not_on_a_host_without_apt(self):
with pytest.raises(NotImplementedError):
_Host(apt=False).check_https_source()
class TestTheMixin:
def test_it_is_a_package_proxy(self):
assert issubclass(AptProxyMixin, PackageProxyMixin)
@@ -371,6 +553,10 @@ class TestTheMixin:
assert "Proxy::c=DIRECT" in host.fetched[0]
def _step(result, name):
return next(s for s in result["steps"] if s["step"] == name)
class TestConfigurePackageProxy:
def test_a_fresh_host(self):
host = _Host()
@@ -378,7 +564,14 @@ class TestConfigurePackageProxy:
result = host.configure_package_proxy("http://c:3142")
assert result["success"] is True
assert [s["step"] for s in result["steps"]] == ["configure", "conflicts", "verify", "fetch"]
assert [s["step"] for s in result["steps"]] == [
"configure",
"conflicts",
"verify",
"fetch",
"https",
]
assert "no https source" in result["steps"][-1]["output"]
assert host.files[f"{_D}00proxy"] == 'Acquire::http::Proxy "http://c:3142";\n'
assert "InRelease" in host.fetched[0] and "DIRECT" not in host.fetched[0]
@@ -410,7 +603,7 @@ class TestConfigurePackageProxy:
def test_a_setting_it_cannot_comment_out_fails_verification(self):
host = _Host({"/etc/apt/apt.conf": 'Acquire::http { Proxy "http://older:8080"; };\n'})
host._effective = lambda: "http://older:8080"
host._effective = lambda scheme="http": "http://older:8080"
result = host.configure_package_proxy("http://c:3142")
@@ -428,6 +621,100 @@ class TestConfigurePackageProxy:
assert result["steps"][-1]["step"] == "fetch"
assert "Connection refused" in result["steps"][-1]["output"]
def test_an_https_source_the_cache_tunnels_stays_with_it(self):
host = _Host(https_sources=[_DOCKER])
result = host.configure_package_proxy("http://c:3142")
assert result["success"] is True
assert host.files[f"{_D}00proxy"] == 'Acquire::http::Proxy "http://c:3142";\n'
assert result["steps"][-1]["output"] == (
f"fetched {_DOCKER}InRelease through http://c:3142"
)
def test_an_https_source_the_cache_refuses_goes_direct(self):
"""NetOrk/netork#1811: apt-cacher-ng without a PassThroughPattern answers
403 to every https source apt sends it."""
host = _Host(https_sources=[_DOCKER], tunnel=False)
result = host.configure_package_proxy("http://c:3142")
assert result["success"] is True
assert host.files[f"{_D}00proxy"] == (
'Acquire::http::Proxy "http://c:3142";\nAcquire::https::Proxy "DIRECT";\n'
)
output = result["steps"][-1]["output"]
assert "http://c:3142 refused https" in output and "403 CONNECT denied" in output
assert f"https sources go direct now: fetched {_DOCKER}InRelease" in output
def test_a_later_https_line_to_the_cache_is_commented_out(self):
"""Ubuntu's installer writes the https proxy too; it would win over 00proxy."""
curtin = f"{_D}90curtin-aptproxy"
host = _Host(
{
curtin: (
'Acquire::http::Proxy "http://c:3142";\n'
'Acquire::https::Proxy "http://c:3142";\n'
)
},
https_sources=[_DOCKER],
tunnel=False,
)
result = host.configure_package_proxy("http://c:3142")
assert result["success"] is True
assert host.files[curtin] == (
'Acquire::http::Proxy "http://c:3142";\n// Acquire::https::Proxy "http://c:3142";\n'
)
assert f"{curtin}:2 (http://c:3142) commented out" in result["steps"][-1]["output"]
def test_an_https_proxy_of_its_own_is_not_the_fix_s(self):
host = _Host(
{f"{_D}01https": 'Acquire::https::Proxy "http://corp:3128";\n'},
https_sources=[_DOCKER],
tunnel=False,
)
result = host.configure_package_proxy("http://c:3142")
assert result["success"] is True
assert "no https source goes through http://c:3142" in result["steps"][-1]["output"]
assert f"{_D}00proxy" in host.files and "https" not in host.files[f"{_D}00proxy"]
def test_an_http_error_through_the_tunnel_is_a_tunnel(self):
"""The mirror answered, so the cache let the source through."""
host = _Host(https_sources=[_DOCKER])
host._fetch_https = lambda command: _fetch(_HTTP_ERROR, 100)
result = host.configure_package_proxy("http://c:3142")
assert result["success"] is True
assert "https" not in host.files[f"{_D}00proxy"]
def test_refused_and_no_way_direct_fails(self):
host = _Host(https_sources=[_DOCKER], tunnel=False, direct_rc=100)
result = host.configure_package_proxy("http://c:3142")
assert result["success"] is False
assert result["steps"][-1]["step"] == "https"
assert "Could not resolve" in result["steps"][-1]["output"]
def test_a_later_https_line_it_cannot_comment_out_fails(self):
host = _Host(
{"/etc/apt/apt.conf": 'Acquire::https { Proxy "http://c:3142"; };\n'},
https_sources=[_DOCKER],
tunnel=False,
)
host._effective = lambda scheme="http": "http://c:3142"
result = host.configure_package_proxy("http://c:3142")
assert result["success"] is False
assert result["steps"][-1]["step"] == "https"
assert "/etc/apt/apt.conf:1" in result["steps"][-1]["output"]
def test_without_an_http_source_the_fetch_says_so(self):
host = _Host()
real = host._run_apt_command
@@ -440,7 +727,7 @@ class TestConfigurePackageProxy:
result = host.configure_package_proxy("http://c:3142")
assert result["success"] is True
assert "no http source" in result["steps"][-1]["output"]
assert "no http source" in _step(result, "fetch")["output"]
def test_a_direct_source_is_not_proof(self):
host = _Host()
@@ -451,7 +738,7 @@ class TestConfigurePackageProxy:
result = host.configure_package_proxy("http://c:3142")
assert "no http source" in result["steps"][-1]["output"]
assert "no http source" in _step(result, "fetch")["output"]
@pytest.mark.parametrize("url", ['http://x"; Acquire::x "y', "ftp://c", "c:3142"])
def test_a_url_that_would_break_the_file_is_refused(self, url):
@@ -531,3 +818,9 @@ def test_the_comment_out_command():
"sed -i -e '2{/^[[:space:]]*Acquire::http::Proxy[[:space:]]/Is|^|// |}' "
"-e '4{/^[[:space:]]*Acquire::http::Proxy[[:space:]]/Is|^|// |}' /etc/apt/apt.conf"
)
def test_the_comment_out_command_for_https():
assert comment_out_command("/etc/apt/apt.conf", [3], scheme="https") == (
"sed -i -e '3{/^[[:space:]]*Acquire::https::Proxy[[:space:]]/Is|^|// |}' /etc/apt/apt.conf"
)
+27
View File
@@ -0,0 +1,27 @@
"""An ARP or neighbour entry whose age the device does not report says so.
NAPALM's ``get_arp_table`` and ``get_ipv6_neighbors_table`` require an ``age``
on every entry, and a driver that could not read one used to fill in ``0.0``.
To a caller that is the freshest sighting there is: netOrk took a neighbour the
device had not confirmed for hours as seen this second (NetOrk/netork#1327).
Unknown is ``-1.0``, as in NAPALM's own IOS driver; ``0.0`` means "just now".
"""
from __future__ import annotations
from napalm_device_types import ARP_AGE_UNKNOWN
from napalm_device_types.arp import ARP_AGE_UNKNOWN as FROM_MODULE
def test_an_unknown_age_is_minus_one():
assert ARP_AGE_UNKNOWN == -1.0
assert isinstance(ARP_AGE_UNKNOWN, float)
def test_the_package_exports_the_modules_constant():
assert ARP_AGE_UNKNOWN is FROM_MODULE
def test_no_real_age_can_be_mistaken_for_unknown():
# An age is seconds since the entry was last confirmed: never negative.
assert ARP_AGE_UNKNOWN < 0
+168
View File
@@ -0,0 +1,168 @@
"""A hypervisor says which cluster it belongs to, and what that cluster holds.
A Proxmox node is one member of a cluster and can read the whole of it; a vCenter
manages several clusters and standalone hosts at once; a standalone host is a
cluster of one. One getter serves all three: ``get_clusters()`` lists every
cluster the connection can see, each with its nodes, its storages and its
networks, and which nodes see which. What a caller does with that (placement,
headroom, identity) is the caller's.
A new VM can be placed on a named node (``create_vm_from_cloud_init(node=…)``),
so a manager that runs several hosts takes the caller's choice instead of making
its own.
The declarations sit under ``TYPE_CHECKING`` (see test_role_contracts), so the
signatures are read from the source rather than from the class.
"""
from __future__ import annotations
import ast
import inspect
from typing import NotRequired, get_type_hints
import napalm_device_types.hypervisor as hypervisor_module
from napalm_device_types import (
BackupDict,
BackupJobDict,
ClusterDict,
ClusterNetworkDict,
ClusterNodeDict,
ClusterStorageDict,
HAResourceDict,
HypervisorDriver,
)
def _declared(name: str) -> ast.FunctionDef:
tree = ast.parse(inspect.getsource(hypervisor_module))
for node in ast.walk(tree):
if isinstance(node, ast.FunctionDef) and node.name == name:
return node
raise AssertionError(f"HypervisorDriver does not declare {name}()")
class TestClusterListing:
def test_is_declared(self):
fn = _declared("get_clusters")
assert ast.unparse(fn.returns) == "list[ClusterDict]"
def test_absent_until_a_driver_implements_it(self):
"""netOrk probes capabilities with hasattr; a hypervisor that cannot
describe its cluster must not seem to."""
assert not hasattr(HypervisorDriver, "get_clusters")
class TestClusterShape:
def test_cluster(self):
assert get_type_hints(ClusterDict, include_extras=True) == {
"id": str | None,
"name": str | None,
"quorate": bool | None,
"nodes": list[ClusterNodeDict],
"storages": list[ClusterStorageDict],
"networks": list[ClusterNetworkDict],
"ha_resources": NotRequired[list[HAResourceDict] | None],
"backup_jobs": NotRequired[list[BackupJobDict] | None],
"backups": NotRequired[list[BackupDict] | None],
}
def test_cluster_keys_added_later_stay_optional(self):
"""A driver written before HA and backups were part of the report stays valid."""
assert ClusterDict.__required_keys__ == frozenset(
{"id", "name", "quorate", "nodes", "storages", "networks"}
)
def test_node(self):
hints = get_type_hints(ClusterNodeDict, include_extras=True)
assert hints == {
"name": str,
"status": str,
"local": bool,
"maintenance": bool | None,
"cpu_threads": int | None,
"cpu_cores": int | None,
"cpu_model": str | None,
"cpu_flags": list[str] | None,
"cpu_usage": float | None,
"memory_total": int | None,
"memory_used": int | None,
"votes": NotRequired[int | None],
}
assert "votes" not in ClusterNodeDict.__required_keys__
def test_storage(self):
assert get_type_hints(ClusterStorageDict) == {
"name": str,
"type": str,
"shared": bool,
"content": list[str],
"nodes": list[str],
"total": int | None,
"used": int | None,
}
def test_network(self):
hints = get_type_hints(ClusterNetworkDict, include_extras=True)
assert hints == {
"name": str,
"kind": str,
"vlan_aware": bool,
"fixed_vlan_tag": NotRequired[int | None],
"nodes": list[str],
}
assert ClusterNetworkDict.__required_keys__ == frozenset(
{"name", "kind", "vlan_aware", "nodes"}
)
class TestHAAndBackupShapes:
def test_ha_resource(self):
hints = get_type_hints(HAResourceDict, include_extras=True)
assert hints == {
"vmid": int,
"kind": str,
"requested": str,
"status": str | None,
"node": str | None,
"group": NotRequired[str | None],
"rules": list[str],
}
assert HAResourceDict.__required_keys__ == frozenset(
{"vmid", "kind", "requested", "status", "node", "rules"}
)
def test_backup_job(self):
assert get_type_hints(BackupJobDict) == {
"id": str,
"enabled": bool,
"schedule": str | None,
"storage": str | None,
"mode": str | None,
"all": bool,
"vmids": list[int],
"exclude": list[int],
"pool": str | None,
"node": str | None,
}
def test_backup(self):
assert get_type_hints(BackupDict) == {
"vmid": int,
"storage": str,
"time": int,
"size": int | None,
"count": int,
}
class TestCreateVmTakesANode:
def test_node_is_an_optional_keyword(self):
fn = _declared("create_vm_from_cloud_init")
kwonly = {
arg.arg: default
for arg, default in zip(fn.args.kwonlyargs, fn.args.kw_defaults, strict=True)
}
assert "node" in kwonly
default = kwonly["node"]
assert isinstance(default, ast.Constant) and default.value is None
+1 -2
View File
@@ -30,8 +30,7 @@ def test_a_missing_license_may_say_what_the_device_withholds_without_it():
# A Supermicro X10 with only SFT-OOB-LIC answers Redfish but keeps its drives and
# firmware inventory until DCMS is activated: partly licensed, not unlicensed.
hints = get_type_hints(LicenseDict)
# Python 3.10 keeps NotRequired[...] around the type; later versions strip it.
assert "list[str]" in str(hints["withholds"])
assert hints["withholds"] == list[str]
assert "withholds" not in LicenseDict.__required_keys__
+81
View File
@@ -14,6 +14,7 @@ from napalm_device_types import (
EnsurePackageMixin,
FirewallDriver,
HypervisorDriver,
LldpAgentMixin,
ManagementControllerDriver,
MediaDriver,
OSDriver,
@@ -169,11 +170,91 @@ class TestEnsurePackage:
assert "no candidate" in result["output"]
class _Lldp(LldpAgentMixin):
"""The generic steps over ensure_package and manage_service; the driver's configure."""
def __init__(self, refuse: str = ""):
self.calls: list = []
self.refuse = refuse
def ensure_package(self, name):
self.calls.append(("ensure_package", name))
ok = self.refuse != "install"
return {"success": ok, "steps": [], "output": "installed" if ok else "E: no candidate"}
def _write_lldp_agent_config(self, interfaces):
self.calls.append(("configure", interfaces))
return {"ok": self.refuse != "configure", "output": "interfaces set"}
def manage_service(self, name, action):
self.calls.append((action, name))
ok = self.refuse != action
return {"success": ok, "output": "" if ok else f"cannot {action}"}
class TestLldpAgent:
def test_install_configure_enable_and_restart_on_the_caller_s_interfaces(self):
agent = _Lldp()
result = agent.configure_lldp_agent(["br-lan"])
assert result["success"] is True
assert agent.calls == [
("ensure_package", "lldpd"),
("configure", ["br-lan"]),
("enable", "lldpd"),
("restart", "lldpd"),
]
assert [s["step"] for s in result["steps"]] == ["install", "configure", "start"]
def test_a_failed_install_configures_nothing(self):
agent = _Lldp(refuse="install")
result = agent.configure_lldp_agent(["br-lan"])
assert result["success"] is False
assert agent.calls == [("ensure_package", "lldpd")]
assert "no candidate" in result["output"]
def test_an_agent_that_will_not_start_at_boot_is_not_restarted(self):
agent = _Lldp(refuse="enable")
result = agent.configure_lldp_agent(["br-lan"])
assert result["success"] is False
assert ("restart", "lldpd") not in agent.calls
assert "cannot enable" in result["output"]
def test_a_failed_restart_fails_the_fix(self):
result = _Lldp(refuse="restart").configure_lldp_agent(["br-lan"])
assert result["success"] is False
assert result["steps"][-1]["step"] == "start"
def test_the_package_and_service_names_are_the_driver_s(self):
agent = _Lldp()
agent.LLDP_AGENT_PACKAGE = "lldpd-full"
agent.LLDP_AGENT_SERVICE = "lldpd.service"
agent.configure_lldp_agent(["eth0"])
assert ("ensure_package", "lldpd-full") in agent.calls
assert ("restart", "lldpd.service") in agent.calls
def test_no_interface_is_refused_before_anything_changes(self):
agent = _Lldp()
with pytest.raises(ValueError):
agent.configure_lldp_agent([])
assert agent.calls == []
@pytest.mark.parametrize("base", ROLE_BASES, ids=lambda b: b.__name__)
@pytest.mark.parametrize(
"method",
[
"configure_snmp_agent",
"configure_lldp_agent",
"ensure_package",
"configure_package_proxy",
"grant_container_engine_access",
+27
View File
@@ -0,0 +1,27 @@
"""shutdown_host: the contract for shutting the device itself down cleanly.
netOrk's graph workflows switch a hypervisor off when it has nothing to do
(NetOrk/netork#1636): through its own API, never by pulling the power. Like
``reboot_host`` it is declared, not implemented, so ``hasattr`` tells netOrk
whether a driver can.
"""
from __future__ import annotations
import inspect
from napalm_device_types import DeviceTypeDriver, HostShutdownMixin
def test_every_device_type_driver_carries_the_declaration():
assert issubclass(DeviceTypeDriver, HostShutdownMixin)
def test_declared_not_implemented():
assert not hasattr(DeviceTypeDriver, "shutdown_host")
def test_declaration_documents_the_contract():
source = inspect.getsource(HostShutdownMixin)
assert "def shutdown_host(self) -> None" in source
assert "RuntimeError" in source
+692
View File
@@ -0,0 +1,692 @@
"""Personal accounts on a host: reading them, adding one, locking one again.
The host is a fake that keeps accounts, groups, sudo rules and key files in
memory and answers the fixed commands the mixin and a driver send. A driver's
own commands (create, open, lock, leave a group) are stand-ins here; the real
ones are tested in the drivers.
"""
from __future__ import annotations
import shlex
from dataclasses import dataclass, field
import pytest
from napalm_device_types.channel import CommandResult
from napalm_device_types.local_accounts import (
LocalAccountsMixin,
authorized_keys_paths,
sudoers_rule,
sudoers_rule_path,
)
from napalm_device_types.models import FixStepOutcomeDict
ALICE_KEY = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAlice alice@laptop"
OWN_KEY = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQOwn own@desktop"
HASH = "$6$salt$" + "x" * 86
#: Shell syntax a command for root must not hold: each is one fixed command.
SHELL_SYNTAX = (";", "&&", "||", "|", ">", "<", "$(", "`")
@dataclass
class _Account:
uid: int
home: str
shell: str = "/bin/sh"
groups: list[str] = field(default_factory=list)
locked: bool = False
password: str | None = None
class _Host:
"""What the fake host holds, and the commands it ran."""
def __init__(self) -> None:
self.login = "netadmin"
self.accounts: dict[str, _Account] = {
"netadmin": _Account(1000, "/home/netadmin", groups=["netadmin", "wheel"]),
"daemon": _Account(1, "/usr/sbin"),
}
#: Path -> content, for the sudoers drop-ins and the key files.
self.files: dict[str, str] = {}
#: Users a rule outside the drop-in directory lets run everything.
self.other_rules: set[str] = set()
#: Whether sudo reads the drop-in directory at all.
self.includes_dropins = True
#: Commands that fail, by their first words ("as <command>" for one run as
#: the person); and commands that succeed without doing anything.
self.failing: set[str] = set()
self.ignored: set[str] = set()
self.ran: list[tuple[str, bool]] = []
self.staged = 0
def is_admin(self, name: str) -> bool:
rule = self.files.get(sudoers_rule_path(_Driver.SUDOERS_DIR, name))
return name in self.other_rules or (self.includes_dropins and rule is not None)
def run(self, command: str, privileged: bool, stdin: bytes | None) -> CommandResult:
self.ran.append((command, privileged))
if stdin is not None:
assert not privileged, "a privileged command takes no stdin"
return self._stage(command, stdin)
words = shlex.split(command)
if " ".join(words[:2]) in self.failing or words[0] in self.failing:
return CommandResult("", f"{words[0]}: failed", 1)
if words[:3] == ["sudo", "-n", "-u"]:
assert privileged, "only root changes to another user"
return self._as_user(words[3], words[4:])
handler = getattr(self, "_" + words[0].replace("-", "_"), None)
assert handler is not None, f"unexpected command: {command}"
return handler(words[1:], privileged)
# -- staging, as the login user ---------------------------------------------
def _stage(self, command: str, stdin: bytes) -> CommandResult:
assert "mktemp" in command
if "mktemp" in self.failing:
return CommandResult("", "mktemp: No space left on device", 1)
self.staged += 1
path = f"/tmp/tmp.{self.staged}"
self.files[path] = stdin.decode()
return CommandResult(path, "", 0)
def _rm(self, args: list[str], privileged: bool) -> CommandResult:
for path in args[args.index("--") + 1 :]:
if path.startswith("/tmp/"):
assert not privileged, "the staged file is the login user's"
self.files.pop(path, None)
return CommandResult("", "", 0)
# -- the account database -------------------------------------------------------
def _getent(self, args: list[str], privileged: bool) -> CommandResult:
assert args[0] == "passwd" and not privileged
a = self.accounts.get(args[1])
if a is None:
return CommandResult("", "", 2)
return CommandResult(f"{args[1]}:*:{a.uid}:{a.uid}::{a.home}:{a.shell}\n", "", 0)
def _id(self, args: list[str], privileged: bool) -> CommandResult:
if args == ["-un"]:
return CommandResult(f"{self.login}\n", "", 0)
assert args[0] == "-Gn" and not privileged
a = self.accounts.get(args[1])
if a is None:
return CommandResult("", f"id: {args[1]}: no such user", 1)
return CommandResult(" ".join(a.groups) + "\n", "", 0)
def _sudo(self, args: list[str], privileged: bool) -> CommandResult:
assert args[:2] == ["-l", "-U"] and privileged
name = args[2]
if self.is_admin(name):
out = f"User {name} may run the following commands on host:\n (ALL) ALL\n"
else:
out = f"User {name} is not allowed to run sudo on host.\n"
return CommandResult(out, "", 0)
def _visudo(self, args: list[str], privileged: bool) -> CommandResult:
assert args[0] == "-cf" and privileged
content = self.files[args[1]]
ok = content.endswith(" ALL=(ALL) ALL\n") and "\n" not in content[:-1]
return CommandResult("parsed OK" if ok else "syntax error", "", 0 if ok else 1)
def _install(self, args: list[str], privileged: bool) -> CommandResult:
assert privileged
source, target = args[args.index("--") + 1 :]
assert target.startswith(_Driver.SUDOERS_DIR + "/"), "root installs only sudo rules"
assert args[:6] == ["-m", "440", "-o", "root", "-g", _Driver.ROOT_GROUP]
self.files[target] = self.files[source]
return CommandResult("", "", 0)
def _test(self, args: list[str], privileged: bool) -> CommandResult:
assert args[0] == "-e"
return CommandResult("", "", 0 if args[1] in self.files else 1)
# -- the person's own files, as that person ------------------------------------
def _as_user(self, name: str, words: list[str]) -> CommandResult:
a = self.accounts[name]
if a.locked:
return CommandResult("", "sudo: account expired", 1)
rest = words[words.index("--") + 1 :] if "--" in words else words[1:]
if f"as {words[0]}" in self.failing:
return CommandResult("", f"{words[0]}: Permission denied", 1)
if f"as {words[0]}" in self.ignored:
return CommandResult("", "", 0)
assert all(p.startswith(a.home + "/") or p.startswith("/tmp/") for p in rest)
if words[0] == "cat":
return CommandResult(self.files.get(rest[0], ""), "", 0)
if words[0] == "mkdir":
return CommandResult("", "", 0)
if words[0] == "install":
assert words[1:3] == ["-m", "600"]
self.files[rest[1]] = self.files[rest[0]]
return CommandResult("", "", 0)
if words[0] == "rm":
for path in rest:
self.files.pop(path, None)
return CommandResult("", "", 0)
raise AssertionError(f"unexpected command as {name}: {words}")
# -- the driver's own commands (stand-ins) -------------------------------------
def _mkuser(self, args: list[str], privileged: bool) -> CommandResult:
assert privileged
name, password = args[0], (args[1] if len(args) > 1 else None)
uid = 1001 + len(self.accounts)
self.accounts[name] = _Account(uid, f"/home/{name}", groups=[name], password=password)
return CommandResult("", "", 0)
def _openuser(self, args: list[str], privileged: bool) -> CommandResult:
assert privileged
a = self.accounts[args[0]]
a.locked = False
if len(args) > 1:
a.password = args[1]
return CommandResult("", "", 0)
def _lockuser(self, args: list[str], privileged: bool) -> CommandResult:
assert privileged
self.accounts[args[0]].locked = "lockuser" not in self.ignored
return CommandResult("", "", 0)
def _islocked(self, args: list[str], privileged: bool) -> CommandResult:
assert privileged
return CommandResult("yes\n" if self.accounts[args[0]].locked else "no\n", "", 0)
def _leavegroup(self, args: list[str], privileged: bool) -> CommandResult:
assert privileged
self.accounts[args[0]].groups.remove(args[1])
return CommandResult("", "", 0)
class _Driver(LocalAccountsMixin):
ACCOUNT_SHELL = "/bin/sh"
ACCOUNT_ADMIN_GROUPS = ("wheel", "sudo")
SUDOERS_DIR = "/etc/sudoers.d"
ROOT_GROUP = "wheel"
def __init__(self, host: _Host) -> None:
self.host = host
def run_command(
self,
command: str,
*,
privileged: bool = False,
timeout: float = 60,
stdin: bytes | None = None,
) -> CommandResult:
return self.host.run(command, privileged, stdin)
def _account_create(self, name: str, password_hash: str | None) -> list[list[str]]:
return [["mkuser", name, *([password_hash] if password_hash else [])]]
def _account_open(
self, name: str, password_hash: str | None, *, locked: bool
) -> list[list[str]]:
return [["openuser", name, *([password_hash] if password_hash else [])]]
def _account_lock(self, name: str, *, locked: bool) -> list[list[str]]:
return [] if locked else [["lockuser", name]]
def _account_locked(self, name: str, passwd: list[str]) -> bool:
result = self.run_command(shlex.join(["islocked", name]), privileged=True)
return result.stdout.strip() == "yes"
def _account_leave_group(self, name: str, group: str, groups: list[str]) -> list[str]:
return ["leavegroup", name, group]
@pytest.fixture
def host() -> _Host:
return _Host()
@pytest.fixture
def driver(host: _Host) -> _Driver:
return _Driver(host)
def _root_commands(host: _Host) -> list[str]:
return [c for c, privileged in host.ran if privileged]
def _add_alice(host: _Host, *, locked: bool = False, keys: str = "") -> None:
host.accounts["alice"] = _Account(
1001, "/home/alice", groups=["alice", "wheel"], locked=locked, password=HASH
)
if keys:
host.files["/home/alice/.ssh/authorized_keys"] = keys
# -- the helpers ---------------------------------------------------------------------
def test_the_sudo_rule_names_the_account_and_asks_for_its_password() -> None:
assert sudoers_rule("alice") == "alice ALL=(ALL) ALL\n"
def test_the_rule_file_has_no_dot_sudo_would_skip() -> None:
# sudo ignores a file in an included directory whose name holds a dot.
path = sudoers_rule_path("/etc/sudoers.d", "a.lice")
assert path.startswith("/etc/sudoers.d/")
assert "." not in path.rsplit("/", 1)[1]
assert sudoers_rule_path("/etc/sudoers.d", "a.lice") != sudoers_rule_path(
"/etc/sudoers.d", "a_lice"
)
def test_both_key_files_sshd_reads_by_default() -> None:
assert authorized_keys_paths("/home/alice") == (
"/home/alice/.ssh/authorized_keys",
"/home/alice/.ssh/authorized_keys2",
)
# -- reading --------------------------------------------------------------------------
def test_an_account_is_read_with_its_groups_lock_sudo_and_keys(
host: _Host, driver: _Driver
) -> None:
_add_alice(host, keys=ALICE_KEY + "\n" + OWN_KEY + "\n")
host.other_rules.add("alice")
[account] = driver.get_local_accounts(["alice"])
assert account == {
"name": "alice",
"uid": 1001,
"home": "/home/alice",
"shell": "/bin/sh",
"groups": ["alice", "wheel"],
"locked": False,
"admin": True,
"authorized_keys": [ALICE_KEY, OWN_KEY],
}
def test_an_account_that_is_not_there_is_left_out(driver: _Driver) -> None:
assert driver.get_local_accounts(["bob"]) == []
def test_a_locked_accounts_keys_that_cannot_be_read_are_unknown(
host: _Host, driver: _Driver
) -> None:
_add_alice(host, locked=True, keys=ALICE_KEY + "\n")
[account] = driver.get_local_accounts(["alice"])
assert account["locked"] is True
assert account["authorized_keys"] is None
def test_an_account_without_a_key_file_has_no_keys(host: _Host, driver: _Driver) -> None:
_add_alice(host)
[account] = driver.get_local_accounts(["alice"])
assert account["authorized_keys"] == []
def test_a_failing_read_raises_rather_than_report_nothing(host: _Host, driver: _Driver) -> None:
_add_alice(host)
host.failing.add("sudo -l")
with pytest.raises(RuntimeError, match="alice"):
driver.get_local_accounts(["alice"])
@pytest.mark.parametrize("name", ["", "Alice", "1alice", "al ice", "a;b", "-alice", "a" * 33])
def test_a_name_no_guest_takes_is_refused(driver: _Driver, name: str) -> None:
with pytest.raises(ValueError, match="account name"):
driver.get_local_accounts([name])
# -- adding ---------------------------------------------------------------------------
def test_a_new_account_gets_the_key_and_the_sudo_rule(host: _Host, driver: _Driver) -> None:
result = driver.add_local_account("alice", ALICE_KEY, password_hash=HASH, admin=True)
assert result["success"], result["output"]
assert [s["step"] for s in result["steps"]] == ["account", "sudo", "keys"]
assert host.accounts["alice"].password == HASH
assert host.files["/home/alice/.ssh/authorized_keys"] == ALICE_KEY + "\n"
assert host.files["/etc/sudoers.d/80-account-alice"] == "alice ALL=(ALL) ALL\n"
account = result["account"]
assert account is not None
assert account["admin"] is True and account["authorized_keys"] == [ALICE_KEY]
def test_every_command_root_runs_is_one_fixed_command(host: _Host, driver: _Driver) -> None:
driver.add_local_account("alice", ALICE_KEY, password_hash=HASH, admin=True)
driver.lock_local_account("alice")
for command in _root_commands(host):
assert not any(s in command for s in SHELL_SYNTAX), command
def test_the_staged_files_are_removed_again(host: _Host, driver: _Driver) -> None:
driver.add_local_account("alice", ALICE_KEY, password_hash=HASH, admin=True)
assert host.staged == 2
assert not [p for p in host.files if p.startswith("/tmp/")]
def test_an_existing_account_keeps_its_own_keys(host: _Host, driver: _Driver) -> None:
_add_alice(host, keys=OWN_KEY + "\n")
result = driver.add_local_account("alice", ALICE_KEY)
assert result["success"], result["output"]
assert host.files["/home/alice/.ssh/authorized_keys"] == f"{OWN_KEY}\n{ALICE_KEY}\n"
def test_a_line_without_a_key_is_kept_and_not_taken_for_one(host: _Host, driver: _Driver) -> None:
_add_alice(host, keys="# the laptop\n")
result = driver.add_local_account("alice", ALICE_KEY)
assert result["success"], result["output"]
assert host.files["/home/alice/.ssh/authorized_keys"] == f"# the laptop\n{ALICE_KEY}\n"
def test_a_key_already_there_is_not_written_again(host: _Host, driver: _Driver) -> None:
# The same key with options in front and another comment is the same key.
_add_alice(host, keys=f'from="10.0.0.0/8" {ALICE_KEY.rsplit(" ", 1)[0]} old\n')
result = driver.add_local_account("alice", ALICE_KEY)
assert result["success"], result["output"]
assert host.staged == 0
assert "already" in result["steps"][-1]["output"]
def test_a_locked_account_is_opened_again(host: _Host, driver: _Driver) -> None:
_add_alice(host, locked=True)
result = driver.add_local_account("alice", ALICE_KEY, password_hash=HASH)
assert result["success"], result["output"]
assert host.accounts["alice"].locked is False
assert result["account"] is not None and result["account"]["locked"] is False
def test_without_sudo_the_rule_and_the_admin_groups_go(host: _Host, driver: _Driver) -> None:
_add_alice(host)
host.files["/etc/sudoers.d/80-account-alice"] = sudoers_rule("alice")
result = driver.add_local_account("alice", ALICE_KEY, admin=False)
assert result["success"], result["output"]
assert "/etc/sudoers.d/80-account-alice" not in host.files
assert host.accounts["alice"].groups == ["alice"]
def test_sudo_another_rule_still_grants_fails_the_step(host: _Host, driver: _Driver) -> None:
_add_alice(host)
host.other_rules.add("alice")
result = driver.add_local_account("alice", ALICE_KEY, admin=False)
assert not result["success"]
sudo = next(s for s in result["steps"] if s["step"] == "sudo")
assert not sudo["ok"] and "another rule" in sudo["output"]
# The key still went in: the steps after the account are independent.
assert host.files["/home/alice/.ssh/authorized_keys"] == ALICE_KEY + "\n"
def test_a_rule_sudo_does_not_read_fails_the_step(host: _Host, driver: _Driver) -> None:
host.includes_dropins = False
result = driver.add_local_account("alice", ALICE_KEY, password_hash=HASH, admin=True)
sudo = next(s for s in result["steps"] if s["step"] == "sudo")
assert not sudo["ok"] and "/etc/sudoers.d" in sudo["output"]
def test_a_rule_visudo_rejects_is_not_installed(host: _Host, driver: _Driver) -> None:
host.failing.add("visudo")
result = driver.add_local_account("alice", ALICE_KEY, password_hash=HASH, admin=True)
assert not result["success"]
assert "/etc/sudoers.d/80-account-alice" not in host.files
assert not [p for p in host.files if p.startswith("/tmp/")]
def test_a_failed_create_stops_before_sudo_and_keys(host: _Host, driver: _Driver) -> None:
host.failing.add("mkuser")
result = driver.add_local_account("alice", ALICE_KEY, admin=True)
assert not result["success"]
assert [s["step"] for s in result["steps"]] == ["account"]
assert result["account"] is None
@pytest.mark.parametrize(
"key",
["", "not a key", ALICE_KEY + "\n" + OWN_KEY, "ssh-ed25519", "ssh-ed25519 AAAA;rm"],
)
def test_a_public_key_that_is_none_is_refused(driver: _Driver, key: str) -> None:
with pytest.raises(ValueError, match="public key"):
driver.add_local_account("alice", key)
@pytest.mark.parametrize("password_hash", ["secret", "$6$salt$ab cd", "$2b$10$" + "x" * 53])
def test_a_password_hash_the_guest_does_not_take_is_refused(
driver: _Driver, password_hash: str
) -> None:
# This driver takes SHA-512 crypt: neither plain text nor bcrypt.
with pytest.raises(ValueError, match="password hash"):
driver.add_local_account("alice", ALICE_KEY, password_hash=password_hash)
def test_the_login_users_own_account_is_never_touched(host: _Host, driver: _Driver) -> None:
for change in (
lambda: driver.add_local_account("netadmin", ALICE_KEY),
lambda: driver.lock_local_account("netadmin"),
):
with pytest.raises(ValueError, match="logs in"):
change()
assert _root_commands(host) == []
def test_nothing_changes_when_the_login_user_cannot_be_told(host: _Host, driver: _Driver) -> None:
_add_alice(host)
host.login = ""
with pytest.raises(RuntimeError, match="logs in"):
driver.lock_local_account("alice")
assert _root_commands(host) == []
def test_a_system_account_is_never_touched(host: _Host, driver: _Driver) -> None:
with pytest.raises(ValueError, match="system account"):
driver.lock_local_account("daemon")
assert _root_commands(host) == []
# -- locking --------------------------------------------------------------------------
def test_locking_takes_keys_sudo_and_login_away(host: _Host, driver: _Driver) -> None:
_add_alice(host, keys=ALICE_KEY + "\n" + OWN_KEY + "\n")
host.files["/home/alice/.ssh/authorized_keys2"] = OWN_KEY + "\n"
host.files["/etc/sudoers.d/80-account-alice"] = sudoers_rule("alice")
result = driver.lock_local_account("alice")
assert result["success"], result["output"]
assert [s["step"] for s in result["steps"]] == ["keys", "lock", "sudo"]
a = host.accounts["alice"]
assert a.locked and a.groups == ["alice"] and a.password == HASH
assert not [p for p in host.files if p.startswith("/home/alice/")]
assert "/etc/sudoers.d/80-account-alice" not in host.files
account = result["account"]
assert account is not None and account["locked"] and not account["admin"]
def test_a_rule_left_elsewhere_is_named_but_harmless_once_locked(
host: _Host, driver: _Driver
) -> None:
_add_alice(host)
host.other_rules.add("alice")
result = driver.lock_local_account("alice")
assert result["success"], result["output"]
sudo = next(s for s in result["steps"] if s["step"] == "sudo")
assert "another rule" in sudo["output"] and "locked" in sudo["output"]
def test_the_lock_comes_even_when_the_keys_could_not_go(host: _Host, driver: _Driver) -> None:
_add_alice(host, locked=True, keys=ALICE_KEY + "\n")
result = driver.lock_local_account("alice")
assert not result["success"]
keys = next(s for s in result["steps"] if s["step"] == "keys")
assert not keys["ok"]
assert next(s for s in result["steps"] if s["step"] == "lock")["ok"]
assert host.accounts["alice"].locked
def test_locking_an_account_that_is_not_there_fails(driver: _Driver) -> None:
result = driver.lock_local_account("bob")
assert not result["success"]
assert "no account" in result["output"]
assert result["account"] is None
# -- what fails is said ---------------------------------------------------------------
@pytest.mark.parametrize("failing", ["getent", "id -Gn"])
def test_an_account_that_cannot_be_read_raises(host: _Host, driver: _Driver, failing: str) -> None:
_add_alice(host)
host.failing.add(failing)
with pytest.raises(RuntimeError, match="alice"):
driver.get_local_accounts(["alice"])
@pytest.mark.parametrize(
("failing", "step", "said"),
[
("rm -f", "sudo", "80-account-alice"),
("leavegroup", "sudo", "out of wheel"),
("as cat", "keys", "cannot read"),
("as install", "keys", "authorized_keys"),
("mktemp", "keys", "cannot stage"),
],
)
def test_a_step_that_fails_says_why(
host: _Host, driver: _Driver, failing: str, step: str, said: str
) -> None:
_add_alice(host, keys=OWN_KEY + "\n")
host.failing.add(failing)
result = driver.add_local_account("alice", ALICE_KEY)
failed = next(s for s in result["steps"] if s["step"] == step)
assert not failed["ok"] and said in failed["output"]
assert not result["success"]
def test_a_rule_that_cannot_be_installed_fails_the_step(host: _Host, driver: _Driver) -> None:
host.failing.add("install -m")
result = driver.add_local_account("alice", ALICE_KEY, password_hash=HASH, admin=True)
sudo = next(s for s in result["steps"] if s["step"] == "sudo")
assert not sudo["ok"] and "80-account-alice" in sudo["output"]
assert not [p for p in host.files if p.startswith("/tmp/")]
@pytest.mark.parametrize(
("failing", "ignored", "step", "said"),
[
({"lockuser"}, set(), "lock", "lockuser: failed"),
(set(), {"lockuser"}, "lock", "still not locked"),
({"as rm"}, set(), "keys", "cannot remove"),
(set(), {"as rm"}, "keys", "still has keys"),
],
)
def test_a_lock_that_did_not_take_says_so(
host: _Host, driver: _Driver, failing: set[str], ignored: set[str], step: str, said: str
) -> None:
_add_alice(host, keys=ALICE_KEY + "\n")
host.failing |= failing
host.ignored |= ignored
result = driver.lock_local_account("alice")
failed = next(s for s in result["steps"] if s["step"] == step)
assert not failed["ok"] and said in failed["output"]
assert not result["success"]
def test_an_account_that_cannot_be_read_back_is_said(host: _Host, driver: _Driver) -> None:
_add_alice(host)
host.failing.add("sudo -l")
result = driver.lock_local_account("alice")
assert result["account"] is None
assert "[read back] cannot ask sudo about alice" in result["output"]
# -- a key store of the driver's own ---------------------------------------------------
class _OwnKeyStore(_Driver):
"""A driver whose host keeps keys in a place of its own."""
def __init__(self, host: _Host) -> None:
super().__init__(host)
self.store: dict[str, list[str]] = {}
def _account_read_keys(self, name: str, home: str) -> list[str] | None:
return list(self.store.get(name, []))
def _account_add_key(self, name: str, home: str, public_key: str) -> FixStepOutcomeDict:
self.store.setdefault(name, []).append(f"{home} {public_key}")
return {"ok": True, "output": "stored"}
def _account_remove_keys(self, name: str, home: str) -> FixStepOutcomeDict:
self.store.pop(name, None)
return {"ok": True, "output": "dropped"}
def test_a_driver_can_keep_the_keys_elsewhere(host: _Host) -> None:
driver = _OwnKeyStore(host)
added = driver.add_local_account("alice", ALICE_KEY)
assert added["success"], added["output"]
assert driver.store == {"alice": [f"/home/alice {ALICE_KEY}"]}
assert added["account"] is not None
assert added["account"]["authorized_keys"] == [f"/home/alice {ALICE_KEY}"]
assert not [p for p in host.files if p.startswith("/home/")]
locked = driver.lock_local_account("alice")
assert locked["success"], locked["output"]
assert driver.store == {}
def test_a_key_file_the_host_adds_is_read_too(host: _Host) -> None:
class _ExtraFile(_Driver):
def _account_key_files(self, name: str, home: str) -> tuple[str, ...]:
return (*super()._account_key_files(name, home), f"/home/{name}/extra")
_add_alice(host, keys=OWN_KEY + "\n")
host.files["/home/alice/extra"] = ALICE_KEY + "\n"
[account] = _ExtraFile(host).get_local_accounts(["alice"])
assert account["authorized_keys"] == [OWN_KEY, ALICE_KEY]
+25
View File
@@ -0,0 +1,25 @@
"""A network target can say which physical NICs it leaves the host through.
A VM on a VLAN-aware bridge without a VLAN tag only gets an address when the
switch port behind the bridge carries an untagged network. To look that port up,
the caller needs the bridge's physical NICs: their LLDP neighbours name the
switch and the port (NetOrk/netork#1325). They come in groups, one per way out
of the bridge: a bond's slaves together, a single NIC alone. A bond has no
neighbour of its own, and its slaves are one link, so one slave's neighbour
speaks for the others when they report none.
"""
from __future__ import annotations
from typing import get_type_hints
from napalm_device_types.models import NetworkTargetDict
def test_uplinks_is_an_optional_list_of_nic_groups():
assert "uplinks" in NetworkTargetDict.__optional_keys__
assert get_type_hints(NetworkTargetDict)["uplinks"] == list[list[str]]
def test_contract_core_stays_required():
assert {"name", "kind", "vlan_aware"} <= NetworkTargetDict.__required_keys__
+301
View File
@@ -0,0 +1,301 @@
"""netOrk's own account on a host: created through the account it logs in with today.
The fake host of :mod:`tests.test_local_accounts`, taught the rule that lets
an account run everything as root without a password.
"""
from __future__ import annotations
import shlex
import pytest
from napalm_device_types.channel import CommandResult
from napalm_device_types.local_accounts import (
service_sudoers_rule,
service_sudoers_rule_path,
sudoers_rule_path,
)
from tests.test_local_accounts import (
OWN_KEY,
SHELL_SYNTAX,
_Account,
_Driver,
_Host,
_root_commands,
)
NETORK_KEY = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINetork netork@central"
class _ServiceHost(_Host):
"""The fake host, whose sudo also takes the rule without a password."""
def is_passwordless(self, name: str) -> bool:
rule = self.files.get(service_sudoers_rule_path(_Driver.SUDOERS_DIR, name))
return self.includes_dropins and rule is not None
def _sudo(self, args: list[str], privileged: bool) -> CommandResult:
name = args[2]
if not self.is_passwordless(name):
return super()._sudo(args, privileged)
out = f"User {name} may run the following commands on host:\n (ALL) NOPASSWD: ALL\n"
return CommandResult(out, "", 0)
def _visudo(self, args: list[str], privileged: bool) -> CommandResult:
content = self.files[args[1]]
if content.endswith(" ALL=(ALL) NOPASSWD: ALL\n") and "\n" not in content[:-1]:
return CommandResult("parsed OK", "", 0)
return super()._visudo(args, privileged)
@pytest.fixture
def host() -> _ServiceHost:
return _ServiceHost()
@pytest.fixture
def driver(host: _ServiceHost) -> _Driver:
return _Driver(host)
def test_the_rule_runs_everything_as_root_without_a_password() -> None:
assert service_sudoers_rule("netork") == "netork ALL=(ALL) NOPASSWD: ALL\n"
def test_the_rule_has_a_file_of_its_own_no_personal_rule_shares() -> None:
path = service_sudoers_rule_path("/etc/sudoers.d", "net.ork_1")
assert path == "/etc/sudoers.d/85-service-net_2eork__1"
assert path != sudoers_rule_path("/etc/sudoers.d", "net.ork_1")
def test_a_new_service_account_gets_the_key_and_sudo_without_a_password(
host: _ServiceHost, driver: _Driver
) -> None:
result = driver.add_service_account("netork", NETORK_KEY)
assert result["success"], result["output"]
assert [s["step"] for s in result["steps"]] == ["account", "sudo", "keys"]
assert host.accounts["netork"].password is None
assert host.files["/home/netork/.ssh/authorized_keys"] == NETORK_KEY + "\n"
assert host.is_passwordless("netork")
account = result["account"]
assert account is not None and account["admin"]
def test_every_command_root_runs_is_one_fixed_command(host: _ServiceHost, driver: _Driver) -> None:
driver.add_service_account("netork", NETORK_KEY)
for command in _root_commands(host):
assert not any(s in command for s in SHELL_SYNTAX), command
def test_an_existing_account_keeps_its_keys_and_gets_the_rule(
host: _ServiceHost, driver: _Driver
) -> None:
host.accounts["netork"] = _Account(1001, "/home/netork", groups=["netork"])
host.files["/home/netork/.ssh/authorized_keys"] = OWN_KEY + "\n"
result = driver.add_service_account("netork", NETORK_KEY)
assert result["success"], result["output"]
assert host.files["/home/netork/.ssh/authorized_keys"] == f"{OWN_KEY}\n{NETORK_KEY}\n"
assert host.is_passwordless("netork")
def test_an_existing_system_account_may_become_netorks(host: _ServiceHost, driver: _Driver) -> None:
host.accounts["netork"] = _Account(998, "/var/lib/netork", groups=["netork"])
result = driver.add_service_account("netork", NETORK_KEY)
assert result["success"], result["output"]
def test_a_locked_account_is_opened_again(host: _ServiceHost, driver: _Driver) -> None:
host.accounts["netork"] = _Account(1001, "/home/netork", groups=["netork"], locked=True)
result = driver.add_service_account("netork", NETORK_KEY)
assert result["success"], result["output"]
assert not host.accounts["netork"].locked
def test_the_account_netork_logs_in_with_already_only_gets_what_it_lacks(
host: _ServiceHost, driver: _Driver
) -> None:
host.login = "netork"
host.accounts["netork"] = _Account(1001, "/home/netork", groups=["netork"])
host.files["/home/netork/.ssh/authorized_keys"] = NETORK_KEY + "\n"
result = driver.add_service_account("netork", NETORK_KEY)
assert result["success"], result["output"]
assert not any(c.startswith("openuser") for c in _root_commands(host))
assert "already" in result["output"]
def test_root_is_never_netorks_account(host: _ServiceHost, driver: _Driver) -> None:
host.accounts["toor"] = _Account(0, "/root", groups=["wheel"])
with pytest.raises(ValueError, match="root"):
driver.add_service_account("toor", NETORK_KEY)
assert not _root_commands(host)
@pytest.mark.parametrize("name", ["Netork", "net ork", "-n", ""])
def test_a_name_no_host_takes_is_refused(driver: _Driver, name: str) -> None:
with pytest.raises(ValueError):
driver.add_service_account(name, NETORK_KEY)
def test_a_key_that_is_none_is_refused(driver: _Driver) -> None:
with pytest.raises(ValueError):
driver.add_service_account("netork", "not a key")
def test_a_rule_sudo_does_not_read_fails_the_step(host: _ServiceHost, driver: _Driver) -> None:
host.includes_dropins = False
result = driver.add_service_account("netork", NETORK_KEY)
assert not result["success"]
sudo = next(s for s in result["steps"] if s["step"] == "sudo")
assert not sudo["ok"] and "does not read it" in sudo["output"]
def test_a_rule_with_a_password_is_not_enough(host: _ServiceHost, driver: _Driver) -> None:
host.other_rules.add("netork")
host.includes_dropins = False
result = driver.add_service_account("netork", NETORK_KEY)
assert not result["success"]
def test_a_failed_create_stops_before_sudo_and_keys(host: _ServiceHost, driver: _Driver) -> None:
host.failing.add("mkuser")
result = driver.add_service_account("netork", NETORK_KEY)
assert not result["success"]
assert [s["step"] for s in result["steps"]] == ["account"]
assert result["account"] is None
def test_the_staged_files_are_removed_again(host: _ServiceHost, driver: _Driver) -> None:
driver.add_service_account("netork", NETORK_KEY)
assert not [p for p in host.files if p.startswith("/tmp/")]
def test_a_personal_account_of_the_same_name_keeps_its_own_rule_apart(
host: _ServiceHost, driver: _Driver
) -> None:
driver.add_service_account("netork", NETORK_KEY)
rules = [c for c in _root_commands(host) if c.startswith("install")]
assert [shlex.split(c)[-1] for c in rules] == [
service_sudoers_rule_path(_Driver.SUDOERS_DIR, "netork")
]
# -- a fixed uid (infrastructure BAUSTANDARD A) ------------------------------------------
class _UidDriver(_Driver):
def _account_create_with_uid(self, name: str, uid: int) -> list[list[str]]:
return [["mkuseruid", name, str(uid)]]
def _mkuseruid(host: _ServiceHost):
def handler(args: list[str], privileged: bool) -> CommandResult:
assert privileged
name, uid = args[0], int(args[1])
host.accounts[name] = _Account(uid, f"/home/{name}", groups=[name])
return CommandResult("", "", 0)
return handler
def _getent_uid(host: _ServiceHost, original):
def handler(args: list[str], privileged: bool) -> CommandResult:
if args[1].isdigit():
owner = next((n for n, a in host.accounts.items() if a.uid == int(args[1])), None)
if owner is None:
return CommandResult("", "", 2)
a = host.accounts[owner]
return CommandResult(f"{owner}:*:{a.uid}:{a.uid}::{a.home}:{a.shell}\n", "", 0)
return original(args, privileged)
return handler
@pytest.fixture
def uid_driver(host: _ServiceHost) -> _UidDriver:
host._mkuseruid = _mkuseruid(host) # type: ignore[attr-defined]
host._getent = _getent_uid(host, host._getent) # type: ignore[method-assign]
return _UidDriver(host)
def test_a_new_account_gets_the_uid_asked_for(host: _ServiceHost, uid_driver: _UidDriver) -> None:
result = uid_driver.add_service_account("netork", NETORK_KEY, uid=900)
assert result["success"], result["output"]
assert host.accounts["netork"].uid == 900
account = result["account"]
assert account is not None and account["uid"] == 900
def test_a_uid_another_account_has_stops_before_anything_is_made(
host: _ServiceHost, uid_driver: _UidDriver
) -> None:
host.accounts["fwupd-refresh"] = _Account(900, "/run/fwupd")
result = uid_driver.add_service_account("netork", NETORK_KEY, uid=900)
assert not result["success"]
assert "fwupd-refresh" in result["output"]
assert "netork" not in host.accounts
def test_an_existing_account_keeps_its_uid_and_says_so(
host: _ServiceHost, uid_driver: _UidDriver
) -> None:
host.accounts["netork"] = _Account(1001, "/home/netork", groups=["netork"])
result = uid_driver.add_service_account("netork", NETORK_KEY, uid=900)
assert result["success"], result["output"]
assert host.accounts["netork"].uid == 1001
assert "uid 1001" in result["output"]
def test_a_uid_root_has_is_refused(uid_driver: _UidDriver) -> None:
with pytest.raises(ValueError):
uid_driver.add_service_account("netork", NETORK_KEY, uid=0)
def test_a_driver_that_cannot_set_a_uid_says_so(host: _ServiceHost, driver: _Driver) -> None:
result = driver.add_service_account("netork", NETORK_KEY, uid=900)
assert not result["success"]
assert "uid" in result["output"]
# -- another service's account of the same name (bw-01: the satellite's) ------------------
@pytest.mark.parametrize(
"shell", ["/usr/sbin/nologin", "/sbin/nologin", "/bin/false", "/usr/bin/false"]
)
def test_an_account_that_cannot_log_in_is_another_services_and_left_alone(
host: _ServiceHost, driver: _Driver, shell: str
) -> None:
host.accounts["netork"] = _Account(
999, "/home/netork", shell=shell, groups=["netork", "docker"]
)
with pytest.raises(ValueError, match="cannot log in"):
driver.add_service_account("netork", NETORK_KEY)
assert not _root_commands(host)
assert service_sudoers_rule_path(_Driver.SUDOERS_DIR, "netork") not in host.files
+19 -1
View File
@@ -22,7 +22,7 @@ from napalm_device_types import (
def test_a_profile_says_where_each_value_lives():
assert set(get_type_hints(SnmpHealthProfile)) == {"uptime", "cpu", "memory", "interfaces"}
assert set(get_type_hints(SnmpValueSpec)) == {"oids", "table", "fold", "pattern"}
assert set(get_type_hints(SnmpValueSpec)) == {"oids", "table", "fold", "pattern", "unknown"}
assert set(get_type_hints(SnmpMemorySpec)) == {"total", "used", "free", "percent", "unit"}
assert set(get_type_hints(SnmpInterfacesSpec)) == {"skip", "tx_errors_are_drops"}
@@ -42,6 +42,24 @@ def test_a_driver_declares_its_profile_as_a_class_attribute():
assert Driver.SNMP_HEALTH_PROFILE["cpu"]["oids"][0] == "1.3.6.1.4.1.9.1"
def test_a_value_the_agent_gets_wrong_is_declared_unknown():
"""Neither the standard reading nor a default: the application reports none."""
assert get_type_hints(SnmpValueSpec)["unknown"] is bool
class Driver:
SNMP_HEALTH_PROFILE: ClassVar[SnmpHealthProfile] = {"cpu": {"unknown": True}}
assert Driver.SNMP_HEALTH_PROFILE["cpu"].get("unknown") is True
def test_the_shapes_live_in_their_own_module_and_models_still_exports_them():
from napalm_device_types import models, snmp_health_profile
for name in ("SnmpHealthProfile", "SnmpValueSpec", "SnmpMemorySpec", "SnmpInterfacesSpec"):
assert getattr(models, name) is getattr(snmp_health_profile, name), name
assert getattr(snmp_health_profile, name).__module__ == snmp_health_profile.__name__
def test_the_collector_in_the_base_package_warns_that_it_is_deprecated():
async def get(oid):
return None
+250
View File
@@ -0,0 +1,250 @@
"""Exempting the managing client from sshd's per-source penalties (NetOrk/netork#1800).
OpenSSH 9.8 penalises a source whose connections do not authenticate and then
drops every connection from it. A security scanner's probes earn the managing
client exactly that, and its next session -- a poll, an Ansible run -- is
turned away. ``exempt_from_ssh_penalties`` puts the address the host sees the
client at into ``PerSourcePenaltyExemptList``, where that directive exists.
The real outputs this is held to were taken from Debian 13 (OpenSSH 10.0p2) and
Debian 12 (OpenSSH 9.2p1, which knows no penalties and rejects the directive).
"""
from __future__ import annotations
import re
import pytest
from napalm_device_types.channel import CommandResult
from napalm_device_types.ssh_server import (
DROPIN_DIR,
MAIN_CONFIG,
PENALTY_DROPIN_PATH,
OpenSshServerMixin,
client_address,
is_exempt,
parse_penalty_exemptions,
)
#: ``sshd -T | grep -i penalt`` on Debian 13, OpenSSH 10.0p2, as installed.
DEBIAN_13 = (
"persourcepenaltyexemptlist none\n"
"persourcepenalties crash:90 authfail:5 noauth:1 grace-exceeded:10 refuseconnection:10"
" max:600 min:15 max-sources4:65536 max-sources6:65536 overflow:permissive"
" overflow6:permissive\n"
)
# ── reading ──────────────────────────────────────────────────────────────
def test_a_new_sshd_has_penalties_and_no_exemption():
assert parse_penalty_exemptions(DEBIAN_13) == []
def test_the_exemptions_sshd_uses_are_read():
text = DEBIAN_13.replace("none", "10.40.100.10,192.0.2.0/24")
assert parse_penalty_exemptions(text) == ["10.40.100.10", "192.0.2.0/24"]
def test_an_sshd_before_9_8_has_no_penalties():
"""It prints neither line -- and refuses the directive, so nothing may be written."""
assert parse_penalty_exemptions("port 22\nmacs hmac-sha2-256\n") is None
@pytest.mark.parametrize(
("value", "address"),
[
("10.40.100.10 41866 10.40.100.118 22", "10.40.100.10"),
("fd00::10 41866 fd00::118 22", "fd00::10"),
("", None),
("not-an-address 1 2 3", None),
("10.40.100.10 41866", None),
],
)
def test_the_client_address_comes_from_ssh_connection(value, address):
assert client_address(value) == address
def test_an_address_inside_an_exempt_network_is_exempt():
assert is_exempt("10.40.100.10", ["192.0.2.0/24", "10.40.100.0/24"])
assert not is_exempt("10.40.100.10", ["10.40.100.11", "bogus"])
# ── the mixin, against a fake host ───────────────────────────────────────
class PenaltyHost(OpenSshServerMixin):
"""An sshd that reads the main config and the drop-in directory, first value wins."""
def __init__(
self,
*,
main: str = f"Include {DROPIN_DIR}/*.conf\nPort 22\n",
penalties: bool = True,
client: str = "10.40.100.10 41866 10.40.100.118 22",
) -> None:
self.files: dict[str, str] = {MAIN_CONFIG: main}
self.penalties = penalties
self.client = client
self.syntax_ok = True
self.reloads = 0
self.ran: list[str] = []
def _config_lines(self) -> list[str]:
lines: list[str] = []
for line in self.files[MAIN_CONFIG].splitlines():
if re.match(r"\s*include\s", line, re.I):
for path in sorted(p for p in self.files if p.startswith(DROPIN_DIR + "/")):
lines += self.files[path].splitlines()
elif re.match(r"\s*match\s", line, re.I):
break
else:
lines.append(line)
return lines
def exempt(self) -> str:
for line in self._config_lines():
parts = line.split()
if len(parts) == 2 and parts[0].lower() == "persourcepenaltyexemptlist":
return parts[1]
return "none"
def run_command(self, command, *, privileged=False, timeout=60, stdin=None):
self.ran.append(command)
if "SSH_CONNECTION" in command:
assert not privileged, "it is the login session's, not sudo's"
return CommandResult(self.client, "", 0)
if "mktemp" in command:
path = f"/tmp/netork-ssh.{len(self.files)}"
self.files[path] = (stdin or b"").decode()
return CommandResult(path, "", 0)
assert privileged, command
m = re.search(r"install -m \"\$m\" -- '?([^' ]+)'? '?([^' ]+)'?", command)
if m:
target = re.search(r"mv -f -- '?[^' ]+'? '?([^' ;]+)'?", command).group(1)
self.files[target] = self.files.pop(m.group(1))
return CommandResult("", "", 0)
if "sshd -T" in command:
if not self.penalties:
return CommandResult("port 22\n", "", 0)
return CommandResult(DEBIAN_13.replace("none", self.exempt()), "", 0)
if "sshd -t" in command:
return CommandResult(
"", "" if self.syntax_ok else "bad line 1", 0 if self.syntax_ok else 255
)
m = re.match(r"cat -- '?([^' ]+)'?$", command)
if m:
path = m.group(1)
if path in self.files:
return CommandResult(self.files[path], "", 0)
return CommandResult("", "No such file", 1)
m = re.match(r"rm -f -- '?([^' ]+)'?$", command)
if m:
self.files.pop(m.group(1), None)
return CommandResult("", "", 0)
if "reload" in command or "HUP" in command:
self.reloads += 1
return CommandResult("", "", 0)
if command.startswith("mkdir"):
return CommandResult("", "", 0)
raise AssertionError(f"unexpected command: {command}")
def test_the_client_address_is_exempted_in_a_dropin():
host = PenaltyHost()
lines: list[str] = []
result = host.exempt_from_ssh_penalties(log=lines.append)
assert result.applied and not result.rolled_back
assert result.address == "10.40.100.10"
assert result.path == PENALTY_DROPIN_PATH
assert host.exempt() == "10.40.100.10"
assert host.reloads == 1
assert lines
def test_the_hosts_own_exemptions_stay():
host = PenaltyHost(
main=f"Include {DROPIN_DIR}/*.conf\nPerSourcePenaltyExemptList 192.0.2.0/24\n"
)
result = host.exempt_from_ssh_penalties()
assert result.applied
assert host.exempt() == "192.0.2.0/24,10.40.100.10"
def test_nothing_is_written_to_an_sshd_without_penalties():
"""OpenSSH before 9.8 does not start with the directive in its config."""
host = PenaltyHost(penalties=False)
result = host.exempt_from_ssh_penalties()
assert not result.applied and not result.rolled_back
assert PENALTY_DROPIN_PATH not in host.files
assert host.reloads == 0
assert "9.8" in result.message
def test_nothing_changes_when_the_address_is_already_exempt():
host = PenaltyHost(
main=f"Include {DROPIN_DIR}/*.conf\nPerSourcePenaltyExemptList 10.40.100.0/24\n"
)
result = host.exempt_from_ssh_penalties()
assert not result.applied
assert PENALTY_DROPIN_PATH not in host.files
assert "already" in result.message
def test_an_sshd_config_without_the_include_is_left_alone():
host = PenaltyHost(main="Port 22\n")
result = host.exempt_from_ssh_penalties()
assert not result.applied
assert host.files[MAIN_CONFIG] == "Port 22\n"
assert DROPIN_DIR in result.message
def test_a_rejected_config_is_taken_back():
host = PenaltyHost()
host.syntax_ok = False
result = host.exempt_from_ssh_penalties()
assert result.rolled_back and not result.applied
assert PENALTY_DROPIN_PATH not in host.files
def test_an_exemption_sshd_does_not_use_is_taken_back():
"""A drop-in read before ours sets the directive; ours is valid and ignored."""
host = PenaltyHost()
host.files[f"{DROPIN_DIR}/00-aaa.conf"] = "PerSourcePenaltyExemptList 192.0.2.1\n"
result = host.exempt_from_ssh_penalties()
assert result.rolled_back
assert PENALTY_DROPIN_PATH not in host.files
def test_it_is_taken_back_when_a_new_session_cannot_get_in():
host = PenaltyHost()
result = host.exempt_from_ssh_penalties(verify=lambda: False)
assert result.rolled_back
assert PENALTY_DROPIN_PATH not in host.files
assert host.reloads == 2
def test_a_session_without_a_client_address_raises():
host = PenaltyHost(client="")
with pytest.raises(RuntimeError):
host.exempt_from_ssh_penalties()
+168
View File
@@ -0,0 +1,168 @@
"""A connection sshd turns away before the SSH banner is tried again (NetOrk/netork#1800).
OpenSSH 9.8+ drops every connection from a source it has just penalised
(``PerSourcePenalties``): it accepts the TCP connection and closes it at once,
and paramiko reports "Error reading SSH protocol banner". The penalty lasts
seconds to minutes, so a connection that waits a little gets in.
"""
from __future__ import annotations
import socket
import threading
import paramiko
import pytest
from napalm.base.exceptions import ConnectionException
from napalm_device_types import ssh_host_keys
from napalm_device_types.ssh_host_keys import (
REFUSED_RETRY_WAITS,
HostKeyCheck,
refused_before_banner,
retry_refused,
)
def _banner_error() -> paramiko.SSHException:
try:
try:
raise EOFError()
except EOFError as eof:
raise paramiko.SSHException("Error reading SSH protocol banner") from eof
except paramiko.SSHException as exc:
return exc
class TestWhatCountsAsRefused:
def test_paramikos_banner_error(self):
assert refused_before_banner(_banner_error())
def test_wrapped_by_a_driver(self):
try:
try:
raise _banner_error()
except paramiko.SSHException as exc:
raise ConnectionException(f"cannot connect: {exc}") from exc
except ConnectionException as wrapped:
assert refused_before_banner(wrapped)
@pytest.mark.parametrize(
"exc",
[
paramiko.AuthenticationException("Authentication failed."),
TimeoutError("timed out"),
ConnectionRefusedError(111, "Connection refused"),
ValueError("anything else"),
],
)
def test_not_a_failed_login_a_silent_host_or_a_closed_port(self, exc):
"""Those do not end by waiting: retrying them only delays the error."""
assert not refused_before_banner(exc)
class TestRetry:
def test_it_waits_and_tries_again(self):
slept: list[float] = []
attempts = iter([_banner_error(), _banner_error(), "connected"])
def attempt():
value = next(attempts)
if isinstance(value, BaseException):
raise value
return value
assert retry_refused(attempt, sleep=slept.append) == "connected"
assert slept == list(REFUSED_RETRY_WAITS[:2])
def test_it_gives_up_with_the_last_error(self):
slept: list[float] = []
def attempt():
raise _banner_error()
with pytest.raises(paramiko.SSHException):
retry_refused(attempt, sleep=slept.append)
assert slept == list(REFUSED_RETRY_WAITS)
def test_other_errors_are_not_retried(self):
slept: list[float] = []
def attempt():
raise paramiko.AuthenticationException("Authentication failed.")
with pytest.raises(paramiko.AuthenticationException):
retry_refused(attempt, sleep=slept.append)
assert slept == []
def test_the_waits_outlast_the_penalty_seen(self):
"""16 s on a new Ubuntu 26.04 VM on .50."""
assert sum(REFUSED_RETRY_WAITS) > 16
class _PenalisingServer:
"""A TCP server that closes the first *drops* connections at once, as a penalised
source's are, and then speaks SSH with any password."""
def __init__(self, drops: int) -> None:
self.drops = drops
self.listener = socket.create_server(("127.0.0.1", 0))
self.port = self.listener.getsockname()[1]
self.key = paramiko.ECDSAKey.generate()
self._thread = threading.Thread(target=self._serve, daemon=True)
self._thread.start()
def _serve(self) -> None:
while True:
try:
conn, _ = self.listener.accept()
except OSError:
return
if self.drops:
self.drops -= 1
conn.close()
continue
transport = paramiko.Transport(conn)
transport.add_server_key(self.key)
class _Any(paramiko.ServerInterface):
def get_allowed_auths(self, username):
return "password"
def check_auth_password(self, username, password):
return paramiko.AUTH_SUCCESSFUL
try:
transport.start_server(server=_Any())
except Exception:
pass
self.transport = transport
return
def close(self) -> None:
self.listener.close()
transport = getattr(self, "transport", None)
if transport:
transport.close()
def test_host_key_check_connects_once_the_penalty_is_over(monkeypatch):
server = _PenalisingServer(drops=2)
slept: list[float] = []
monkeypatch.setattr(ssh_host_keys.time, "sleep", slept.append)
client = paramiko.SSHClient()
try:
HostKeyCheck({}, "127.0.0.1", server.port).connect(
client,
username="admin",
password="s3cret",
look_for_keys=False,
allow_agent=False,
timeout=5,
banner_timeout=5,
)
assert client.get_transport().is_authenticated()
assert slept == list(REFUSED_RETRY_WAITS[:2])
finally:
client.close()
server.close()
+395
View File
@@ -0,0 +1,395 @@
"""The sudo password an earlier driver left in shell histories, and taking it back.
The scripts are run for real, with ``sh``, against a home directory in a temp
directory: what they count, delete and report is what a host would see. Root's
part -- giving files back to root -- runs with a ``chown`` that only records,
so the test needs no root.
"""
from __future__ import annotations
import getpass
import os
import subprocess
from pathlib import Path
import pytest
from napalm_device_types.channel import CommandResult
from napalm_device_types.sudo_history import HISTORY_BACKUP_SUFFIX, SudoHistoryMixin
MARKER = "| sudo -S -p "
LEAKED = "echo 'hunter2' | sudo -S -p \"\" apt-get update\n"
OTHER = "ls -la\n"
class _Host(SudoHistoryMixin):
"""A driver whose host is this machine, with *home* as the login user's home."""
SUDO_HISTORY_MARKER = MARKER
def __init__(self, home: Path, bin_dir: Path, *, root: bool = False) -> None:
self.home = home
self.bin_dir = bin_dir
self.root = root
#: Who ``id -un`` says the login user is; None for this machine's user name.
self.user: str | None = None
self.ran: list[tuple[str, bool]] = []
def run_command(
self,
command: str,
*,
privileged: bool = False,
timeout: float = 60,
stdin: bytes | None = None,
) -> CommandResult:
# The password goes ahead of a privileged command's stdin: there must be none.
assert not (privileged and stdin is not None)
self.ran.append((command, privileged))
env = {
**os.environ,
"HOME": str(self.home),
"PATH": f"{self.bin_dir}:{os.environ['PATH']}",
}
if command.strip() == "id -u; id -un" and self.root:
return CommandResult("0\nroot\n", "", 0)
if command.strip() == "id -u; id -un":
# Never this machine's uid: CI runs as root, which the owner step skips.
return CommandResult(f"4242\n{self.user or getpass.getuser()}\n", "", 0)
done = subprocess.run( # noqa: S603 - the test's own scripts
["sh", "-c", command], capture_output=True, text=True, env=env, timeout=timeout
)
return CommandResult(done.stdout, done.stderr, done.returncode)
@pytest.fixture
def bin_dir(tmp_path: Path) -> Path:
"""A ``chown`` that records its arguments instead of changing anything."""
d = tmp_path / "bin"
d.mkdir()
chown = d / "chown"
chown.write_text(f'#!/bin/sh\nprintf "%s\\n" "$*" >> {tmp_path / "chowned"}\n')
chown.chmod(0o755)
return d
@pytest.fixture
def home(tmp_path: Path) -> Path:
h = tmp_path / "home"
h.mkdir()
return h
def _chowned(tmp_path: Path) -> list[str]:
log = tmp_path / "chowned"
return log.read_text().splitlines() if log.exists() else []
class TestCounting:
def test_counts_the_lines_per_history_file_without_reading_them_out(self, home, bin_dir):
(home / ".bash_history").write_text(OTHER + LEAKED * 3)
(home / ".zsh_history").write_text(": 1700000000:0;" + LEAKED)
(home / ".sh_history").write_text(OTHER)
host = _Host(home, bin_dir)
found = host.get_history_sudo_lines()
assert found == [
{"file": "~/.bash_history", "lines": 3},
{"file": "~/.zsh_history", "lines": 1},
]
# Counted as the login user, and nothing of the content came back.
assert all(not privileged for _cmd, privileged in host.ran)
assert "hunter2" not in repr(found)
def test_no_history_no_lines(self, home, bin_dir):
assert _Host(home, bin_dir).get_history_sudo_lines() == []
def test_the_pattern_is_a_fixed_string(self, home, bin_dir):
# "-S" is no regex trouble, but "|" would be an alternation in an ERE.
(home / ".bash_history").write_text("sudo -S -p\n| sudo\n")
assert _Host(home, bin_dir).get_history_sudo_lines() == []
def test_a_driver_that_never_typed_the_password_counts_nothing(self, home, bin_dir):
(home / ".bash_history").write_text(LEAKED)
host = _Host(home, bin_dir)
host.SUDO_HISTORY_MARKER = None
assert host.get_history_sudo_lines() == []
assert host.ran == []
def test_an_unreadable_history_raises_rather_than_reporting_none(self, home, bin_dir):
if os.geteuid() == 0:
pytest.skip("root reads every file")
history = home / ".bash_history"
history.write_text(LEAKED)
history.chmod(0)
try:
with pytest.raises(RuntimeError):
_Host(home, bin_dir).get_history_sudo_lines()
finally:
history.chmod(0o600)
class TestRemoving:
def test_deletes_exactly_those_lines_and_says_how_many(self, home, bin_dir):
(home / ".bash_history").write_text(OTHER + LEAKED + "cd /etc\n" + LEAKED)
(home / ".ash_history").write_text(LEAKED)
host = _Host(home, bin_dir)
result = host.remove_history_sudo_lines()
assert result["success"] is True
assert (home / ".bash_history").read_text() == OTHER + "cd /etc\n"
assert (home / ".ash_history").read_text() == ""
assert result["removed"] == 3
assert result["files"] == [
{"file": "~/.bash_history", "before": 2, "after": 0},
{"file": "~/.ash_history", "before": 1, "after": 0},
]
assert "~/.bash_history: 2 lines removed (2 before, 0 after)" in result["output"]
def test_no_backup_with_the_password_is_left_behind(self, home, bin_dir):
(home / ".bash_history").write_text(LEAKED)
_Host(home, bin_dir).remove_history_sudo_lines()
assert sorted(p.name for p in home.iterdir()) == [".bash_history"]
assert not (home / f".bash_history{HISTORY_BACKUP_SUFFIX}").exists()
def test_the_history_runs_as_the_login_user(self, home, bin_dir):
(home / ".bash_history").write_text(LEAKED)
host = _Host(home, bin_dir)
host.remove_history_sudo_lines()
history_scripts = [(c, p) for c, p in host.ran if ".bash_history" in c]
assert history_scripts and all(not p for _c, p in history_scripts)
def test_a_clean_history_is_left_as_it_is(self, home, bin_dir):
history = home / ".bash_history"
history.write_text(OTHER)
before = history.stat().st_ino
result = _Host(home, bin_dir).remove_history_sudo_lines()
assert result["success"] is True
assert result["removed"] == 0
assert result["files"] == [{"file": "~/.bash_history", "before": 0, "after": 0}]
assert history.stat().st_ino == before # not rewritten
def test_a_history_that_is_a_link_is_left_alone_and_the_fix_fails(
self, home, bin_dir, tmp_path
):
target = tmp_path / "elsewhere"
target.write_text(LEAKED)
(home / ".bash_history").symlink_to(target)
result = _Host(home, bin_dir).remove_history_sudo_lines()
assert result["success"] is False
assert target.read_text() == LEAKED
assert (home / ".bash_history").is_symlink()
assert "~/.bash_history: a symbolic link, left alone (1 line)" in result["output"]
def test_the_lines_of_a_driver_that_never_typed_the_password_are_not_looked_for(
self, home, bin_dir
):
host = _Host(home, bin_dir)
host.SUDO_HISTORY_MARKER = None
result = host.remove_history_sudo_lines()
assert result["success"] is True
assert result["removed"] == 0
def test_an_unreadable_history_fails_the_step(self, home, bin_dir):
if os.geteuid() == 0:
pytest.skip("root reads every file")
history = home / ".bash_history"
history.write_text(LEAKED)
history.chmod(0)
try:
result = _Host(home, bin_dir).remove_history_sudo_lines()
finally:
history.chmod(0o600)
assert result["success"] is False
assert "~/.bash_history: cannot be read" in result["output"]
def test_a_sed_that_fails_keeps_the_lines_and_leaves_no_backup(self, home, bin_dir):
(bin_dir / "sed").write_text(
'#!/bin/sh\nfor a; do f="$a"; done\ncp "$f" "$f.napalm-bak"\nexit 4\n'
)
(bin_dir / "sed").chmod(0o755)
(home / ".bash_history").write_text(LEAKED * 2)
result = _Host(home, bin_dir).remove_history_sudo_lines()
assert result["success"] is False
assert "~/.bash_history: could not be rewritten (2 lines kept)" in result["output"]
assert sorted(p.name for p in home.iterdir()) == [".bash_history"]
def test_a_sed_that_lost_the_file_gets_it_back(self, home, bin_dir):
(bin_dir / "sed").write_text(
'#!/bin/sh\nfor a; do f="$a"; done\nmv "$f" "$f.napalm-bak"\nexit 4\n'
)
(bin_dir / "sed").chmod(0o755)
(home / ".bash_history").write_text(OTHER + LEAKED)
result = _Host(home, bin_dir).remove_history_sudo_lines()
assert result["success"] is False
assert (home / ".bash_history").read_text() == OTHER + LEAKED
assert sorted(p.name for p in home.iterdir()) == [".bash_history"]
def test_a_script_that_dies_says_why(self, home, bin_dir):
host = _Host(home, bin_dir)
host.run_command = lambda command, **kw: CommandResult("", "sh: killed", 137) # type: ignore[method-assign]
result = host.remove_history_sudo_lines()
assert result["success"] is False
assert "sh: killed" in result["output"]
def test_a_line_it_does_not_know_is_ignored(self):
files: list = []
said: list[str] = []
assert SudoHistoryMixin._history_line(["what", "ever"], files, said) is True
assert SudoHistoryMixin._history_line([], files, said) is True
assert files == [] and said == []
class TestOwners:
def _host(self, home, bin_dir, files):
host = _Host(home, bin_dir)
host.SUDO_ROOT_FILES = tuple(str(f) for f in files)
return host
def test_a_file_the_login_user_owns_goes_back_to_root(self, home, bin_dir, tmp_path):
conf = tmp_path / "snmpd.conf"
conf.write_text("x")
result = self._host(home, bin_dir, [conf]).remove_history_sudo_lines()
assert _chowned(tmp_path) == [f"-h 0:0 {conf}"]
assert result["owners_restored"] == [str(conf)]
assert result["success"] is True
def test_only_as_root_and_without_stdin(self, home, bin_dir, tmp_path):
conf = tmp_path / "snmpd.conf"
conf.write_text("x")
host = self._host(home, bin_dir, [conf])
host.remove_history_sudo_lines()
privileged = [c for c, p in host.ran if p]
assert len(privileged) == 1 and "chown" in privileged[0]
# The login user's name is quoted into root's script, never taken from stdin.
assert getpass.getuser() in privileged[0]
def test_a_link_or_a_missing_file_is_not_touched(self, home, bin_dir, tmp_path):
real = tmp_path / "real"
real.write_text("x")
link = tmp_path / "00proxy"
link.symlink_to(real)
result = self._host(home, bin_dir, [link, tmp_path / "missing"]).remove_history_sudo_lines()
assert _chowned(tmp_path) == []
assert result["owners_restored"] == []
assert result["success"] is True
def test_a_file_someone_else_owns_is_not_touched(self, home, bin_dir, tmp_path):
conf = tmp_path / "snmpd.conf"
conf.write_text("x")
host = self._host(home, bin_dir, [conf])
host.user = "nobody-here-4242"
result = host.remove_history_sudo_lines()
assert _chowned(tmp_path) == []
assert result["owners_restored"] == []
def test_a_root_login_has_nothing_to_give_back(self, home, bin_dir, tmp_path):
conf = tmp_path / "snmpd.conf"
conf.write_text("x")
host = self._host(home, bin_dir, [conf])
host.root = True
result = host.remove_history_sudo_lines()
assert _chowned(tmp_path) == []
assert result["success"] is True
def test_the_owners_are_set_even_when_the_history_could_not_be_cleaned(
self, home, bin_dir, tmp_path
):
target = tmp_path / "elsewhere"
target.write_text(LEAKED)
(home / ".bash_history").symlink_to(target)
conf = tmp_path / "snmpd.conf"
conf.write_text("x")
result = self._host(home, bin_dir, [conf]).remove_history_sudo_lines()
assert result["success"] is False
assert [s["step"] for s in result["steps"]] == ["history", "owners"]
assert result["owners_restored"] == [str(conf)]
def test_without_knowing_the_login_user_nothing_changes_owner(self, home, bin_dir, tmp_path):
conf = tmp_path / "snmpd.conf"
conf.write_text("x")
host = self._host(home, bin_dir, [conf])
real = host.run_command
def run(command, **kw):
if command.startswith("id "):
return CommandResult("", "id: not found", 127)
return real(command, **kw)
host.run_command = run # type: ignore[method-assign]
result = host.remove_history_sudo_lines()
assert result["success"] is False
assert "cannot tell who the login user is" in result["output"]
assert _chowned(tmp_path) == []
def test_a_sudo_that_refuses_fails_the_step(self, home, bin_dir, tmp_path):
conf = tmp_path / "snmpd.conf"
conf.write_text("x")
host = self._host(home, bin_dir, [conf])
real = host.run_command
def run(command, *, privileged=False, **kw):
if privileged:
return CommandResult("", "sudo: a password is required", 1)
return real(command, privileged=privileged, **kw)
host.run_command = run # type: ignore[method-assign]
result = host.remove_history_sudo_lines()
assert result["success"] is False
assert "sudo: a password is required" in result["output"]
def test_a_refused_chown_fails_the_step(self, home, bin_dir, tmp_path):
(bin_dir / "chown").write_text("#!/bin/sh\necho 'Operation not permitted' >&2\nexit 1\n")
conf = tmp_path / "snmpd.conf"
conf.write_text("x")
result = self._host(home, bin_dir, [conf]).remove_history_sudo_lines()
assert result["success"] is False
assert result["owners_restored"] == []
assert f"{conf}: could not be given back to root" in result["output"]
def test_exported_from_the_package():
import napalm_device_types
assert napalm_device_types.SudoHistoryMixin is SudoHistoryMixin
+285
View File
@@ -0,0 +1,285 @@
"""TLS certificates: trust on first use by fingerprint, a changed one refused before any login.
The layer above keeps the store (NetOrk/netork#1167, NAPALM/napalm-hpe-aruba-procurve#17).
A driver gets the fingerprint it should expect through ``optional_args["tls_fingerprints"]``
(by port) and reports every certificate it sees through
``optional_args["tls_certificate_observer"]``. The handshake tests run real TLS against an
in-process server on a socket pair: what matters is the certificate the server presents in
a real handshake, which a fake could not show.
"""
from __future__ import annotations
import datetime
import hashlib
import socket
import ssl
import threading
import pytest
import requests
from cryptography import x509
from cryptography.hazmat.primitives import hashes, serialization
from cryptography.hazmat.primitives.asymmetric import ec
from cryptography.x509.oid import NameOID
from napalm.base.exceptions import ConnectionException
from napalm_device_types.tls_certificates import (
KNOWN_FINGERPRINTS_ARG,
OBSERVER_ARG,
CertificateCheck,
TlsCertificateMismatch,
fetch_certificate,
fingerprint,
normalise_fingerprint,
)
DER = b"not really a certificate, but bytes are bytes"
DER_HEX = hashlib.sha256(DER).hexdigest().upper()
DER_FP = "SHA256:" + ":".join(DER_HEX[i : i + 2] for i in range(0, 64, 2))
OTHER_FP = "SHA256:" + ":".join(["AB"] * 32)
def _self_signed(tmp_path, name: str = "switch") -> tuple[bytes, str, str]:
"""A self-signed certificate, as a ProCurve ships one: (DER, cert file, key file)."""
key = ec.generate_private_key(ec.SECP256R1())
subject = x509.Name([x509.NameAttribute(NameOID.COMMON_NAME, name)])
now = datetime.datetime.now(datetime.UTC)
cert = (
x509.CertificateBuilder()
.subject_name(subject)
.issuer_name(subject)
.public_key(key.public_key())
.serial_number(x509.random_serial_number())
.not_valid_before(now - datetime.timedelta(days=1))
.not_valid_after(now + datetime.timedelta(days=1))
.sign(key, hashes.SHA256())
)
cert_file, key_file = tmp_path / f"{name}.crt", tmp_path / f"{name}.key"
cert_file.write_bytes(cert.public_bytes(serialization.Encoding.PEM))
key_file.write_bytes(
key.private_bytes(
serialization.Encoding.PEM,
serialization.PrivateFormat.PKCS8,
serialization.NoEncryption(),
)
)
return cert.public_bytes(serialization.Encoding.DER), str(cert_file), str(key_file)
class _TlsServer:
"""A TLS server on one end of a socket pair; it handshakes and then waits."""
def __init__(self, cert_file: str, key_file: str) -> None:
self.client_sock, server_sock = socket.socketpair()
context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
context.load_cert_chain(cert_file, key_file)
self._server_sock = server_sock
self._context = context
self.received = b""
self._thread = threading.Thread(target=self._serve, daemon=True)
self._thread.start()
def _serve(self) -> None:
try:
tls = self._context.wrap_socket(self._server_sock, server_side=True)
self.received = tls.recv(1024)
except (OSError, ssl.SSLError): # the client hung up after the handshake
pass
def connect(self, address, timeout=None):
return self.client_sock
def close(self) -> None:
self.client_sock.close()
self._server_sock.close()
self._thread.join(timeout=5)
@pytest.fixture
def certificate(tmp_path):
return _self_signed(tmp_path)
@pytest.fixture
def server(certificate):
_der, cert_file, key_file = certificate
s = _TlsServer(cert_file, key_file)
yield s
s.close()
def _check(expected: str | None, seen: list, port: int = 443) -> CertificateCheck:
args: dict = {OBSERVER_ARG: seen.append}
if expected is not None:
args[KNOWN_FINGERPRINTS_ARG] = {str(port): expected}
return CertificateCheck(args, "10.0.0.5", port)
class TestFingerprint:
def test_is_sha256_of_the_der_certificate_in_colon_hex(self):
assert fingerprint(DER) == DER_FP
@pytest.mark.parametrize(
"value",
[
DER_FP,
DER_FP.lower(),
DER_HEX,
DER_HEX.lower(),
DER_FP.removeprefix("SHA256:"),
" ".join(DER_HEX[i : i + 2] for i in range(0, 64, 2)),
"sha256:" + DER_HEX,
],
)
def test_a_pin_may_be_written_as_openssl_or_a_browser_shows_it(self, value):
assert normalise_fingerprint(value) == DER_FP
@pytest.mark.parametrize(
"value", ["", "AB:CD", "SHA1:" + DER_HEX, "ZZ" * 32, DER_HEX + "AB", 42]
)
def test_anything_else_is_refused(self, value):
with pytest.raises(ValueError):
normalise_fingerprint(value)
class TestKnownFingerprints:
def test_the_pin_is_the_one_for_the_port_connected_to(self):
args = {KNOWN_FINGERPRINTS_ARG: {"443": DER_HEX}}
assert CertificateCheck(args, "10.0.0.5", 443).expected == DER_FP
assert CertificateCheck(args, "10.0.0.5", 8443).expected is None
def test_a_port_may_be_given_as_a_number(self):
args = {KNOWN_FINGERPRINTS_ARG: {8443: DER_FP}}
assert CertificateCheck(args, "10.0.0.5", 8443).expected == DER_FP
def test_a_malformed_mapping_is_refused(self):
with pytest.raises(ValueError):
CertificateCheck({KNOWN_FINGERPRINTS_ARG: [DER_FP]}, "10.0.0.5", 443)
def test_without_optional_args_it_is_first_use(self):
assert CertificateCheck(None, "10.0.0.5", 443).expected is None
class TestVerify:
def test_first_use_accepts_and_reports_only_once_observed(self):
seen: list = []
check = _check(None, seen)
observation = check.verify(DER)
assert observation.accepted is True
assert seen == []
check.observe()
assert seen == [
{
"host": "10.0.0.5",
"port": 443,
"seen": {"fingerprint": DER_FP},
"expected": None,
"accepted": True,
}
]
def test_the_pinned_certificate_is_accepted(self):
seen: list = []
check = _check(DER_HEX.lower(), seen)
assert check.verify(DER).accepted is True
check.observe()
assert seen[0]["expected"] == {"fingerprint": DER_FP}
assert seen[0]["accepted"] is True
def test_another_certificate_is_refused_reported_and_named(self):
seen: list = []
with pytest.raises(TlsCertificateMismatch) as caught:
_check(OTHER_FP, seen).verify(DER)
assert caught.value.observation.seen == DER_FP
assert caught.value.observation.expected == OTHER_FP
assert seen == [
{
"host": "10.0.0.5",
"port": 443,
"seen": {"fingerprint": DER_FP},
"expected": {"fingerprint": OTHER_FP},
"accepted": False,
}
]
def test_the_mismatch_is_a_connection_failure_that_names_both(self):
with pytest.raises(ConnectionException) as caught:
_check(OTHER_FP, []).verify(DER)
message = str(caught.value)
assert "10.0.0.5:443" in message
assert DER_FP in message
assert OTHER_FP in message
def test_a_failing_observer_does_not_break_the_connection(self):
def boom(_observation):
raise RuntimeError("store down")
check = CertificateCheck({OBSERVER_ARG: boom}, "10.0.0.5", 443)
check.verify(DER)
check.observe()
def test_observe_before_verify_is_a_mistake(self):
with pytest.raises(RuntimeError):
_check(None, []).observe()
class TestHandshake:
def test_fetch_returns_the_certificate_the_server_presents(self, server, certificate):
der, _cert, _key = certificate
assert fetch_certificate("10.0.0.5", 443, 5, connect=server.connect) == der
def test_verify_server_refuses_another_certificate_before_anything_is_sent(
self, server, tmp_path
):
other_der, _c, _k = _self_signed(tmp_path, "impostor")
seen: list = []
check = _check(fingerprint(other_der), seen)
with pytest.raises(TlsCertificateMismatch):
check.verify_server(5, connect=server.connect)
server.close()
assert server.received == b""
assert seen[0]["accepted"] is False
def test_verify_server_accepts_on_first_use(self, server, certificate):
der, _cert, _key = certificate
observation = _check(None, []).verify_server(5, connect=server.connect)
assert observation.seen == fingerprint(der)
class TestRequests:
def test_pin_makes_every_connection_present_the_verified_certificate(self):
check = _check(None, [])
check.verify(DER)
session = requests.Session()
check.pin(session)
adapter = session.get_adapter("https://10.0.0.5/rest/v7/")
assert adapter.poolmanager.connection_pool_kw["assert_fingerprint"] == DER_HEX.lower()
proxy = adapter.proxy_manager_for("http://proxy.example:3128")
assert proxy.connection_pool_kw["assert_fingerprint"] == DER_HEX.lower()
def test_pin_before_verify_is_a_mistake(self):
with pytest.raises(RuntimeError):
_check(None, []).pin(requests.Session())
def test_a_changed_certificate_on_a_pinned_session_is_a_mismatch(self):
from urllib3.util.ssl_ import assert_fingerprint
seen: list = []
check = _check(DER_FP, seen)
check.verify(DER)
impostor = b"another certificate"
with pytest.raises(TlsCertificateMismatch) as caught, check.refusing():
try:
assert_fingerprint(impostor, DER_HEX)
except Exception as exc:
raise requests.exceptions.SSLError(exc) from exc
assert caught.value.observation.seen == fingerprint(impostor)
assert caught.value.observation.expected == DER_FP
assert seen[-1]["accepted"] is False
def test_refusing_lets_other_errors_through(self):
check = _check(None, [])
check.verify(DER)
with pytest.raises(requests.exceptions.ConnectionError), check.refusing():
raise requests.exceptions.ConnectionError("no route to host")
+65
View File
@@ -0,0 +1,65 @@
"""Moving a guest to another node of its cluster, and a node's HA maintenance mode.
``get_vm_migration_preflight`` says where a guest could go and what holds it
back, ``migrate_vm`` moves it and waits, ``set_node_maintenance`` switches a
node's HA maintenance mode. Declared only, under ``TYPE_CHECKING``: netOrk
probes each with ``hasattr`` and hides what a driver lacks (VMware has none of
them yet).
"""
from __future__ import annotations
import ast
import inspect
from typing import get_type_hints
import pytest
import napalm_device_types.hypervisor as hypervisor_module
from napalm_device_types import HypervisorDriver, MigrationPreflightDict
def _declared(name: str) -> ast.FunctionDef:
tree = ast.parse(inspect.getsource(hypervisor_module))
for node in ast.walk(tree):
if isinstance(node, ast.FunctionDef) and node.name == name:
return node
raise AssertionError(f"HypervisorDriver does not declare {name}()")
def _signature(name: str) -> str:
fn = _declared(name)
return f"({ast.unparse(fn.args)}) -> {ast.unparse(fn.returns) if fn.returns else None}"
class TestDeclarations:
@pytest.mark.parametrize(
"method", ["get_vm_migration_preflight", "migrate_vm", "set_node_maintenance"]
)
def test_absent_until_a_driver_implements_it(self, method):
assert not hasattr(HypervisorDriver, method)
def test_preflight(self):
assert _signature("get_vm_migration_preflight") == (
"(self, name: str) -> MigrationPreflightDict"
)
def test_migrate(self):
assert _signature("migrate_vm") == (
"(self, name: str, target_node: str, online: bool=True, "
"target_storage: str | None=None, timeout: int=3600) -> None"
)
def test_maintenance(self):
assert _signature("set_node_maintenance") == "(self, node: str, enabled: bool) -> None"
class TestPreflightShape:
def test_keys(self):
assert get_type_hints(MigrationPreflightDict) == {
"running": bool,
"allowed_nodes": list[str],
"refused_nodes": dict[str, str],
"local_disks": list[str],
"local_resources": list[str],
}
+53
View File
@@ -0,0 +1,53 @@
"""A provisioning that failed after the hypervisor gave the VM its id says
which VM it left behind, so the caller can remove it (NAPALM/napalm-proxmox#28)."""
from __future__ import annotations
import pickle
import pytest
from napalm_device_types import VMProvisionIncomplete
from napalm_device_types.provisioning import VMProvisionIncomplete as from_module
def _raise_from(cause: Exception) -> VMProvisionIncomplete:
try:
try:
raise cause
except Exception as exc:
raise VMProvisionIncomplete(str(exc), vmid="101", node="pve1") from exc
except VMProvisionIncomplete as caught:
return caught
def test_it_is_exported_from_the_package_root():
assert VMProvisionIncomplete is from_module
def test_it_names_the_vm_left_behind():
caught = _raise_from(RuntimeError("disk import failed"))
assert (caught.vmid, caught.node) == ("101", "pve1")
def test_it_keeps_the_failure_and_its_words():
cause = RuntimeError("disk import failed")
caught = _raise_from(cause)
assert caught.__cause__ is cause
assert str(caught) == "disk import failed"
def test_a_caller_catching_runtime_error_still_catches_it():
# The contract has create_vm_from_cloud_init raise RuntimeError on failure.
with pytest.raises(RuntimeError):
raise VMProvisionIncomplete("start failed", vmid="7", node="n")
def test_vmid_and_node_are_keyword_only():
with pytest.raises(TypeError):
VMProvisionIncomplete("failed", "101", "pve1") # type: ignore[misc]
def test_it_survives_pickling_as_a_task_result_would():
caught = pickle.loads(pickle.dumps(VMProvisionIncomplete("failed", vmid="101", node="pve1")))
assert (str(caught), caught.vmid, caught.node) == ("failed", "101", "pve1")
+28
View File
@@ -0,0 +1,28 @@
"""begin_stop_vm: ask for a VM's clean shutdown and return at once.
``stop_vm`` blocks until the VM is stopped, with a limit of the driver's own.
A caller that keeps its waits as timers (netOrk's graph workflows,
NetOrk/netork#1636) asks with ``begin_stop_vm`` and reads ``get_vm_status``
until the VM is stopped. A driver mixes it in when it can; no role demands it.
"""
from __future__ import annotations
import inspect
from napalm_device_types import HypervisorDriver, VmStopRequestMixin
def test_no_role_demands_it():
assert not issubclass(HypervisorDriver, VmStopRequestMixin)
def test_declared_not_implemented():
assert not hasattr(VmStopRequestMixin, "begin_stop_vm")
def test_declaration_documents_the_contract():
source = inspect.getsource(VmStopRequestMixin)
assert "def begin_stop_vm(self, name: str) -> None" in source
assert "ValueError" in source
assert "RuntimeError" in source