- CI moves to .gitea/workflows/ci.yml and calls the shared driver workflow
of NAPALM/driver-standard; .github/ is gone. napalm-device-types comes
from git until 4.22.0 (AptProxyMixin) is on PyPI; the floor stays at
4.22.0.
- requires-python >=3.10 (napalm-device-types needs it), py.typed, SPDX
license string, a dev extra with floors (pytest-mock and responses are
dropped: no test uses them), links to git.netork.io (README included).
pytest no longer adds --cov by itself; CI passes it.
- mypy strict for the package; exempt for now, with 204 errors between
them: napalm_proxmox.config_mixin, .driver, .interfaces_mixin,
.lldp_mixin, .routing_mixin, .sdn_mixin, .system_mixin, .utils,
.vm_contract_mixin, .vm_mixin and .vm_provision_mixin. They are recorded
in the quality baseline with the long and complex functions and the two
unannotated public methods.
- Coverage floor 68 % (measured 68.94 %).
- driver.py: the logger is defined after the imports (E402). The lazy
paramiko client is annotated Any, as _node_ssh already is; the quoted
paramiko.SSHClient named a module that is never imported there (F821).
- Unused locals are gone: get_facts' hostname, get_users' groups (both
read a value nobody used), the ARP flags column is _flags; the output is
unchanged.
- Long lines are wrapped: the apt-get and snmpd.conf commands are split
into adjacent string literals, the power-action error message and the
system mixin's docstring are wrapped; the text is unchanged.
- get_vm_status re-raises with raise ... from e (B904).
- The two except Exception: pass (closing the SSH session, the is_alive
probe) now log at debug level (B110), so a swallowed failure leaves a
trace.
- nosec for the empty "password" field of NAPALM's get_users shape and
the ssh_password_auth flag (B105); all three are false positives.
- tests: unused loop variables.
Mechanical: ruff format, and ruff check --fix with the rule set every NAPALM
driver uses (E, F, I, UP, B at line length 100, Python 3.10). What ruff
cannot fix comes in the next commit.
The node is Debian: AptProxyMixin gives it get_package_proxy(),
get_package_proxy_report(), check_package_proxy() and the fix
configure_package_proxy(), as on any Debian host. Logged in as root, the
transports are the exec path; the file is written without an exit inside
_rc's braces, which would end the shell before the status (see #30).
Requires napalm-device-types 4.22.0; version 0.7.0.
Mixes in napalm-device-types' ServicePackagesMixin; the transport is the
node's exec path, as for the kernel facts. Read-only. Requires
napalm-device-types 4.20.0; version 0.6.0.
The driver accepted any SSH host key. It now verifies through
napalm-device-types' HostKeyCheck (>= 4.19.0): with a known key in
optional_args["ssh_host_keys"] any other key is refused before login, and
every key seen is reported to optional_args["ssh_host_key_observer"].
Without either argument it connects as before (first use).
get_device_warnings is gone; netOrk names the warnings from facts
(netOrk #1111). lldpd and the pending updates were already facts
(get_packages, get_available_updates). The subscription is now
get_licenses() from napalm-device-types 4.14's LicenseMixin: one
pve-subscription entry, the API status mapped to a LicenseStatus.
The old check compared against "NotFound" while the API answers
"notfound" (seen on all four nodes), so it never fired; the mapping is
case-insensitive.
fix_snmp overwrote snmpd.conf with an application's values (community
"public", "Managed by netOrk", netork@localhost) through a fixed
/tmp/netork_snmpd.conf, and opened ufw for a /24 guessed from ss;
install_lldpd was an action of its own. Neither read an exit status.
The driver implements napalm-device-types 4.13's templates instead:
- configure_snmp_agent(settings): net-snmp installed, snmpd.conf with
one rocommunity line per allowed source (mktemp, install as root),
ufw opened for those sources only (pve-firewall stays the
operator's), snmpd enabled and restarted;
- get_management_peer() from SSH_CONNECTION;
- install_package(name), as the contract says (raises when apt fails),
so ensure_package comes from the base package.
Every step reads its exit status.
BREAKING CHANGE: run_device_action is removed. Requires
napalm-device-types 4.13.
Part of #24.
Mixes in napalm-device-types 4.9's SystemIdentityMixin over the exec
path, which runs as root: the node's SMBIOS system UUID, product serial
and board serial, vendor placeholders empty. pve-dual, a Supermicro
X10DRL-i, reports the product serial 0123456789; its board serial and
UUID are what tie it to its BMC.
create_vm_from_cloud_init(guest_os=...) with GUEST_AGENTS from
napalm-device-types 3.0 (QEMU_GUEST_AGENTS: Linux, FreeBSD, OpenBSD). An
unknown guest_os is refused before anything is created. Found on FreeBSD
15.1 and OpenBSD 7.9 cloud images (NetOrk/netork#793):
- OS type `other` for the BSDs. OpenBSD's qemu-ga only works over ISA
serial, and only as the second port: the image keeps its console on
com0, so the VM gets `serial0: socket` next to `agent: 1,type=isa`.
- A BSD guest gets a network-config v2 snippet of its own
(`cicustom: user=...,network=...`, MAC read back from net0). FreeBSD's
nuageinit fails on the v1 Proxmox generates and then skips runcmd,
which starts the guest agent. Linux keeps Proxmox's own.
- Packed images (FreeBSD's .qcow2.xz) are unpacked on the node after the
packed file's checksum is verified; only the unpacked file is cached.
download-url unpacks ISOs only, so they always take the SSH path.
Fixed on the way:
- get_vm_status skipped loopback only when it was called `lo`; OpenBSD's
agent lists `lo0` first, so 127.0.0.1 would have been the VM's IP.
Loopback is now recognised by its address.
- destroy_vm read cicustom after deleting the VM, when its config was gone,
so snippets stayed on the node; it now reads them first and removes all
of them (#15).
test_destroy_vm_already_stopped left the stop task a bare MagicMock, so
_wait_for_task never saw "stopped" and looped for the full 60 s timeout
with time.sleep patched out. Every call landed in mock_calls: the test
took 60 s and up to 6 GB, and the new CI's 3.12 job was killed for it
(exit 137).
The stop call now raises, as Proxmox does for a VM that is not running --
which is the case the test is named for. The suite drops from 63 s to 4 s.
The same job as napalm-fritzbox and napalm-opnsense: Python 3.10, 3.11 and
3.12, pytest, then wheel and sdist. napalm-device-types comes from
git.netork.io first, because PyPI has an unrelated package of that name.
ProxmoxDriver mixes in ListeningSocketsMixin (napalm-device-types 2.4.0)
and carries its command over the exec path, which runs as root either
way: whether pveproxy, a Ceph manager or anything else on the node listens
on an address reachable from outside it.
For netOrk#658.
A plain apt-get upgrade holds back what needs new or removed packages and
can leave a node half updated; Proxmox documents apt full-upgrade. netOrk
reads FULL_UPGRADE when it upgrades the host (MVP 5).
For netOrk MVP 5, on napalm-device-types 2.3.0:
- get_available_updates reads `apt list --upgradable` over the exec path
(APT_UPGRADABLE_COMMAND), so each update carries its suite and security
status; the APT API, which names only "Debian"/"Proxmox", is the fallback
with security unknown. It raises when neither answers instead of returning
[] -- it used to swallow every error.
- refresh_available_updates(): POST nodes/{n}/apt/update.
- HostStatusMixin over the exec path (reboot required, self-patching).
get_services() read list-unit-files' second column, which since systemd 245
is followed by a preset column -- "enabled enabled" never equalled
"enabled", so every service on every node was reported disabled (#6).
manage_service() ended in "|| true" and returned success whatever happened.
Both now come from napalm-device-types' SystemdServicesMixin (2.2.0): the
enabled state is UnitFileState from systemctl show, MainPID comes along in
the same round trip, and an action reports systemctl's exit status. The
driver keeps only _run_service_command(), its exec path as root. The exit
status marker keeps the output non-empty, so _exec_ssh_command's SSH fallback
on an empty API answer can no longer run an action twice.
The listing no longer includes templates, static unit files and aliases that
are not loaded (on a PVE 9 node: 280 entries -> 172).
Closes#6
A Proxmox node runs its own kernel under every guest, which makes it the host
where a kernel CVE's preconditions matter most. ProxmoxDriver mixes in
KernelFactsMixin from napalm-device-types and supplies only the transport,
the existing exec path.
Requires napalm-device-types 2.1.0.
A VM created without a cpu argument gets Proxmox's API default, kvm64: no
AES-NI and no AVX. MongoDB 5.0 and later exit with "Illegal instruction" on
it, which is how a Graylog provisioned through netOrk failed (netork#494).
Every VM built by hand in the same cluster uses host or x86-64-v2-AES; only
the ones this driver created were left on kvm64.
create_vm_from_cloud_init now always passes cpu=, defaulting to x86-64-v2-AES
as the Proxmox GUI does since PVE 8, and takes cpu_type for a different one.
get_vm_cpu_types() lists x86-64-v2-AES (default), x86-64-v3 and host. The
named models carry the cpuinfo flags they add over qemu64, following
Proxmox's own definitions (CPUConfig.pm), and are available when the node's
CPU has all of them, read from /nodes/{node}/status. host lists the node's
own flags, so on a CPU without AVX it does not pretend to offer any.
A model asked for by name is checked against the node before a VMID is
allocated: one the CPU cannot run would only fail at VM start, after the disk
import, leaving a half-built VM behind. The default is not checked, so a
plain create costs no extra API call. VMCpuTypeDict is imported for type
checking only, so this works with an older napalm_device_types.
Provisioning downloaded every cloud image over SSH into
/var/lib/vz/template/netork-images, on the node's root filesystem. On a
small root that fills up and takes Proxmox down with it (netOrk #480).
When the node has an active storage with content type "import" (Proxmox
8.2+), Proxmox now does it itself: download-url with checksum
verification into that storage, then import-from as the root disk. The
file is named after a hash of the full URL and reused when present.
Proxmox takes the format from the extension and has no ".img", so
Ubuntu's qcow2 .img is stored as .qcow2 -- a wrong guess fails at import
instead of attaching a qcow2 container as a raw disk.
Without an import storage, or for an image type Proxmox cannot import,
the SSH download is used as before.
_resolve_node() took the first entry of GET /nodes. In a cluster that
lists every member, so a node polled without an explicit `node` driver
argument talked to whichever member came first: pve-dual reported
pve-02's name and VMs, and netOrk's VM sync moved pve-02's VM devices
over to it.
Resolve through GET /cluster/status instead: the entry marked local,
then a match by IP or (short) name, then the sole node of a standalone
host, and otherwise raise rather than guess.
The lookup no longer swallows API errors either. A TLS verification
failure used to leave the IP as the node name, so open() succeeded and
every getter failed quietly while the poll reported success with empty
data. It now surfaces as a ConnectionException from open().
Refs NetOrk/netork#417, NetOrk/netork#418
get_vm_snapshots, create_vm_snapshot, delete_vm_snapshot and
rollback_vm_snapshot for VMs and containers, so netOrk's snapshot view
works on Proxmox as it does on VMware. Proxmox lists the live state as a
pseudo-snapshot named "current"; it is never reported or addressable.
Containers have no RAM state, so include_memory is ignored for them.
reboot_host() restarts the node with POST /nodes/{node}/status
command=reboot instead of /sbin/reboot over SSH.
start_vm, stop_vm, reboot_vm, suspend_vm and get_vm_config existed only
as declarations. netOrk called Proxmox's own power_vm and read a VM's
raw config through _node_api(), so no other hypervisor could serve the
same endpoints. These let netOrk talk to every hypervisor alike.
The power methods accept a VM's name or vmid, wait for the Proxmox task,
and raise ValueError/RuntimeError as the contract says instead of
returning a result dict. A forced reboot of a container is stop + start,
since LXC has no reset; suspending a container is refused. power_vm is
unchanged for existing callers.
get_vm_config moves the config parsing netOrk did in
_parse_proxmox_hw_config into the driver and returns a VMConfigDict:
disks with storage and size, NICs with model, MAC, bridge and VLAN, CPU
topology, firmware, machine type and PCI/USB passthrough.
get_vms reports vmid as a string ("100"), following
napalm-device-types 2.0, still ordered numerically.
`get_packages` named the Debian source package and never its version, so a
consumer was handed two numbers on different axes and no way to tell.
OSV states Debian ranges in *source* versions. libldb2 is
2:2.11.0+samba4.22.11+dfsg-… while its source, samba, is 2:4.22.11+dfsg-…;
comparing the first against a samba range is meaningless, and dpkg reads
ldb's 2.11.0 as older than the 2:4.17.4+dfsg-1 that fixed CVE-2022-44640.
Reporting the source without its version is worse than reporting neither,
because it looks usable.
Measured on three live Proxmox nodes: every one of their 2 349 packages
was in that state — 802 of 802, 774 of 774, 773 of 773 — while twenty
non-Proxmox hosts had both fields. It was not a parsing bug. The
dpkg-query format string never asked for ${source:Version}, so nothing
downstream could have recovered it.
Now asked for and reported, with the same fallback napalm-linux uses:
dpkg leaves the field empty when it equals Version, and an older dpkg
leaves it empty because it does not know the field at all. Neither may
produce a package without a coordinate.
tests/test_packages.py covers all of it, including that the *query* names
the field — the assertion that would have caught this.
dpkg-query now also reports ${source:Package} as source_package, so consumers
can match installed binaries to the correct Debian source (e.g. openssh-server
-> openssh) for accurate OSV vulnerability lookups.
Closes netork#115.
The suite had been red long enough that it stopped being read. Four of the
fourteen failures were the tests being right.
`interfaces_mixin.py` used `re.match` without importing `re`, so
`get_mac_address_table` raised NameError against any node with a Linux bridge.
The tests never reached that line: they mocked the API call underneath
`_exec_ssh_command`, which takes two positional arguments where the doubles
accepted one, and which base64-wraps the command — so a fixture keyed on
"bridge fdb" appearing in the text matched nothing and the helper returned "".
They mock `_exec_ssh_command` itself now, which is the driver's own seam.
`is_alive` called `_resolve_node()`, which returns early without touching the
API whenever a node was configured through optional_args. A dead connection
reported itself alive. It probes `GET /version` now.
The documented `realm` optional_arg was read into `self._realm` in `__init__`
and then never used. Proxmox authenticates against "<user>@<realm>" and rejects
a bare username, so the option had no effect and callers had to know to type the
realm themselves.
`get_vlans` filtered out entries with no member ports on one return path while
the OVS path returned them, so a configured SDN VNet was visible or invisible
depending on which branch ran. A VNet exists on the node whether or not anything
is attached to it, and netOrk's VLAN discovery reads this.
`get_ipv6_neighbors_table` was simply missing and fell through to NAPALM's stub;
it is implemented against `ip -6 neigh show`, dropping FAILED entries.
The rest were stale tests. The DNS fixture put an FQDN where a search domain
belongs, which made `get_facts` build "pve1.pve1.example.com" and look like a
driver bug. The LLDP fixture was a simplified shape that real `lldpcli show
neighbors summary` does not produce — the parser matches on the ", via: LLDP"
that follows the interface name. And `test_bridge_vlan_show_parsing` covered a
fallback that was replaced by VM-config scanning, asserting an "interfaces" key
this method has never returned; it is now a test of the fallback that exists.
Both were facts about this driver that netOrk kept in hardcoded driver-name
sets, each duplicated across a file pair (netork#113). The driver is the right
place for them: everything runs over the PVE REST API, and a node reboots
through a full init sequence plus storage checks before it is worth polling.
get_device_warnings() now returns only {code, meta} — severity, title,
message, and action are resolved centrally by netork's
WARNING_CATALOG (netork/core/device_warnings.py), not by the driver.
Keeps this driver independent of netork and avoids per-vendor drift in
how the same warning code is presented.
_download_cloud_image() cached downloaded images under just the URL's
basename (e.g. ubuntu-26.04-server-cloudimg-amd64.img). Ubuntu's per-build
download URLs change daily under that same stable basename
(.../release-20260713/... vs .../release-20260714/...), so a previous
day's cached file satisfied the "already cached" check and got checksum-
verified against the *new* day's expected hash from NetOrk's daily catalog
sync — failing outright and aborting the whole provisioning job, even
though a plain retry would have re-downloaded and succeeded (the bad file
was already being deleted on mismatch, just never re-fetched).
Found live during a NetOrk deploy: "Checksum mismatch for
https://cloud-images.ubuntu.com/.../release-20260713/
ubuntu-26.04-server-cloudimg-amd64.img: expected 0826c500..., got
3ee4f67f...".
Fix: key the cache path on a hash of the full URL (not just the
basename), and retry the download once after a checksum-mismatch cleanup
before raising.
Passed destroy_unreferenced_disks (underscore) as a kwarg to proxmoxer's
delete(), but Proxmox's actual DELETE /nodes/{node}/qemu/{vmid} parameter
is hyphenated (destroy-unreferenced-disks). proxmoxer forwards kwargs to
the request verbatim with no underscore-to-hyphen translation, so Proxmox
rejected every call with "property is not defined in schema" before ever
touching the VM — the VM stayed fully intact (config, disks) despite the
caller believing destroy had at least been attempted. Fixed by building
the params as a dict (bypassing the Python-identifier restriction) with
the correct hyphenated key.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
christianmanivong
Christian Manivong