napalm-device-types reads the system firmware in the same round trip
(bios_vendor, bios_version, bios_date). The driver only carries the
command across, so its test checks the identity it reads and passes
against napalm-device-types with and without the firmware.
NetOrk/netork#2518
The mktemp write of /etc/hosts made set_hostname one line shorter, and
the quality ratchet fails on a baseline entry that is no longer reached
(stale). Lowered with napalm-standard ratchet --write.
napalm-device-types 4.36.0 (released after main's last CI run) changed
PosixTimeSyncMixin: get_time_sync() reports DHCP-provided servers apart
(dhcp_ntp_servers), and configure_time_sync() no longer comments out the
DHCP client's chrony sourcedir; it gives back one that 4.35.0 took out
(NetOrk/netork#2380). CI installs the newest release, so five time-sync
tests failed on every branch.
The tests now expect that behaviour, read DHCP's servers apart, and check
that a node 4.35.0 aligned gets its DHCP sourcedir back. The floor moves
to 4.36.0, so the lowest-versions job tests the same behaviour; README,
CHANGELOG and the module docstring say what a write leaves alone.
napalm-device-types 4.36.0 (released after main's last CI run) changed
PosixTimeSyncMixin: get_time_sync() reports DHCP-provided servers apart
(dhcp_ntp_servers), and configure_time_sync() no longer comments out the
DHCP client's chrony sourcedir; it gives back one that 4.35.0 took out
(NetOrk/netork#2380). CI installs the newest release, so five time-sync
tests failed on every branch.
The tests now expect that behaviour, read DHCP's servers apart, and check
that a node 4.35.0 aligned gets its DHCP sourcedir back. The floor moves
to 4.36.0, so the lowest-versions job tests the same behaviour; README,
CHANGELOG and the module docstring say what a write leaves alone.
The new /etc/hosts was written as root to the fixed /tmp/_netork_hosts
and moved over /etc/hosts. A local user could create that file (or a
symlink) beforehand: root's write went into the user's file, and the
user owned /etc/hosts afterwards (NetOrk/netork#2446).
The file is now made with mktemp beside the target (/etc/.hosts.XXXXXX,
root's directory, same filesystem), written, set to 644 and moved into
place; a failed step removes it.
`postconf -e 'myhostname = <fqdn>'`, `hostname '<name>'` and the
/etc/pve/nodes move put the name between single quotes of their own, so
a name with a single quote ended them and the rest ran as root
(NetOrk/netork#2456). Each value is now passed through shlex.quote.
/etc/hostname, /etc/hosts and /etc/mailname were already written
through base64.
netOrk keeps one list of NTP servers and a time zone per site and aligns
a device that differs (NetOrk/netork#2360, #2365). Both drivers now
implement napalm-device-types' TimeSyncMixin (4.35.0) and write both
values.
Neither API keeps time servers: /nodes/{node}/time knows only the zone,
and both products run chrony on Debian. The servers are therefore
PosixTimeSyncMixin's, over the driver's existing SSH session
(run_command, root): read from chrony's configuration, written to
/etc/chrony/sources.d/netork.sources with every other source commented
out, then a reload or a restart of chrony.
The zone is read with them from the host (timedatectl: the zone the
clock really runs in), and from GET /nodes/{node}/time only where the
host names none. It is written through PUT /nodes/{node}/time rather
than timedatectl set-timezone: Proxmox's own call validates the name and
keeps /etc/timezone, which its web UI and API show, in step with
/etc/localtime.
One device is one node: in a PVE cluster every node has its own clock,
and only the node this connection resolved is read and written.
get_ntp_servers() read GET /nodes/{node}/ntp, a path Proxmox VE does not
have, so it always reported no servers. It now reports what
get_time_sync() reads, on the backup server too.
netOrk holds a host's restart after patching while a backup runs on it, and
a guest's restart while its hypervisor is backing it up (NetOrk/netork#2318).
It asks right before the restart, so both drivers answer
get_running_backups() from napalm-device-types 4.34.0:
- PVE: the node's active vzdump tasks, each with the guests it is taking now
(its own guest, or for a job the guests locked for their backup), and a
guest locked without a task on its own, since the task list can lag and a
lock can outlive its task.
- PBS: the running backup, sync, verify, gc, prune and tape tasks; a restore
or housekeeping is no backup.
A failed read raises, so netOrk reads it as "not checked", never as "none".
Both drivers mix in napalm-device-types' LogSourcesMixin (4.30.0) and
carry its command over the exec path, which runs as root either way.
Refs NetOrk/netork#2049
The guard is not about the link: napalm-device-types now writes the file a
link names. /etc/cron.d/vzdump points into /etc/pve, the cluster file
system, where installing a file with a mode and owner is not safe.
Refs #56
Both drivers mix in napalm-device-types' PosixScheduledJobsMixin over the
node's root SSH session: get_scheduled_jobs reads cron, run-parts, anacron
and every systemd timer in one round trip; manage_scheduled_job enables,
disables or runs one now; put_ and delete_scheduled_job write and remove an
application's own job.
What Proxmox ships is platform, so it is only switched and run: pve-*
units, pvesr.timer and /etc/cron.d/vzdump on Proxmox VE, proxmox-backup*
units on a backup server. A vzdump cron line is only run, because the file
is a link into /etc/pve and rewriting it would replace the link with a
plain file.
Backup jobs (/cluster/backup), replication and a backup server's verify,
prune, sync and GC jobs run in Proxmox's own daemons and stay with the
backup features.
Requires napalm-device-types 4.26.0.
Closes#56
get_clusters() lists a pbs storage like any other, so nothing told an
application which Proxmox Backup Server and datastore it writes to.
Each pbs storage now carries ClusterStorageDict.backup_server
(napalm-device-types 4.25.0): server, port, datastore, namespace and
fingerprint from /storage, never the password or encryption key. An
unreadable /storage leaves it None; other storages carry no such key.
This was step 6 of #52 and was left out there.
A second driver in this package, ProxmoxBackupDriver. Its API and web UI
are on 8007, not on PVE's 8006, and its node is always localhost; open()
proves the login with GET /version, so a bad token fails there.
Roles: BackupServerDriver (napalm-device-types 4.25.0) first, then
StorageDriver for the disks and ZFS pools the server manages. It reads
datastores, the backup groups of every namespace with their newest
snapshot, a group's snapshots, the GC, verify, prune, sync and tape jobs
with how their last run ended, and the remotes without their secrets;
it starts garbage collection and jobs, verifies, protects and deletes
snapshots, and follows the tasks. As a host it reports facts,
interfaces, CPU and memory, disks, pools, its pbs-subscription and the
same packages, updates, host readers and sshd hardening as a PVE node.
A privilege-separated token without an ACL entry gets an empty list from
PBS, not a refusal, so every read checks /access/permissions and raises
PermissionError. A removable datastore that is not mounted is reported
as unmounted and left out of the group listing.
The fixture is what a PBS 4.2 answered, sanitised; groups, snapshots and
jobs it did not have yet follow the API schema.
Proxmox Backup Server speaks the same REST dialect, logs in the same way
and runs on Debian too, so what a second driver needs moves out of
ProxmoxDriver first:
- napalm_proxmox._base: the connection (proxmoxer service PVE or PBS,
each with its port), the token split, the SSH session, run_command,
the node helpers and the DMI read.
- napalm_proxmox.host_mixin: packages and APT updates, the subscription,
restart and shutdown through the API, root file writes and the
transports of the host readers (kernel, listening sockets, identity,
systemd, host status, apt's proxy).
The PVE driver behaves as before; what it hands proxmoxer is pinned by
tests first, and it now names service="PVE" explicitly. get_packages is
split so the dpkg line parser stands alone; get_facts and get_packages
leave the quality baseline.
A Proxmox node runs Debian's stock OpenSSH, so OpenSshServerMixin works
there as on a plain Linux host. What it needed was run_command: an exec
channel on the node's SSH session, via napalm-device-types'
run_on_transport. The USES_SSH comment said there was no SSH session;
it now says what the flag means to netOrk.
_write_snippet put the snippet storage's path and the snippet's file
name unquoted into the command that writes the snippet as root on the
node; a space, ; or $(...) in either was acted on by the shell. Both now
go through shlex.quote, and a file name that is not one plain name
(a path, a shell character) is refused before anything runs.
Closes#47
paramiko's look_for_keys and allow_agent stayed at their default True,
so the node login also tried every key in the running process's own
~/.ssh and any SSH agent it could reach. The driver standard lets a
driver authenticate only with what it was handed; both are now False
on every connect.
Closes#48
lldpcli's text was read with 2>/dev/null || true, so lldpd not running,
lldpctl missing or the command not running at all came back as a node
without neighbours. Capabilities were lldpd's symbols lower-cased (tel,
wlan, docsis) rather than NAPALM's names.
The node now runs napalm-device-types' LLDPCTL_KEYVALUE_COMMAND, whose
output carries lldpctl's exit status; parse_lldpctl_keyvalue raises on a
failure, an empty answer raises too, and the lldp_neighbors projections
give NAPALM's capability names and the chassis MAC. The module is under
strict mypy now; the floor on napalm-device-types rises to 4.24.0.
Closes#44
_exec_ssh_command returned "" on any failure (no SSH credential, a
refused login or host key, an unreachable node, a timeout), so a
getter that could not look read like one that found nothing; on
API-token-only nodes get_snmp_config answered None and the SSH-backed
getters empty lists on every poll. It now raises ConnectionException
with the cause (SshHostKeyMismatch as it is), and refuses before
connecting when there is neither a password nor a usable key.
It returned stderr when stdout was empty; output is stdout now. It
first posted every command to /nodes/{node}/execute, which runs a
batch of API calls rather than a shell; that dead round trip is gone.
The key types, the key before the password, the host-key check and the
single connect path were done in f491f9f and 6bb3035. The shared
transport is NAPALM/napalm-device-types#35.
Closes#17
The cloud-image download put image_url, hand-wrapped in single quotes,
and the cache path built from its basename unquoted into commands that
run as root on the node: a quote, $(...), ; or space in a URL from a
release feed was acted on by the shell. The URL and every path built
from it now go through shlex.quote (exists check, wget, mv, checksum,
rm, unpack, qm importdisk), and the cache file name keeps only the
characters Proxmox keeps in a content file name, as the import volume
name already did. The checksum algorithm is checked against the ones
Proxmox knows, since it is part of the command name.
The file-name helpers move to image_names.py; vm_provision_mixin.py
would otherwise cross the 1000-line limit. _download_cloud_image
drops below 50 lines, so its baseline entry goes.
Closes#43
get_clusters() reads /cluster/ha/* (PVE 8 and 9), /cluster/backup with pool
members, the newest backup per guest on each backup storage (PBS included) and
/cluster/config/nodes votes; each read is best-effort. migrate_vm() moves a
guest of the connected node and follows the task (HA-managed guests through
the HA manager), raising with the task log's ERROR lines;
get_vm_migration_preflight() maps PVE's own pre-flight; set_node_maintenance()
runs ha-manager over SSH.
Part of NetOrk/netork#1820 (stage 7: #759, #760, #761).
A failure after the VM shell existed (disk import, NIC or snippet config,
start) raised without the VMID, so the caller could not remove the
half-built VM and it stayed on the node. It now raises napalm-device-types'
VMProvisionIncomplete with vmid and node, the failure as its cause. A
failure before the shell exists raises as before.
The NIC config strings are built by a helper of their own, which keeps
create_vm_from_cloud_init within its quality baseline.
Closes#28
The rename moves /etc/pve/nodes/<old> while the node still runs as <old>,
which Proxmox does not support for a cluster member. set_hostname now
checks for a corosync.conf first and raises before touching anything on
a member, or when the check cannot be read; a single node renames as
before.
The check is the file, not `pvecm status`: pvecm decides membership by
/etc/pve/corosync.conf itself, and its status also fails on a member
whose corosync is down. /etc/corosync/corosync.conf covers a member
whose /etc/pve is not mounted.
Closes#21
wget ran with -q, so a failure carried no reason, and a failed
download left its partial .tmp in the cache, taking the same space
again on every retry. wget now runs with -nv and the .tmp is removed
on failure, keeping wget's exit status.
Closes#3
The probe pattern expected "from" after icmp_seq, but iputils prints
"bytes from <ip>: icmp_seq=... time=... ms", so results was always
empty. Replies by name ("from host (ip)") and over IPv6 are read too.
Closes#12
DELETE /nodes/{node}/qemu/{vmid} starts a task and returns its UPID;
destroy_vm ignored it and returned while the VM could still exist, and
a failed delete went unnoticed. The task is now awaited. Reading the
snippets before the delete, the issue's first point, was fixed in
ecff2b4.
Closes#15
Only RSA keys were read, and a key was used only when no password was
set; with the API password always present it never was. The key is now
read as Ed25519, ECDSA or RSA and offered first, the password second.
The rest of the issue (the API exec branch, errors read as empty
output) belongs to the shared transport of NAPALM/napalm-device-types#35.
Refs #17
The getter ran "chronyc tracking || ntpq -pn" and parsed ntpq's format.
Proxmox VE ships chrony, so ntpq never ran and nothing matched. chrony
is now read in CSV mode (sources plus sourcestats); ntpq stays the
fallback and keeps peers without a tally mark.
Closes#18
The passwd filter's precedence was inverted: it listed uid < 1000
(daemon, bin, ...) and skipped regular users. The level check looked
for "Administrator" in the user record, which never names roles. The
level now comes from /access/acl (Administrator on /, by user or
group), root@pam always 15.
Closes#19
apt-get ran with "|| true" and the result was always success, so a
failed upgrade read as done. The exit status is read now and a failure
names it.
Closes#20
The rename replaced \b<old>\b, which also matches before "." and "-":
"pve.old.lan pve" became "new.dom.old.lan new.dom" and "pve-backup"
became "new-backup". Whole whitespace-separated names are renamed now,
comments left alone.
The live move of /etc/pve/nodes/<old> the issue also asks about is not
changed here; it needs a decision on clustered nodes.
Refs #21
lldpcli prints one Interface block per neighbour. The summary parser
reused the port's last entry, so a second neighbour overwrote the
first, and a neighbour without SysName was dropped. Each block now
starts an entry of its own.
Closes#22
Proxmox's sshkeys holds one OpenSSH key per line, URL-encoded. The keys
were joined with ";", so only the first one worked, and quote() kept
"/" unescaped, which Proxmox's urlencoded format refuses.
Closes#23
The write ended in `exit $s`, which ends the shell inside the braces
_rc wraps around every command, so __PVE_RC was never printed and the
step always failed. The write now shares the apt file writer, which
ends in a test instead of exit.
Closes#30
b1ba991 removed _VIRTUAL_IFACE_PREFIXES while _is_physical_uplink still
read it. The AttributeError was swallowed by _find_switch_uplink, so
set_vlan never found an uplink on any node. Restore the constant and
cover both helpers with tests.
Closes#37
Every PVE node can read its whole cluster, so get_clusters() reports the
one it belongs to: name and quorum from /cluster/status, every member
(offline ones as offline, without capacity), CPU and memory from
/cluster/resources and each online node's cpuinfo, maintenance from the
HA manager where it can be read, storages grouped by name with the nodes
they are available on, and bridges and vnets with the nodes that carry
them. A standalone node is a cluster of one.
The bridge part of get_network_targets() moved to network_targets.py so
both use it. create_vm_from_cloud_init() takes node= and refuses any node
but its own before creating anything.
Refs NetOrk/netork#484, NetOrk/netork#1822
netOrk's VM dialog follows a bridge's physical NICs to the switch port
behind it, to warn when a VM without a VLAN tag would land nowhere. The
driver knows those NICs from the node's network config.
uplinks is read from bridge_ports (Linux bridge) or ovs_ports (OVS
bridge), one group per bridge port: a bond (slaves / ovs_bonds) as its
NICs together, a NIC alone. Internal ports (OVSIntPort) and VLAN
subinterfaces leave out, as they never carry the bridge's untagged
traffic to the switch. A port missing from the config or of a kind the
driver cannot place leaves the key out rather than guessing.
shutdown_host() posts command=shutdown to the node itself; its guests
stop as on any shutdown. begin_stop_vm(name) asks a guest for its clean
shutdown and returns once Proxmox took it, without waiting for the
task: the caller reads get_vm_status until it stopped. Both raise
RuntimeError when Proxmox refuses; an unknown guest is a ValueError.
The contracts HostShutdownMixin and VmStopRequestMixin of
napalm-device-types (NAPALM/napalm-device-types#55); for netOrk's graph
workflows (NetOrk/netork#1636).
- CI moves to .gitea/workflows/ci.yml and calls the shared driver workflow
of NAPALM/driver-standard; .github/ is gone. napalm-device-types comes
from git until 4.22.0 (AptProxyMixin) is on PyPI; the floor stays at
4.22.0.
- requires-python >=3.10 (napalm-device-types needs it), py.typed, SPDX
license string, a dev extra with floors (pytest-mock and responses are
dropped: no test uses them), links to git.netork.io (README included).
pytest no longer adds --cov by itself; CI passes it.
- mypy strict for the package; exempt for now, with 204 errors between
them: napalm_proxmox.config_mixin, .driver, .interfaces_mixin,
.lldp_mixin, .routing_mixin, .sdn_mixin, .system_mixin, .utils,
.vm_contract_mixin, .vm_mixin and .vm_provision_mixin. They are recorded
in the quality baseline with the long and complex functions and the two
unannotated public methods.
- Coverage floor 68 % (measured 68.94 %).
- driver.py: the logger is defined after the imports (E402). The lazy
paramiko client is annotated Any, as _node_ssh already is; the quoted
paramiko.SSHClient named a module that is never imported there (F821).
- Unused locals are gone: get_facts' hostname, get_users' groups (both
read a value nobody used), the ARP flags column is _flags; the output is
unchanged.
- Long lines are wrapped: the apt-get and snmpd.conf commands are split
into adjacent string literals, the power-action error message and the
system mixin's docstring are wrapped; the text is unchanged.
- get_vm_status re-raises with raise ... from e (B904).
- The two except Exception: pass (closing the SSH session, the is_alive
probe) now log at debug level (B110), so a swallowed failure leaves a
trace.
- nosec for the empty "password" field of NAPALM's get_users shape and
the ssh_password_auth flag (B105); all three are false positives.
- tests: unused loop variables.
Mechanical: ruff format, and ruff check --fix with the rule set every NAPALM
driver uses (E, F, I, UP, B at line length 100, Python 3.10). What ruff
cannot fix comes in the next commit.