The mktemp write of /etc/hosts made set_hostname one line shorter, and
the quality ratchet fails on a baseline entry that is no longer reached
(stale). Lowered with napalm-standard ratchet --write.
napalm-device-types 4.36.0 (released after main's last CI run) changed
PosixTimeSyncMixin: get_time_sync() reports DHCP-provided servers apart
(dhcp_ntp_servers), and configure_time_sync() no longer comments out the
DHCP client's chrony sourcedir; it gives back one that 4.35.0 took out
(NetOrk/netork#2380). CI installs the newest release, so five time-sync
tests failed on every branch.
The tests now expect that behaviour, read DHCP's servers apart, and check
that a node 4.35.0 aligned gets its DHCP sourcedir back. The floor moves
to 4.36.0, so the lowest-versions job tests the same behaviour; README,
CHANGELOG and the module docstring say what a write leaves alone.
napalm-device-types 4.36.0 (released after main's last CI run) changed
PosixTimeSyncMixin: get_time_sync() reports DHCP-provided servers apart
(dhcp_ntp_servers), and configure_time_sync() no longer comments out the
DHCP client's chrony sourcedir; it gives back one that 4.35.0 took out
(NetOrk/netork#2380). CI installs the newest release, so five time-sync
tests failed on every branch.
The tests now expect that behaviour, read DHCP's servers apart, and check
that a node 4.35.0 aligned gets its DHCP sourcedir back. The floor moves
to 4.36.0, so the lowest-versions job tests the same behaviour; README,
CHANGELOG and the module docstring say what a write leaves alone.
The new /etc/hosts was written as root to the fixed /tmp/_netork_hosts
and moved over /etc/hosts. A local user could create that file (or a
symlink) beforehand: root's write went into the user's file, and the
user owned /etc/hosts afterwards (NetOrk/netork#2446).
The file is now made with mktemp beside the target (/etc/.hosts.XXXXXX,
root's directory, same filesystem), written, set to 644 and moved into
place; a failed step removes it.
`postconf -e 'myhostname = <fqdn>'`, `hostname '<name>'` and the
/etc/pve/nodes move put the name between single quotes of their own, so
a name with a single quote ended them and the rest ran as root
(NetOrk/netork#2456). Each value is now passed through shlex.quote.
/etc/hostname, /etc/hosts and /etc/mailname were already written
through base64.
netOrk keeps one list of NTP servers and a time zone per site and aligns
a device that differs (NetOrk/netork#2360, #2365). Both drivers now
implement napalm-device-types' TimeSyncMixin (4.35.0) and write both
values.
Neither API keeps time servers: /nodes/{node}/time knows only the zone,
and both products run chrony on Debian. The servers are therefore
PosixTimeSyncMixin's, over the driver's existing SSH session
(run_command, root): read from chrony's configuration, written to
/etc/chrony/sources.d/netork.sources with every other source commented
out, then a reload or a restart of chrony.
The zone is read with them from the host (timedatectl: the zone the
clock really runs in), and from GET /nodes/{node}/time only where the
host names none. It is written through PUT /nodes/{node}/time rather
than timedatectl set-timezone: Proxmox's own call validates the name and
keeps /etc/timezone, which its web UI and API show, in step with
/etc/localtime.
One device is one node: in a PVE cluster every node has its own clock,
and only the node this connection resolved is read and written.
get_ntp_servers() read GET /nodes/{node}/ntp, a path Proxmox VE does not
have, so it always reported no servers. It now reports what
get_time_sync() reads, on the backup server too.
netOrk holds a host's restart after patching while a backup runs on it, and
a guest's restart while its hypervisor is backing it up (NetOrk/netork#2318).
It asks right before the restart, so both drivers answer
get_running_backups() from napalm-device-types 4.34.0:
- PVE: the node's active vzdump tasks, each with the guests it is taking now
(its own guest, or for a job the guests locked for their backup), and a
guest locked without a task on its own, since the task list can lag and a
lock can outlive its task.
- PBS: the running backup, sync, verify, gc, prune and tape tasks; a restore
or housekeeping is no backup.
A failed read raises, so netOrk reads it as "not checked", never as "none".
Both drivers mix in napalm-device-types' LogSourcesMixin (4.30.0) and
carry its command over the exec path, which runs as root either way.
Refs NetOrk/netork#2049
The guard is not about the link: napalm-device-types now writes the file a
link names. /etc/cron.d/vzdump points into /etc/pve, the cluster file
system, where installing a file with a mode and owner is not safe.
Refs #56
Both drivers mix in napalm-device-types' PosixScheduledJobsMixin over the
node's root SSH session: get_scheduled_jobs reads cron, run-parts, anacron
and every systemd timer in one round trip; manage_scheduled_job enables,
disables or runs one now; put_ and delete_scheduled_job write and remove an
application's own job.
What Proxmox ships is platform, so it is only switched and run: pve-*
units, pvesr.timer and /etc/cron.d/vzdump on Proxmox VE, proxmox-backup*
units on a backup server. A vzdump cron line is only run, because the file
is a link into /etc/pve and rewriting it would replace the link with a
plain file.
Backup jobs (/cluster/backup), replication and a backup server's verify,
prune, sync and GC jobs run in Proxmox's own daemons and stay with the
backup features.
Requires napalm-device-types 4.26.0.
Closes#56
get_clusters() lists a pbs storage like any other, so nothing told an
application which Proxmox Backup Server and datastore it writes to.
Each pbs storage now carries ClusterStorageDict.backup_server
(napalm-device-types 4.25.0): server, port, datastore, namespace and
fingerprint from /storage, never the password or encryption key. An
unreadable /storage leaves it None; other storages carry no such key.
This was step 6 of #52 and was left out there.
A second driver in this package, ProxmoxBackupDriver. Its API and web UI
are on 8007, not on PVE's 8006, and its node is always localhost; open()
proves the login with GET /version, so a bad token fails there.
Roles: BackupServerDriver (napalm-device-types 4.25.0) first, then
StorageDriver for the disks and ZFS pools the server manages. It reads
datastores, the backup groups of every namespace with their newest
snapshot, a group's snapshots, the GC, verify, prune, sync and tape jobs
with how their last run ended, and the remotes without their secrets;
it starts garbage collection and jobs, verifies, protects and deletes
snapshots, and follows the tasks. As a host it reports facts,
interfaces, CPU and memory, disks, pools, its pbs-subscription and the
same packages, updates, host readers and sshd hardening as a PVE node.
A privilege-separated token without an ACL entry gets an empty list from
PBS, not a refusal, so every read checks /access/permissions and raises
PermissionError. A removable datastore that is not mounted is reported
as unmounted and left out of the group listing.
The fixture is what a PBS 4.2 answered, sanitised; groups, snapshots and
jobs it did not have yet follow the API schema.
Proxmox Backup Server speaks the same REST dialect, logs in the same way
and runs on Debian too, so what a second driver needs moves out of
ProxmoxDriver first:
- napalm_proxmox._base: the connection (proxmoxer service PVE or PBS,
each with its port), the token split, the SSH session, run_command,
the node helpers and the DMI read.
- napalm_proxmox.host_mixin: packages and APT updates, the subscription,
restart and shutdown through the API, root file writes and the
transports of the host readers (kernel, listening sockets, identity,
systemd, host status, apt's proxy).
The PVE driver behaves as before; what it hands proxmoxer is pinned by
tests first, and it now names service="PVE" explicitly. get_packages is
split so the dpkg line parser stands alone; get_facts and get_packages
leave the quality baseline.
A Proxmox node runs Debian's stock OpenSSH, so OpenSshServerMixin works
there as on a plain Linux host. What it needed was run_command: an exec
channel on the node's SSH session, via napalm-device-types'
run_on_transport. The USES_SSH comment said there was no SSH session;
it now says what the flag means to netOrk.
_write_snippet put the snippet storage's path and the snippet's file
name unquoted into the command that writes the snippet as root on the
node; a space, ; or $(...) in either was acted on by the shell. Both now
go through shlex.quote, and a file name that is not one plain name
(a path, a shell character) is refused before anything runs.
Closes#47
paramiko's look_for_keys and allow_agent stayed at their default True,
so the node login also tried every key in the running process's own
~/.ssh and any SSH agent it could reach. The driver standard lets a
driver authenticate only with what it was handed; both are now False
on every connect.
Closes#48
lldpcli's text was read with 2>/dev/null || true, so lldpd not running,
lldpctl missing or the command not running at all came back as a node
without neighbours. Capabilities were lldpd's symbols lower-cased (tel,
wlan, docsis) rather than NAPALM's names.
The node now runs napalm-device-types' LLDPCTL_KEYVALUE_COMMAND, whose
output carries lldpctl's exit status; parse_lldpctl_keyvalue raises on a
failure, an empty answer raises too, and the lldp_neighbors projections
give NAPALM's capability names and the chassis MAC. The module is under
strict mypy now; the floor on napalm-device-types rises to 4.24.0.
Closes#44
_exec_ssh_command returned "" on any failure (no SSH credential, a
refused login or host key, an unreachable node, a timeout), so a
getter that could not look read like one that found nothing; on
API-token-only nodes get_snmp_config answered None and the SSH-backed
getters empty lists on every poll. It now raises ConnectionException
with the cause (SshHostKeyMismatch as it is), and refuses before
connecting when there is neither a password nor a usable key.
It returned stderr when stdout was empty; output is stdout now. It
first posted every command to /nodes/{node}/execute, which runs a
batch of API calls rather than a shell; that dead round trip is gone.
The key types, the key before the password, the host-key check and the
single connect path were done in f491f9f and 6bb3035. The shared
transport is NAPALM/napalm-device-types#35.
Closes#17
The cloud-image download put image_url, hand-wrapped in single quotes,
and the cache path built from its basename unquoted into commands that
run as root on the node: a quote, $(...), ; or space in a URL from a
release feed was acted on by the shell. The URL and every path built
from it now go through shlex.quote (exists check, wget, mv, checksum,
rm, unpack, qm importdisk), and the cache file name keeps only the
characters Proxmox keeps in a content file name, as the import volume
name already did. The checksum algorithm is checked against the ones
Proxmox knows, since it is part of the command name.
The file-name helpers move to image_names.py; vm_provision_mixin.py
would otherwise cross the 1000-line limit. _download_cloud_image
drops below 50 lines, so its baseline entry goes.
Closes#43
get_clusters() reads /cluster/ha/* (PVE 8 and 9), /cluster/backup with pool
members, the newest backup per guest on each backup storage (PBS included) and
/cluster/config/nodes votes; each read is best-effort. migrate_vm() moves a
guest of the connected node and follows the task (HA-managed guests through
the HA manager), raising with the task log's ERROR lines;
get_vm_migration_preflight() maps PVE's own pre-flight; set_node_maintenance()
runs ha-manager over SSH.
Part of NetOrk/netork#1820 (stage 7: #759, #760, #761).
A failure after the VM shell existed (disk import, NIC or snippet config,
start) raised without the VMID, so the caller could not remove the
half-built VM and it stayed on the node. It now raises napalm-device-types'
VMProvisionIncomplete with vmid and node, the failure as its cause. A
failure before the shell exists raises as before.
The NIC config strings are built by a helper of their own, which keeps
create_vm_from_cloud_init within its quality baseline.
Closes#28
The rename moves /etc/pve/nodes/<old> while the node still runs as <old>,
which Proxmox does not support for a cluster member. set_hostname now
checks for a corosync.conf first and raises before touching anything on
a member, or when the check cannot be read; a single node renames as
before.
The check is the file, not `pvecm status`: pvecm decides membership by
/etc/pve/corosync.conf itself, and its status also fails on a member
whose corosync is down. /etc/corosync/corosync.conf covers a member
whose /etc/pve is not mounted.
Closes#21
wget ran with -q, so a failure carried no reason, and a failed
download left its partial .tmp in the cache, taking the same space
again on every retry. wget now runs with -nv and the .tmp is removed
on failure, keeping wget's exit status.
Closes#3
The probe pattern expected "from" after icmp_seq, but iputils prints
"bytes from <ip>: icmp_seq=... time=... ms", so results was always
empty. Replies by name ("from host (ip)") and over IPv6 are read too.
Closes#12