feat(pbs): Proxmox Backup Server as entry point proxmox_backup (#52) #53

Merged
christianmanivong merged 2 commits from feat/proxmox-backup into main 2026-11-02 21:35:53 +00:00
20 changed files with 4567 additions and 605 deletions

No files matched your search

+20 -2
View File
@@ -8,6 +8,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Added
- **Proxmox Backup Server** as a second driver of this package, entry point
`proxmox_backup` (`ProxmoxBackupDriver`, #52). Its API is on **8007**, not 8006; the
node is always `localhost`. Roles: `BackupServerDriver` (napalm-device-types 4.25.0)
first, `StorageDriver` second. It reads datastores (usage, estimated full date,
backend, maintenance or `unmounted`), backup groups in every namespace with their
newest snapshot and its verify state, a group's snapshots, the GC, verify, prune,
sync and tape jobs with the state of their last run, and the remotes without their
secrets; it starts garbage collection and jobs, verifies, protects and deletes
snapshots, and follows the tasks. As a host it reports facts, interfaces, CPU and
memory, disks and ZFS pools, its `pbs-subscription`, packages and updates, and has
the same host readers and sshd hardening as a PVE node.
- A token without an ACL entry gets an empty list from PBS rather than a refusal; every
read checks `/access/permissions` first and raises `PermissionError`.
- The node's sshd can be hardened: `OpenSshServerMixin` (napalm-device-types) gives
`harden_ssh_server` and `exempt_from_ssh_penalties`, so netOrk's SSH algorithm fix
works on a Proxmox node as on a plain Linux host. A node runs Debian's stock
@@ -18,8 +31,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
root; a command that cannot run or outlasts its timeout raises `ConnectionException`.
### Changed
- The floor on napalm-device-types is 4.24.0, the first release with what this driver
uses (its shared `lldpctl` parser).
- The floor on napalm-device-types is 4.25.0, the first release with
`BackupServerDriver` (4.24.0 brought the shared `lldpctl` parser).
- What the two drivers share moved out of `ProxmoxDriver`: the connection, token split,
SSH session and node helpers into `napalm_proxmox._base`, the Debian host's packages,
updates, subscription and the host readers' transports into
`napalm_proxmox.host_mixin`. The PVE driver behaves as before, pinned by tests first;
it now names proxmoxer's `service="PVE"` explicitly.
- Python 3.12 is the floor (`requires-python = ">=3.12"`): netOrk runs every driver
on 3.12, so 3.10 and 3.11 are no longer tested (NAPALM/driver-standard#6).
+70 -5
View File
@@ -1,6 +1,9 @@
# napalm-proxmox
A [NAPALM](https://napalm.readthedocs.io/) driver for [Proxmox VE](https://www.proxmox.com/en/proxmox-virtual-environment/overview) nodes.
[NAPALM](https://napalm.readthedocs.io/) drivers for [Proxmox VE](https://www.proxmox.com/en/proxmox-virtual-environment/overview) nodes
(`proxmox`) and for [Proxmox Backup Server](https://www.proxmox.com/en/proxmox-backup-server/overview) (`proxmox_backup`).
The Proxmox VE driver is described first; the backup server's is
[below](#proxmox-backup-server).
It supports all three networking domains of Proxmox VE:
@@ -146,6 +149,60 @@ What the API does not offer (ARP, LLDP, packages, SNMP config, `cli`, …) runs
node's shell over SSH. Without an SSH password or key — a node reached with an API
token only — those calls raise `ConnectionException` instead of reading as empty.
## Proxmox Backup Server
```python
driver = get_network_driver("proxmox_backup")
device = driver(
hostname="pbs1.example.com",
username="",
password="",
optional_args={
"token_name": "root@pam!monitoring",
"token_value": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
"ssh_private_key_str": open("id_ed25519").read(), # optional, see below
},
)
```
**The API and web UI listen on 8007**, not on Proxmox VE's 8006; `port` defaults to
8007 here. The node is always `localhost`: a backup server has no cluster. The
optional arguments are the ones above, except `node`.
Roles (napalm-device-types): `BackupServerDriver` first, `StorageDriver` second for
the disks and ZFS pools the server manages itself.
| Method | Source |
|--------|--------|
| `get_backup_datastores` | `/config/datastore` + `/status/datastore-usage`: usage, estimated full date, backend (`local`, `s3`, `removable`), maintenance or `unmounted` |
| `get_backup_groups` / `get_backup_snapshots` | `/admin/datastore/{store}/{namespace,groups,snapshots}`, every namespace; a group carries its newest snapshot with its verify state |
| `get_backup_jobs` | `/admin/{gc,verify,prune,sync}`, `/tape/backup`, with the state of the last run |
| `get_backup_remotes` | `/config/remote`, without secrets |
| `start_garbage_collection`, `run_backup_job`, `verify_backup_snapshot` | start a task; `get_backup_task` follows it |
| `set_backup_snapshot_protected`, `delete_backup_snapshot` | on one snapshot |
| `get_facts`, `get_interfaces`, `get_interfaces_ip`, `get_environment` | the API; host name, DMI and MAC addresses over SSH |
| `get_disks`, `get_disk_pools` | `/nodes/localhost/disks/{list,zfs}` |
| `get_licenses` | `pbs-subscription` |
| packages, updates, services, kernel, listening sockets, sshd hardening, `reboot_host`, `shutdown_host` | as on Proxmox VE |
**API token rights.** Reading needs the role `Audit` on `/` (`Sys.Audit`,
`Datastore.Audit`); the remotes need `Remote.Audit` on `/remote` as well, which
`Audit` does not include. A privilege-separated token needs an ACL entry of its own:
```bash
proxmox-backup-manager acl update / Audit --auth-id 'root@pam!monitoring'
```
Without it PBS does not refuse a listing, it answers with an empty one. The driver
therefore checks `/access/permissions` before every read and raises
`PermissionError`, so a token without rights does not read as a server without
backups. The actions need `DatastoreAdmin` on `/datastore` (GC, verify, prune,
protect, delete) and `Sys.Modify` / `Sys.PowerManagement` on `/system` (services,
reboot).
A removable datastore that is not mounted is reported as `maintenance: "unmounted"`
and left out of `get_backup_groups()`; asked for by name, it raises `ValueError`.
## Development
```bash
@@ -164,9 +221,14 @@ ruff check napalm_proxmox
```
napalm_proxmox/
├── __init__.py # Exports ProxmoxDriver
├── driver.py # ProxmoxDriver — all NAPALM methods
└── utils.py # MAC normalisation, CIDR parsing, ARP/OVS parsers
├── __init__.py # Exports ProxmoxDriver and ProxmoxBackupDriver
├── _base.py # What both share: connection, token split, SSH session, node helpers
├── host_mixin.py # Both as Debian hosts: apt, packages, subscription, host readers' transports
├── driver.py # ProxmoxDriver (proxmox)
├── backup_driver.py # ProxmoxBackupDriver (proxmox_backup)
├── pbs_backups.py # The backup-server role: datastores, groups, jobs, remotes, actions
├── pbs_host.py # The backup server as a host: facts, interfaces, disks, ZFS pools
└── utils.py # MAC normalisation, CIDR parsing, ARP/OVS parsers
tests/
├── conftest.py # Shared fixtures and mock API payloads
├── test_connection.py # open / close / is_alive
@@ -176,7 +238,10 @@ tests/
├── test_sdn.py # get_vlans, get_network_instances
├── test_ovs_and_arp.py # get_arp_table, get_mac_address_table, ARP/OVS utils
├── test_misc.py # NTP, SNMP, users, config, ping, traceroute, CLI, LLDP
└── test_utils.py # Unit tests for napalm_proxmox.utils
├── test_utils.py # Unit tests for napalm_proxmox.utils
├── fixtures/pbs_api.json # What a Proxmox Backup Server 4.2 answered (sanitised)
├── pbs_fake.py # A proxmoxer stand-in answering from that recording
└── test_pbs_*.py # The backup server driver
```
## License
+3 -2
View File
@@ -1,5 +1,6 @@
"""NAPALM driver for Proxmox VE."""
"""NAPALM drivers for Proxmox VE and Proxmox Backup Server."""
from napalm_proxmox.backup_driver import ProxmoxBackupDriver
from napalm_proxmox.driver import ProxmoxDriver
__all__ = ["ProxmoxDriver"]
__all__ = ["ProxmoxBackupDriver", "ProxmoxDriver"]
+408
View File
@@ -0,0 +1,408 @@
"""What the Proxmox VE and the Proxmox Backup Server drivers share.
Both products speak the same REST dialect (``/api2/json``, ``/nodes/{node}/...``),
authenticate the same way (``user@realm`` with a password, or an API token
``user@realm!name``) and run on Debian, where the driver opens its own SSH
session as root for what the API does not offer. They differ in the service
proxmoxer talks to, its port (PVE 8006, PBS 8007) and how the node is found.
Optional args
-------------
verify_ssl : bool
Verify TLS certificates (default: True). ``ssl_verify`` and ``verify`` are
accepted as well.
port : int
API port (default: the service's, 8006 for PVE, 8007 for PBS).
realm : str
Realm a bare username logs in to (default: ``pam``).
token_name : str
API token name (e.g. ``root@pam!netork``).
token_value : str
API token secret. With both token_name and token_value, the token is used
instead of the password.
ssh_username, ssh_password, ssh_private_key_str : str
The node's SSH login; the API login's username and password otherwise.
"""
from __future__ import annotations
import io
import logging
from typing import Any
from napalm.base.exceptions import ConnectionException
from napalm_device_types import CommandResult
from napalm_device_types.channel import run_on_transport
from napalm_device_types.ssh_host_keys import HostKeyCheck
logger = logging.getLogger(__name__)
_JsonDict = dict[str, Any]
#: The API port of each proxmoxer service this package drives.
DEFAULT_PORTS = {"PVE": 8006, "PBS": 8007}
class ProxmoxBase:
"""Connection, SSH session and node helpers of a Proxmox product's driver.
A concrete driver sets :attr:`SERVICE`, says how it finds its node
(:meth:`_resolve_node`) and constructs the proxmoxer client
(:meth:`_proxmox_api`), so its own module names the client class.
"""
#: The proxmoxer service: ``"PVE"`` or ``"PBS"``.
SERVICE = "PVE"
def __init__(
self,
hostname: str,
username: str,
password: str,
timeout: int = 60,
optional_args: _JsonDict | None = None,
) -> None:
self.hostname = hostname
self.username = username
self.password = password
self.timeout = timeout
self.optional_args: _JsonDict = optional_args or {}
self._port: int = self.optional_args.get("port", DEFAULT_PORTS[self.SERVICE])
verify: Any = self.optional_args.get(
"verify_ssl",
self.optional_args.get("ssl_verify", self.optional_args.get("verify", True)),
)
self._verify_ssl: bool = verify
self._realm: str = self.optional_args.get("realm", "pam")
self._token_name: str | None = self.optional_args.get("token_name")
self._token_value: str | None = self.optional_args.get("token_value")
self._ssh_username: str | None = self.optional_args.get("ssh_username")
self._ssh_password: str | None = self.optional_args.get("ssh_password")
self._ssh_key: str | None = self.optional_args.get("ssh_private_key_str")
self._api: Any = None # proxmoxer.ProxmoxAPI
self._node_name: str = ""
# paramiko.SSHClient; paramiko is imported lazily (see _node_ssh).
self._ssh_client: Any = None
# ------------------------------------------------------------------ #
# Connection management
# ------------------------------------------------------------------ #
def _proxmox_api(self, **kwargs: Any) -> Any:
"""The proxmoxer client for *kwargs*; the concrete driver's module names the class."""
raise NotImplementedError
def _resolve_node(self) -> str:
"""Find the node this connection talks to and set ``_node_name``.
API errors propagate, so that ``open()`` fails with the real cause.
"""
raise NotImplementedError
def _api_kwargs(self) -> _JsonDict:
# Use the HTTPS backend for proper REST API support.
# The openssh backend tunnels all kwargs through to
# openssh_wrapper.CommandBaseSession, which does not
# accept password/verify_ssl/token params.
# Proxmox authenticates against "<user>@<realm>" and rejects a bare
# username outright. A caller who typed a realm keeps it; one who
# did not gets self._realm, which is what the documented `realm`
# optional_arg is for -- it was read in __init__ and then never
# used, so the option had no effect and a bare username failed.
user = self.username or ""
if user and "@" not in user:
user = f"{user}@{self._realm}"
kwargs: _JsonDict = {
"host": self.hostname,
"user": user,
"password": self.password,
"port": self._port,
"verify_ssl": self._verify_ssl,
"backend": "https",
"service": self.SERVICE,
}
if self._token_name and self._token_value:
kwargs.pop("password", None)
kwargs["token_value"] = self._token_value
# token_name may be in "<user>!<tokenid>" format (e.g.
# "root@pam!netork"). proxmoxer expects them split:
# user="root@pam", token_name="netork"
user_part, _, token_id = self._token_name.partition("!")
if token_id:
kwargs["user"] = user_part
kwargs["token_name"] = token_id
else:
kwargs["token_name"] = self._token_name
return kwargs
def open(self) -> None:
"""Open the connection to the Proxmox API."""
try:
self._api = self._proxmox_api(**self._api_kwargs())
# Validate the connection: finding the node needs a working login.
self._resolve_node()
except Exception as exc:
raise ConnectionException(f"Cannot connect to {self.hostname}: {exc}") from exc
def close(self) -> None:
"""Close the connection."""
self._api = None
if self._ssh_client:
try:
self._ssh_client.close()
except Exception:
logger.debug("%s: closing the SSH session: ignored", self.hostname, exc_info=True)
self._ssh_client = None
def is_alive(self) -> _JsonDict:
"""Return connection liveness.
Probes ``GET /version``, the cheapest endpoint that proves the session
still authenticates. It used to call ``_resolve_node()``, which returns
early without touching the API whenever a node was configured via
optional_args — so a dead connection reported itself alive.
"""
alive = False
if self._api:
try:
self._api.version.get()
alive = True
except Exception:
logger.debug("%s: liveness probe failed", self.hostname, exc_info=True)
return {"is_alive": alive}
# ------------------------------------------------------------------ #
# Internal API helpers
# ------------------------------------------------------------------ #
def _node_api(self) -> Any:
"""Return the API resource for the current node."""
return self._api.nodes(self._node_name)
def _get_node_network(self) -> list[_JsonDict]:
"""Return the node's network interface list from the Proxmox API."""
try:
return self._node_api().network.get() or []
except Exception as exc:
logger.debug("Failed to fetch node network: %s", exc)
return []
def _exec_ssh_command(self, command: str) -> str:
"""Run *command* in the node's shell over SSH and return what it printed.
Stdout only: stderr is not output (a command that wants it says
``2>&1``). The exit status is not checked; a caller that needs it asks
for it (``_rc``). The node has no API endpoint that runs a shell command
(``/nodes/{node}/execute`` runs a batch of API calls), so this is the
one way, the same session ``_run_node_command`` uses (see
NAPALM/napalm-device-types#35 for the shared transport).
:raises ConnectionException: when the command could not run: no SSH
credential, a refused login or host key (``SshHostKeyMismatch``),
an unreachable node, a timeout.
"""
try:
_, stdout, stderr = self._node_ssh().exec_command(command, timeout=self.timeout)
out: str = stdout.read().decode(errors="replace")
err: str = stderr.read().decode(errors="replace").strip()
except ConnectionException:
raise
except _ssh_errors() as exc:
raise ConnectionException(
f"Cannot run a command on {self.hostname} over SSH: {exc}"
) from exc
if err:
logger.debug("%s: stderr of a node command: %s", self.hostname, err[-300:])
return out.strip()
def run_command(
self,
command: str,
*,
privileged: bool = False,
timeout: float = 60,
stdin: bytes | None = None,
) -> CommandResult:
"""Run *command* on an exec channel of the node's SSH session (``CommandChannelMixin``).
stdout, stderr and the exit status come back apart; *stdin*, if given, is
written and then closed. *privileged* needs nothing more: the node is
logged in as root, as for every other command this driver runs there.
:raises ConnectionException: when the command could not run, or did not
finish within *timeout* seconds.
"""
try:
transport = self._node_ssh().get_transport()
return run_on_transport(transport, command, stdin=stdin, timeout=timeout)
except ConnectionException:
raise
except _ssh_errors() as exc:
raise ConnectionException(
f"Cannot run a command on {self.hostname} over SSH: {exc}"
) from exc
def _node_ssh(self) -> Any: # paramiko.SSHClient; paramiko is imported lazily
"""The SSH session to the node on port 22, opened once.
Its host key is the one the caller expects for port 22, or on first use
whichever the node presents; either way it is reported (netork#1167).
:raises ConnectionException: without a password or a usable key.
"""
import paramiko
if self._ssh_client is None:
# A key is tried first; paramiko falls back to the password. Only
# these: not the keys in the process's own ~/.ssh, nor an agent.
connect_kwargs: _JsonDict = {
"username": self._ssh_username or self.username,
"timeout": self.timeout,
"look_for_keys": False,
"allow_agent": False,
}
password = self._ssh_password or self.password
if password:
connect_kwargs["password"] = password
pkey = _load_private_key(self._ssh_key) if self._ssh_key else None
if pkey is not None:
connect_kwargs["pkey"] = pkey
if not password and pkey is None:
# An API-token-only node: nothing to log in with, so no connection.
raise ConnectionException(
f"no SSH credential for {self.hostname}: set ssh_password or "
"ssh_private_key_str"
)
client = paramiko.SSHClient()
try:
HostKeyCheck(self.optional_args, self.hostname, 22).connect(
client, **connect_kwargs
)
except Exception:
client.close()
raise
self._ssh_client = client
return self._ssh_client
# ------------------------------------------------------------------ #
# Node info helpers
# ------------------------------------------------------------------ #
def _get_version_info(self) -> _JsonDict:
"""Return the product's version info from the API."""
try:
return self._api.version.get() or {}
except Exception as exc:
logger.debug("Failed to fetch version info: %s", exc)
return {}
def _get_node_status(self) -> _JsonDict:
"""Return the node's status from the Proxmox API."""
try:
return self._node_api().status.get() or {}
except Exception as exc:
logger.debug("Failed to fetch node status: %s", exc)
return {}
def _get_node_subscription(self) -> _JsonDict:
"""Return subscription status for this node."""
try:
return self._node_api().subscription.get() or {}
except Exception as exc:
logger.debug("Failed to fetch node subscription: %s", exc)
return {}
def _get_node_dns(self) -> _JsonDict:
"""Return DNS configuration for this node."""
try:
return self._node_api().dns.get() or {}
except Exception as exc:
logger.debug("Failed to fetch node DNS: %s", exc)
return {}
def _get_node_time(self) -> _JsonDict:
"""Return time configuration for this node."""
try:
return self._node_api().time.get() or {}
except Exception as exc:
logger.debug("Failed to fetch node time: %s", exc)
return {}
def _hardware(self) -> tuple[str, str, str]:
"""The physical machine's vendor, model and serial number, from the Linux DMI
sysfs entries over SSH; empty strings where they cannot be read."""
# Physical hardware info from Linux DMI sysfs.
# Read each field separately to avoid shell quoting issues with printf.
# Field priority for model:
# product_name — human-readable name on most vendors (e.g. "NUC6CAYH",
# "ThinkCentre M910x")
# product_version — sometimes the marketing name on Lenovo; on Intel NUC
# it is the board part number (less useful as model name)
# We prefer product_name; fall back to product_version only when
# product_name looks like a raw type code (all uppercase + digits, no spaces).
vendor = ""
model = ""
serial = ""
try:
dmi_cmd = (
"v=$(cat /sys/class/dmi/id/sys_vendor 2>/dev/null); "
"n=$(cat /sys/class/dmi/id/product_name 2>/dev/null); "
"r=$(cat /sys/class/dmi/id/product_version 2>/dev/null); "
"s=$(cat /sys/class/dmi/id/product_serial 2>/dev/null); "
'printf \'%s\\n%s\\n%s\\n%s\\n\' "$v" "$n" "$r" "$s"'
)
lines = self._exec_ssh_command(dmi_cmd).splitlines()
if len(lines) >= 4:
vendor = lines[0].strip()
product_name = lines[1].strip()
product_version = lines[2].strip()
serial = lines[3].strip()
_bad = {"none", "n/a", "not specified", "to be filled by o.e.m."}
pv_usable = (
product_version
and product_version.lower() not in _bad
and product_version != product_name
# Only prefer product_version when it contains a space —
# that indicates a human-readable marketing name like
# "ThinkCentre M910x" rather than a part code like "J26843-409".
and " " in product_version
)
model = product_version if pv_usable else product_name
except Exception as exc:
logger.debug("Failed to read DMI info via SSH: %s", exc)
return vendor, model, serial
# ------------------------------------------------------------------ #
# CLI passthrough
# ------------------------------------------------------------------ #
def cli(self, commands: list[str], encoding: str = "text") -> dict[str, str]:
"""Execute a list of shell commands and return their outputs."""
return {cmd: self._exec_ssh_command(cmd) for cmd in commands}
def _load_private_key(text: str) -> Any: # paramiko.PKey; paramiko is imported lazily
"""The SSH private key in *text*, whatever its type, or None when it cannot be read."""
import paramiko
for key_class in (paramiko.Ed25519Key, paramiko.ECDSAKey, paramiko.RSAKey):
try:
return key_class.from_private_key(io.StringIO(text))
except (paramiko.SSHException, ValueError):
continue
logger.warning("The SSH private key could not be read; logging in with the password")
return None
def _ssh_errors() -> tuple[type[Exception], ...]:
"""What a command that could not run raises: the socket's errors, timeouts
included, and paramiko's (a refused login, a closed session)."""
import paramiko
return (OSError, paramiko.SSHException)
+105
View File
@@ -0,0 +1,105 @@
"""NAPALM driver for Proxmox Backup Server (entry point ``proxmox_backup``).
A PBS is a Debian host with one node, always ``localhost`` in its API, and no
cluster. Its API and web UI listen on **8007**, not on PVE's 8006. The
connection, the SSH session and the node helpers are the ones the PVE driver
uses (``napalm_proxmox._base``).
Roles: a backup server first (datastores, backup groups, jobs, remotes), and
storage second for the disks and ZFS pools it manages itself.
"""
from __future__ import annotations
import logging
from typing import Any
from napalm_device_types import (
AptProxyMixin,
BackupServerDriver,
EnsurePackageMixin,
FingerprintRule,
HostStatusMixin,
KernelFactsMixin,
LicenseMixin,
ListeningSocketsMixin,
OpenSshServerMixin,
PortSpec,
ServicePackagesMixin,
StorageDriver,
SystemdServicesMixin,
SystemIdentityMixin,
)
try:
from proxmoxer import ProxmoxAPI
except ImportError as exc: # pragma: no cover - proxmoxer is a dependency
raise ImportError("proxmoxer is required: pip install proxmoxer") from exc
from napalm_proxmox._base import ProxmoxBase
from napalm_proxmox.host_mixin import ProxmoxHostMixin
from napalm_proxmox.pbs_backups import ProxmoxBackupServerMixin
from napalm_proxmox.pbs_host import ProxmoxPbsHostMixin
logger = logging.getLogger(__name__)
#: The node name the PBS API answers to; a PBS has exactly one.
PBS_NODE = "localhost"
class ProxmoxBackupDriver(
ProxmoxBase,
ProxmoxBackupServerMixin,
ProxmoxPbsHostMixin,
ProxmoxHostMixin,
BackupServerDriver,
StorageDriver,
KernelFactsMixin,
ServicePackagesMixin,
ListeningSocketsMixin,
SystemIdentityMixin,
SystemdServicesMixin,
HostStatusMixin,
LicenseMixin,
EnsurePackageMixin,
AptProxyMixin,
OpenSshServerMixin,
):
"""NAPALM driver for Proxmox Backup Server."""
SERVICE = "PBS"
VENDOR = "Proxmox"
DRIVER_NAME = "proxmox_backup"
# netOrk reaches the server through the REST API and does not log in over
# SSH itself; the driver's own SSH session (_node_ssh) runs the shell.
USES_SSH = False
#: A PBS upgrades like a PVE node: a plain ``apt-get upgrade`` holds back
#: what needs new or removed packages.
FULL_UPGRADE = True
PORT_SPECS = [
PortSpec("https", 8007, weight=8.0),
]
SSH_FINGERPRINT = [
FingerprintRule("debian", weight=3.0),
]
HTTP_FINGERPRINT = [
FingerprintRule("proxmox backup server", weight=9.0, mandatory=True),
FingerprintRule("proxmox", weight=5.0),
FingerprintRule("pbs", weight=2.0),
]
platform = "proxmox_backup"
PACKAGE_SOURCE = "pbs"
SUBSCRIPTION_LICENSE = "pbs-subscription"
def _proxmox_api(self, **kwargs: Any) -> Any:
return ProxmoxAPI(**kwargs)
def _resolve_node(self) -> str:
"""The node is always ``localhost``; ``GET /version`` proves the login.
proxmoxer logs in lazily with a token, so without a request here a bad
token would only fail at the first getter.
"""
self._api.version.get()
self._node_name = PBS_NODE
return self._node_name
+12 -334
View File
@@ -12,19 +12,11 @@ NAPALM API allows it.
Optional args
-------------
verify_ssl : bool
Verify TLS certificates (default: True).
port : int
Proxmox API port (default: 8006).
The connection's are the shared base's (``napalm_proxmox._base``; the API port
defaults to 8006 here). In addition:
node : str
Override the target node name (default: auto-detected from hostname).
realm : str
PAM realm (default: ``pam``).
token_name : str
API token name (e.g. ``napalm@pam!mytoken``).
token_value : str
API token secret. When both token_name and token_value are provided,
token-based auth is used instead of password auth.
"""
from __future__ import annotations
@@ -35,7 +27,6 @@ from typing import Any
from napalm.base.exceptions import ConnectionException
from napalm_device_types import (
AptProxyMixin,
CommandResult,
EnsurePackageMixin,
FingerprintRule,
HostStatusMixin,
@@ -50,16 +41,16 @@ from napalm_device_types import (
SystemdServicesMixin,
SystemIdentityMixin,
)
from napalm_device_types.channel import run_on_transport
from napalm_device_types.ssh_host_keys import HostKeyCheck
try:
from proxmoxer import ProxmoxAPI
except ImportError as exc:
raise ImportError("proxmoxer is required: pip install proxmoxer") from exc
from napalm_proxmox._base import ProxmoxBase
from napalm_proxmox.cluster_mixin import ProxmoxClusterMixin
from napalm_proxmox.config_mixin import ProxmoxConfigMixin
from napalm_proxmox.host_mixin import ProxmoxHostMixin
from napalm_proxmox.interfaces_mixin import ProxmoxInterfaceMixin
from napalm_proxmox.lldp_mixin import ProxmoxLLDPMixin
from napalm_proxmox.routing_mixin import ProxmoxRoutingMixin
@@ -84,6 +75,7 @@ _JsonDict = dict[str, Any]
class ProxmoxDriver(
ProxmoxBase,
ProxmoxInterfaceMixin,
ProxmoxSDNMixin,
ProxmoxLLDPMixin,
@@ -96,6 +88,7 @@ class ProxmoxDriver(
ProxmoxVMMigrateMixin,
ProxmoxRoutingMixin,
ProxmoxSystemMixin,
ProxmoxHostMixin,
KernelFactsMixin,
ServicePackagesMixin,
ListeningSocketsMixin,
@@ -143,31 +136,9 @@ class ProxmoxDriver(
timeout: int = 60,
optional_args: _JsonDict | None = None,
) -> None:
self.hostname = hostname
self.username = username
self.password = password
self.timeout = timeout
self.optional_args: _JsonDict = optional_args or {}
self._port: int = self.optional_args.get("port", 8006)
self._verify_ssl: bool = self.optional_args.get(
"verify_ssl",
self.optional_args.get("ssl_verify", self.optional_args.get("verify", True)),
)
self._realm: str = self.optional_args.get("realm", "pam")
self._token_name: str | None = self.optional_args.get("token_name")
self._token_value: str | None = self.optional_args.get("token_value")
super().__init__(hostname, username, password, timeout, optional_args)
self._node: str | None = self.optional_args.get("node")
self._ssh_username: str | None = self.optional_args.get("ssh_username")
self._ssh_password: str | None = self.optional_args.get("ssh_password")
self._ssh_key: str | None = self.optional_args.get("ssh_private_key_str")
self._api: ProxmoxAPI | None = None
self._node_name: str = ""
# paramiko.SSHClient; paramiko is imported lazily (see _node_ssh).
self._ssh_client: Any = None
# Candidate config (merge/replace)
self._candidate_config: str = ""
self._running_config: str = ""
@@ -176,51 +147,8 @@ class ProxmoxDriver(
# Connection management
# ------------------------------------------------------------------ #
def open(self) -> None:
"""Open the connection to the Proxmox API."""
try:
# Use the HTTPS backend for proper REST API support.
# The openssh backend tunnels all kwargs through to
# openssh_wrapper.CommandBaseSession, which does not
# accept password/verify_ssl/token params.
# Proxmox authenticates against "<user>@<realm>" and rejects a bare
# username outright. A caller who typed a realm keeps it; one who
# did not gets self._realm, which is what the documented `realm`
# optional_arg is for -- it was read in __init__ and then never
# used, so the option had no effect and a bare username failed.
user = self.username or ""
if user and "@" not in user:
user = f"{user}@{self._realm}"
kwargs: _JsonDict = {
"host": self.hostname,
"user": user,
"password": self.password,
"port": self._port,
"verify_ssl": self._verify_ssl,
"backend": "https",
}
if self._token_name and self._token_value:
kwargs.pop("password", None)
kwargs["token_value"] = self._token_value
# token_name may be in "<user>!<tokenid>" format (e.g.
# "root@pam!netork"). proxmoxer expects them split:
# user="root@pam", token_name="netork"
user_part, _, token_id = self._token_name.partition("!")
if token_id:
kwargs["user"] = user_part
kwargs["token_name"] = token_id
else:
kwargs["token_name"] = self._token_name
self._api = ProxmoxAPI(**kwargs)
# Validate connection by fetching node status
self._resolve_node()
except Exception as exc:
raise ConnectionException(f"Cannot connect to {self.hostname}: {exc}") from exc
def _proxmox_api(self, **kwargs: Any) -> Any:
return ProxmoxAPI(**kwargs)
def _resolve_node(self) -> str:
"""Resolve the PVE node this connection talks to.
@@ -260,190 +188,10 @@ class ProxmoxDriver(
f"set the 'node' driver argument"
)
def close(self) -> None:
"""Close the connection."""
self._api = None
if self._ssh_client:
try:
self._ssh_client.close()
except Exception:
logger.debug("%s: closing the SSH session: ignored", self.hostname, exc_info=True)
self._ssh_client = None
def is_alive(self) -> _JsonDict:
"""Return connection liveness.
Probes ``GET /version``, the cheapest endpoint that proves the session
still authenticates. It used to call ``_resolve_node()``, which returns
early without touching the API whenever a node was configured via
optional_args — so a dead connection reported itself alive.
"""
alive = False
if self._api:
try:
self._api.version.get()
alive = True
except Exception:
logger.debug("%s: liveness probe failed", self.hostname, exc_info=True)
return {"is_alive": alive}
# ------------------------------------------------------------------ #
# Internal API helpers
# Node info helpers (PVE only)
# ------------------------------------------------------------------ #
def _node_api(self):
"""Return the API resource for the current node."""
return self._api.nodes(self._node_name)
def _get_node_network(self) -> list[_JsonDict]:
"""Return the node's network interface list from the Proxmox API."""
try:
return self._node_api().network.get() or []
except Exception as exc:
logger.debug("Failed to fetch node network: %s", exc)
return []
def _exec_ssh_command(self, command: str) -> str:
"""Run *command* in the node's shell over SSH and return what it printed.
Stdout only: stderr is not output (a command that wants it says
``2>&1``). The exit status is not checked; a caller that needs it asks
for it (``_rc``). The node has no API endpoint that runs a shell command
(``/nodes/{node}/execute`` runs a batch of API calls), so this is the
one way, the same session ``_run_node_command`` uses (see
NAPALM/napalm-device-types#35 for the shared transport).
:raises ConnectionException: when the command could not run: no SSH
credential, a refused login or host key (``SshHostKeyMismatch``),
an unreachable node, a timeout.
"""
try:
_, stdout, stderr = self._node_ssh().exec_command(command, timeout=self.timeout)
out = stdout.read().decode(errors="replace")
err = stderr.read().decode(errors="replace").strip()
except ConnectionException:
raise
except _ssh_errors() as exc:
raise ConnectionException(
f"Cannot run a command on {self.hostname} over SSH: {exc}"
) from exc
if err:
logger.debug("%s: stderr of a node command: %s", self.hostname, err[-300:])
return out.strip()
def run_command(
self,
command: str,
*,
privileged: bool = False,
timeout: float = 60,
stdin: bytes | None = None,
) -> CommandResult:
"""Run *command* on an exec channel of the node's SSH session (``CommandChannelMixin``).
stdout, stderr and the exit status come back apart; *stdin*, if given, is
written and then closed. *privileged* needs nothing more: the node is
logged in as root, as for every other command this driver runs there.
:raises ConnectionException: when the command could not run, or did not
finish within *timeout* seconds.
"""
try:
transport = self._node_ssh().get_transport()
return run_on_transport(transport, command, stdin=stdin, timeout=timeout)
except ConnectionException:
raise
except _ssh_errors() as exc:
raise ConnectionException(
f"Cannot run a command on {self.hostname} over SSH: {exc}"
) from exc
def _node_ssh(self) -> Any: # paramiko.SSHClient; paramiko is imported lazily
"""The SSH session to the node on port 22, opened once.
Its host key is the one the caller expects for port 22, or on first use
whichever the node presents; either way it is reported (netork#1167).
:raises ConnectionException: without a password or a usable key.
"""
import paramiko
if self._ssh_client is None:
# A key is tried first; paramiko falls back to the password. Only
# these: not the keys in the process's own ~/.ssh, nor an agent.
connect_kwargs: _JsonDict = {
"username": self._ssh_username or self.username,
"timeout": self.timeout,
"look_for_keys": False,
"allow_agent": False,
}
password = self._ssh_password or self.password
if password:
connect_kwargs["password"] = password
pkey = _load_private_key(self._ssh_key) if self._ssh_key else None
if pkey is not None:
connect_kwargs["pkey"] = pkey
if not password and pkey is None:
# An API-token-only node: nothing to log in with, so no connection.
raise ConnectionException(
f"no SSH credential for {self.hostname}: set ssh_password or "
"ssh_private_key_str"
)
client = paramiko.SSHClient()
try:
HostKeyCheck(self.optional_args, self.hostname, 22).connect(
client, **connect_kwargs
)
except Exception:
client.close()
raise
self._ssh_client = client
return self._ssh_client
# ------------------------------------------------------------------ #
# Node info helpers
# ------------------------------------------------------------------ #
def _get_version_info(self) -> _JsonDict:
"""Return Proxmox VE version info from the API."""
try:
return self._api.version.get() or {}
except Exception as exc:
logger.debug("Failed to fetch version info: %s", exc)
return {}
def _get_node_status(self) -> _JsonDict:
"""Return the node's status from the Proxmox API."""
try:
return self._node_api().status.get() or {}
except Exception as exc:
logger.debug("Failed to fetch node status: %s", exc)
return {}
def _get_node_subscription(self) -> _JsonDict:
"""Return subscription status for this node."""
try:
return self._node_api().subscription.get() or {}
except Exception as exc:
logger.debug("Failed to fetch node subscription: %s", exc)
return {}
def _get_node_dns(self) -> _JsonDict:
"""Return DNS configuration for this node."""
try:
return self._node_api().dns.get() or {}
except Exception as exc:
logger.debug("Failed to fetch node DNS: %s", exc)
return {}
def _get_node_time(self) -> _JsonDict:
"""Return time configuration for this node."""
try:
return self._node_api().time.get() or {}
except Exception as exc:
logger.debug("Failed to fetch node time: %s", exc)
return {}
def _get_node_ntp(self) -> _JsonDict:
"""Return NTP configuration for this node."""
try:
@@ -478,46 +226,7 @@ class ProxmoxDriver(
release = version.get("release", "")
os_version = f"Proxmox VE {pve_version}" if pve_version else f"Proxmox VE {release}"
# Physical hardware info from Linux DMI sysfs.
# Read each field separately to avoid shell quoting issues with printf.
# Field priority for model:
# product_name — human-readable name on most vendors (e.g. "NUC6CAYH",
# "ThinkCentre M910x")
# product_version — sometimes the marketing name on Lenovo; on Intel NUC
# it is the board part number (less useful as model name)
# We prefer product_name; fall back to product_version only when
# product_name looks like a raw type code (all uppercase + digits, no spaces).
vendor = ""
model = ""
serial = ""
try:
dmi_cmd = (
"v=$(cat /sys/class/dmi/id/sys_vendor 2>/dev/null); "
"n=$(cat /sys/class/dmi/id/product_name 2>/dev/null); "
"r=$(cat /sys/class/dmi/id/product_version 2>/dev/null); "
"s=$(cat /sys/class/dmi/id/product_serial 2>/dev/null); "
'printf \'%s\\n%s\\n%s\\n%s\\n\' "$v" "$n" "$r" "$s"'
)
lines = self._exec_ssh_command(dmi_cmd).splitlines()
if len(lines) >= 4:
vendor = lines[0].strip()
product_name = lines[1].strip()
product_version = lines[2].strip()
serial = lines[3].strip()
_bad = {"none", "n/a", "not specified", "to be filled by o.e.m."}
pv_usable = (
product_version
and product_version.lower() not in _bad
and product_version != product_name
# Only prefer product_version when it contains a space —
# that indicates a human-readable marketing name like
# "ThinkCentre M910x" rather than a part code like "J26843-409".
and " " in product_version
)
model = product_version if pv_usable else product_name
except Exception as exc:
logger.debug("Failed to read DMI info via SSH: %s", exc)
vendor, model, serial = self._hardware()
# Currently-booted kernel release (uname -r) — distinct from any newer
# kernel that is merely installed and pending a reboot.
@@ -538,34 +247,3 @@ class ProxmoxDriver(
"interface_list": iface_list,
"running_kernel": running_kernel,
}
# ------------------------------------------------------------------ #
# CLI passthrough
# ------------------------------------------------------------------ #
def cli(self, commands: list[str], encoding: str = "text") -> dict[str, str]:
"""Execute a list of shell commands and return their outputs."""
return {cmd: self._exec_ssh_command(cmd) for cmd in commands}
def _load_private_key(text: str) -> Any: # paramiko.PKey; paramiko is imported lazily
"""The SSH private key in *text*, whatever its type, or None when it cannot be read."""
import io
import paramiko
for key_class in (paramiko.Ed25519Key, paramiko.ECDSAKey, paramiko.RSAKey):
try:
return key_class.from_private_key(io.StringIO(text))
except (paramiko.SSHException, ValueError):
continue
logger.warning("The SSH private key could not be read; logging in with the password")
return None
def _ssh_errors() -> tuple[type[Exception], ...]:
"""What a command that could not run raises: the socket's errors, timeouts
included, and paramiko's (a refused login, a closed session)."""
import paramiko
return (OSError, paramiko.SSHException)
+308
View File
@@ -0,0 +1,308 @@
"""What a Proxmox VE node and a Proxmox Backup Server share as Debian hosts.
The transports for napalm-device-types' host readers (kernel facts, service
packages, system identity, listening sockets, systemd services, host status,
apt's proxy), packages and APT updates, the subscription, root file writes,
and restarting or shutting the host down. All of it is the same on both
products: the same ``/nodes/{node}/apt``, ``/subscription`` and ``/status``
endpoints, and root's shell over the driver's own SSH session.
"""
from __future__ import annotations
import base64
import ipaddress
import logging
import re
import shlex
from typing import TYPE_CHECKING, Any, cast
from napalm_device_types import APT_UPGRADABLE_COMMAND, parse_apt_upgradable
from napalm_device_types.models import (
FixStepOutcomeDict,
LicenseDict,
LicenseStatus,
PackageDict,
)
logger = logging.getLogger(__name__)
_JsonDict = dict[str, Any]
#: The subscription API's status (lower-cased) as a napalm-device-types
#: ``LicenseStatus``; "new" and "suspended" have none of their own.
_SUBSCRIPTION_STATUS: dict[str, LicenseStatus] = {
"notfound": "none",
"active": "active",
"invalid": "invalid",
"expired": "expired",
}
_PACKAGE_NAME = re.compile(r"^[a-zA-Z0-9][a-zA-Z0-9_.+:~-]*$")
class ProxmoxHostMixin:
"""The Debian host under a Proxmox product."""
#: ``source`` of every package ``get_packages`` reports.
PACKAGE_SOURCE = "pve"
#: ``name`` of the subscription ``get_licenses`` reports.
SUBSCRIPTION_LICENSE = "pve-subscription"
_node_name: str
if TYPE_CHECKING: # pragma: no cover - the driver's (napalm_proxmox._base)
def _exec_ssh_command(self, command: str) -> str: ...
def _node_api(self) -> Any: ...
def _get_node_subscription(self) -> _JsonDict: ...
def _run_kernel_facts_command(self, command: str) -> str:
"""The transport for ``KernelFactsMixin.get_kernel_facts``: the exec path."""
return self._exec_ssh_command(command)
def _run_service_packages_command(self, command: str) -> str:
"""The transport for ``ServicePackagesMixin.get_service_packages``: the exec path."""
return str(self._exec_ssh_command(command))
def _run_system_identity_command(self, command: str, *, privileged: bool) -> str:
"""The transport for ``SystemIdentityMixin.get_system_identity``: the exec path,
which runs as root either way."""
return str(self._exec_ssh_command(command))
def _run_listening_sockets_command(self, command: str, *, privileged: bool) -> str:
"""The transport for ``ListeningSocketsMixin.get_listening_sockets``:
the exec path, which runs as root either way."""
return str(self._exec_ssh_command(command))
def get_packages(self) -> list[PackageDict]:
"""Return installed Debian packages with available-update info.
Installed list comes from ``dpkg-query`` via SSH (the Proxmox API
``/apt/installed`` endpoint is not implemented on PVE 8.x).
Available updates come from the Proxmox API ``/apt/update``.
"""
upgradable = self._upgradable_versions()
# Installed packages via SSH dpkg-query
raw = self._exec_ssh_command(
"dpkg-query -W -f="
"'${Package}\\t${Version}\\t${db:Status-Status}\\t${Installed-Size}"
"\\t${source:Package}\\t${source:Version}\\n'"
" 2>/dev/null"
)
result = []
for line in raw.splitlines():
pkg = parse_dpkg_line(line, self.PACKAGE_SOURCE)
if pkg is not None:
pkg["upgrade_version"] = upgradable.get(pkg["name"], "")
result.append(pkg)
# PackageDict has no source_package, source_version and upgrade_version
# yet (NAPALM/napalm-device-types#85); every reader of these
# packages relies on them, so they stay.
return cast(list[PackageDict], result)
def _upgradable_versions(self) -> dict[str, str]:
"""The candidate version of every upgradable package, from ``/apt/update``."""
upgradable: dict[str, str] = {}
try:
for upd in self._node_api().apt.update.get():
pkg = upd.get("Package", "")
if pkg:
upgradable[pkg] = upd.get("Version", "")
except Exception as exc:
logger.debug("Failed to fetch available APT updates: %s", exc)
return upgradable
def get_licenses(self) -> list[LicenseDict]:
"""The node's subscription, as the API reports it.
A fact for the caller: whether running without one is a problem is not
the driver's to say. An unreadable subscription reports nothing.
"""
sub = self._get_node_subscription()
raw = str(sub.get("status", ""))
if not raw:
return []
lic: LicenseDict = {
"name": self.SUBSCRIPTION_LICENSE,
"status": _SUBSCRIPTION_STATUS.get(raw.lower(), "unknown"),
"detail": str(sub.get("message") or raw),
}
if sub.get("productname"):
lic["edition"] = sub["productname"]
if sub.get("nextduedate"):
lic["expires"] = sub["nextduedate"]
return [lic]
def _run_service_command(self, command: str, *, privileged: bool, timeout: int) -> str:
"""The transport for ``SystemdServicesMixin``: the exec path, as root.
*privileged* needs nothing more on a node the driver reaches as root, and
the exec path keeps its own timeout.
"""
return str(self._exec_ssh_command(command))
def get_available_updates(self) -> list[_JsonDict]:
"""Return the node's upgradable packages, with origin and security status.
``apt list --upgradable`` over the exec path names each candidate's suite
(``trixie-security``); the APT API names only an Origin ("Debian",
"Proxmox") and is the fallback, with the security status unknown.
:raises Exception: when neither answers -- never an empty list for
"could not read".
"""
updates: list[_JsonDict]
try:
raw = self._exec_ssh_command(APT_UPGRADABLE_COMMAND) or ""
updates = [dict(u) for u in parse_apt_upgradable(raw)]
except ValueError as exc:
logger.debug("apt list over the exec path failed, using the API: %s", exc)
updates = self._updates_from_api()
return sorted(updates, key=lambda u: u["name"])
def _updates_from_api(self) -> list[_JsonDict]:
return [
{
"name": upd["Package"],
"current_version": upd.get("OldVersion", ""),
"new_version": upd.get("Version", ""),
"origin": upd.get("Origin"),
"security": None,
}
for upd in self._node_api().apt.update.get()
if upd.get("Package")
]
def refresh_available_updates(self) -> _JsonDict:
"""Resynchronise the node's package index (``POST nodes/{n}/apt/update``)."""
try:
task = self._node_api().apt.update.post()
except Exception as exc:
return {"success": False, "output": str(exc)}
return {"success": True, "output": f"Package index refresh started ({task})"}
def _run_host_status_command(self, command: str) -> str:
"""The transport for ``HostStatusMixin.get_host_status``: the exec path."""
return str(self._exec_ssh_command(command))
def apply_updates(self, packages: list[str]) -> _JsonDict:
"""Upgrade the given packages via ``apt-get install`` over SSH."""
for pkg in packages:
if not re.match(r"^[a-zA-Z0-9_\-\+\.]+$", pkg):
raise ValueError(f"Invalid package name: {pkg!r}")
pkg_args = " ".join(packages)
output, status = self._rc(
f"DEBIAN_FRONTEND=noninteractive apt-get install --only-upgrade -y {pkg_args}"
)
if status == 0:
return {"success": True, "output": output}
error = (
f"apt-get exited with status {status}"
if status >= 0
else "apt-get's exit status could not be read"
)
return {"success": False, "output": output, "error": error}
def _rc(self, command: str) -> tuple[str, int]:
"""Run *command* and return its output and exit status (-1 when unknown)."""
out = self._exec_ssh_command(f"{{ {command}; }} 2>&1; echo __PVE_RC=$?")
text, _, tail = out.rpartition("__PVE_RC=")
try:
return text.strip(), int(tail.strip())
except ValueError:
return out.strip(), -1
def _step(self, command: str, done: str) -> FixStepOutcomeDict:
output, status = self._rc(command)
return {"ok": status == 0, "output": done if status == 0 else output[-300:]}
def _run_apt_command(self, command: str) -> str:
return str(self._exec_ssh_command(command))
def _run_apt_root_command(self, command: str, done: str) -> FixStepOutcomeDict:
return self._step(command, done)
def _write_apt_file(self, path: str, content: str) -> FixStepOutcomeDict:
return self._write_root_file(path, content)
def _write_root_file(self, path: str, content: str) -> FixStepOutcomeDict:
"""Write *content* to *path* as root, mode 644, through a temporary file."""
encoded = base64.b64encode(content.encode()).decode()
# No ``exit`` here: it would end the shell before ``_rc`` reads the status.
return self._step(
f'umask 077 && t=$(mktemp) && echo {encoded} | base64 -d > "$t" && '
f'install -m 644 -o root -g root "$t" {shlex.quote(path)}; '
's=$?; rm -f "$t"; [ "$s" -eq 0 ]',
f"wrote {path}",
)
def install_package(self, name: str, version: str = "") -> None:
"""apt-get install *name* (``PackageManagementMixin``); raises when apt fails."""
if not _PACKAGE_NAME.match(name) or (version and not _PACKAGE_NAME.match(version)):
raise ValueError(f"not a package name: {name!r}")
target = f"{name}={version}" if version else name
output, status = self._rc(f"DEBIAN_FRONTEND=noninteractive apt-get install -y {target}")
if status != 0:
raise RuntimeError(f"apt-get install {target} failed: {output[-300:]}")
def get_management_peer(self) -> str | None:
"""The address the node sees the SSH connection come from (``SSH_CONNECTION``)."""
out = self._exec_ssh_command('printf "%s\\n" "$SSH_CONNECTION"').split()
try:
return str(ipaddress.ip_address(out[0])) if out else None
except ValueError:
return None
def reboot_host(self) -> None:
"""Restart the node through the API (no SSH involved)."""
try:
self._node_api().status.post(command="reboot")
except Exception as exc:
raise RuntimeError(f"Reboot of node {self._node_name!r} refused: {exc}") from exc
def shutdown_host(self) -> None:
"""Shut the node down through the API; on PVE its guests stop as on any shutdown."""
try:
self._node_api().status.post(command="shutdown")
except Exception as exc:
raise RuntimeError(f"Shutdown of node {self._node_name!r} refused: {exc}") from exc
def parse_dpkg_line(line: str, source: str) -> _JsonDict | None:
"""One installed package from a line of the ``dpkg-query`` format above, or
None for a line that is not an installed package."""
parts = line.strip().split("\t")
if len(parts) < 2 or not parts[0]:
return None
name, version = parts[0], parts[1]
status = parts[2] if len(parts) > 2 else "installed"
if status != "installed":
return None
size_kb, source_package_field, source_version_field = (parts[3:] + ["", "", ""])[:3]
# Debian source package (differs from the binary for split packages,
# e.g. openssh-server → openssh). Needed for accurate OSV matching.
source_package = source_package_field or name
# And its version, which is a different number from this package's.
#
# OSV states Debian ranges in *source* versions, so a consumer given
# the source package and only the binary version compares two
# unrelated numbers: libldb2 is 2:2.11.0+samba4.22.11+dfsg-… while
# its source, samba, is 2:4.22.11+dfsg-…. Reporting the source
# without its version is worse than reporting neither, because it
# looks usable. Every package on all three Proxmox nodes was in that
# state — the format string never asked for the field.
#
# dpkg leaves it empty when it equals `Version`, and so does an
# older dpkg that does not know the field at all.
source_version = source_version_field or version
return {
"name": name,
"version": version,
"installed": True,
"description": "",
"size": int(size_kb) * 1024 if size_kb.isdigit() else 0,
"source": source,
"source_package": source_package,
"source_version": source_version,
}
+314
View File
@@ -0,0 +1,314 @@
"""The backup-server role of the Proxmox Backup Server driver.
Datastores, backup groups with their snapshots, the jobs that keep them
(garbage collection, verify, prune, sync, tape) and the remotes, read through
the PBS REST API, plus the actions on them.
A privilege-separated API token without an ACL entry does not get a 403 for a
listing: PBS filters the list by privilege and answers ``[]``. An empty list
therefore says nothing until ``/access/permissions`` shows the privilege, so
every read checks it first.
"""
from __future__ import annotations
from typing import Any
from napalm_device_types import (
BackupDatastoreDict,
BackupGroupDict,
BackupJobKind,
BackupRemoteDict,
BackupRunState,
BackupServerJobDict,
BackupSnapshotDict,
BackupSnapshotRefDict,
BackupTaskDict,
BackupTaskStatusDict,
)
from proxmoxer.core import ResourceException
_JsonDict = dict[str, Any]
#: Where "run this job now" is, per kind; ``{id}`` is the job, for GC the datastore.
_RUN_PATHS: dict[str, str] = {
"gc": "admin/datastore/{id}/gc",
"verify": "admin/verify/{id}/run",
"prune": "admin/prune/{id}/run",
"sync": "admin/sync/{id}/run",
"tape": "tape/backup/{id}",
}
#: Where each kind's jobs are listed with the state of their last run.
_JOB_LISTS: dict[BackupJobKind, str] = {
"gc": "admin/gc",
"verify": "admin/verify",
"prune": "admin/prune",
"sync": "admin/sync",
"tape": "tape/backup",
}
def run_state(text: str | None) -> tuple[BackupRunState | None, str | None]:
"""A job's ``last-run-state`` or a task's ``exitstatus``: the state and the error."""
if not text:
return None, None
if text == "OK":
return "ok", None
if text.startswith("WARNINGS"):
return "warning", text
return "error", text
def _granted(rights: dict[str, dict[str, Any]], privilege: str, path: str) -> bool:
"""Whether *privilege* holds on *path*, given on the path itself or propagated
from a parent (``/access/permissions``: privilege -> propagate flag)."""
for granted_on, privileges in rights.items():
if privilege not in privileges:
continue
if granted_on == path:
return True
parent = granted_on.rstrip("/") + "/"
if path.startswith(parent) and privileges[privilege]:
return True
return False
def _maintenance(config: _JsonDict, mount_status: str | None) -> str | None:
for part in str(config.get("maintenance-mode") or "").split(","):
key, _, value = part.partition("=")
if key == "type" and value:
return value
return "unmounted" if mount_status == "notmounted" else None
def _datastore(usage: _JsonDict, config: _JsonDict) -> BackupDatastoreDict:
"""One datastore from its ``status/datastore-usage`` row and its configuration."""
if usage.get("backend-type") == "s3":
backend = "s3"
elif config.get("backing-device"):
backend = "removable"
else:
backend = "local"
# A removable datastore's path is relative to wherever it is mounted.
path = config.get("path")
full = usage.get("estimated-full-date")
return {
"name": usage["store"],
"path": path if isinstance(path, str) and path.startswith("/") else None,
"backend": backend,
"total": usage.get("total"),
"used": usage.get("used"),
"available": usage.get("avail"),
"estimated_full": full if isinstance(full, int) and full > 0 else None,
"maintenance": _maintenance(config, usage.get("mount-status")),
"comment": config.get("comment"),
}
def _snapshot(raw: _JsonDict) -> BackupSnapshotDict:
state = (raw.get("verification") or {}).get("state")
return {
"time": int(raw["backup-time"]),
"size": raw.get("size"),
"verify": state if state in ("ok", "failed") else None,
"protected": bool(raw.get("protected")),
"comment": raw.get("comment"),
}
def _job(kind: BackupJobKind, raw: _JsonDict) -> BackupServerJobDict:
state, error = run_state(raw.get("last-run-state"))
if state is None and raw.get("last-run-upid") and not raw.get("last-run-endtime"):
state = "running"
sync = kind == "sync"
enabled = not raw.get("disable") if kind == "prune" else bool(raw.get("schedule"))
return {
"id": raw["store"] if kind == "gc" else raw["id"],
"kind": kind,
"datastore": raw.get("store"),
"namespace": raw.get("ns"),
"schedule": raw.get("schedule"),
"enabled": enabled,
"last_run": raw.get("last-run-endtime"),
"last_state": state,
"last_error": error,
"next_run": raw.get("next-run"),
"remote": raw.get("remote") if sync else None,
"direction": (raw.get("sync-direction") or "pull") if sync else None,
"comment": raw.get("comment"),
}
def _snapshot_params(snapshot: BackupSnapshotRefDict) -> _JsonDict:
params: _JsonDict = {
"backup-type": snapshot["group_type"],
"backup-id": snapshot["group_id"],
"backup-time": snapshot["time"],
}
if snapshot["namespace"]:
params["ns"] = snapshot["namespace"]
return params
class ProxmoxBackupServerMixin:
"""``BackupServerDriver`` for Proxmox Backup Server."""
_api: Any
_node_name: str
# ------------------------------------------------------------------ #
# Plumbing
# ------------------------------------------------------------------ #
def _pbs(self, method: str, path: str, **params: Any) -> Any:
"""One API call. A 400 or 404 is the caller's mistake (no such datastore,
job, snapshot or task): ValueError. Everything else propagates."""
try:
return getattr(self._api, method)(path, **params)
except ResourceException as exc:
if exc.status_code in (400, 404):
raise ValueError(f"{method.upper()} {path}: {exc}") from exc
raise
def _require(self, privilege: str, path: str) -> None:
rights = self._pbs("get", "access/permissions") or {}
if not _granted(rights, privilege, path):
raise PermissionError(
f"the API login lacks {privilege} on {path}; the server would answer "
"with an empty list rather than refuse"
)
@staticmethod
def _task(upid: Any, what: str) -> BackupTaskDict:
if not isinstance(upid, str) or not upid:
raise RuntimeError(f"{what}: the server returned no task id")
return {"task": upid}
# ------------------------------------------------------------------ #
# Read
# ------------------------------------------------------------------ #
def get_backup_datastores(self) -> list[BackupDatastoreDict]:
self._require("Datastore.Audit", "/datastore")
configs = {c["name"]: c for c in self._pbs("get", "config/datastore") or []}
return [
_datastore(usage, configs.get(usage["store"], {}))
for usage in self._pbs("get", "status/datastore-usage") or []
]
def get_backup_groups(self, datastore: str = "") -> list[BackupGroupDict]:
"""The groups of *datastore*, or of every datastore whose content can be
read: a removable one that is not mounted is left out of the listing
(``get_backup_datastores`` reports it as ``"unmounted"``), and raises
ValueError when asked for by name."""
self._require("Datastore.Audit", "/datastore")
stores = (
[datastore]
if datastore
else [
d["store"]
for d in self._pbs("get", "admin/datastore") or []
if d.get("mount-status") != "notmounted"
]
)
groups: list[BackupGroupDict] = []
for store in stores:
for ns in self._pbs("get", f"admin/datastore/{store}/namespace") or []:
groups.extend(self._groups_in(store, ns.get("ns") or ""))
return groups
def _groups_in(self, store: str, namespace: str) -> list[BackupGroupDict]:
params = {"ns": namespace} if namespace else {}
snapshots: dict[tuple[str, str], list[BackupSnapshotDict]] = {}
for raw in self._pbs("get", f"admin/datastore/{store}/snapshots", **params) or []:
key = (raw["backup-type"], raw["backup-id"])
snapshots.setdefault(key, []).append(_snapshot(raw))
groups: list[BackupGroupDict] = []
for raw in self._pbs("get", f"admin/datastore/{store}/groups", **params) or []:
own = snapshots.get((raw["backup-type"], raw["backup-id"]), [])
groups.append(
{
"datastore": store,
"namespace": namespace,
"type": raw["backup-type"],
"id": raw["backup-id"],
"owner": raw.get("owner"),
"count": int(raw.get("backup-count") or 0),
"newest": max(own, key=lambda s: s["time"]) if own else None,
"oldest_time": min(s["time"] for s in own) if own else None,
"comment": raw.get("comment"),
}
)
return groups
def get_backup_snapshots(
self, datastore: str, namespace: str, group_type: str, group_id: str
) -> list[BackupSnapshotDict]:
self._require("Datastore.Audit", "/datastore")
params: _JsonDict = {"backup-type": group_type, "backup-id": group_id}
if namespace:
params["ns"] = namespace
raw = self._pbs("get", f"admin/datastore/{datastore}/snapshots", **params) or []
return sorted((_snapshot(s) for s in raw), key=lambda s: s["time"], reverse=True)
def get_backup_jobs(self) -> list[BackupServerJobDict]:
self._require("Datastore.Audit", "/datastore")
jobs: list[BackupServerJobDict] = []
for kind, path in _JOB_LISTS.items():
jobs.extend(_job(kind, raw) for raw in self._pbs("get", path) or [])
return jobs
def get_backup_remotes(self) -> list[BackupRemoteDict]:
self._require("Remote.Audit", "/remote")
return [
{
"name": raw["name"],
"host": raw["host"],
"port": raw.get("port"),
"fingerprint": raw.get("fingerprint"),
"auth_id": raw.get("auth-id"),
"comment": raw.get("comment"),
}
for raw in self._pbs("get", "config/remote") or []
]
# ------------------------------------------------------------------ #
# Actions
# ------------------------------------------------------------------ #
def start_garbage_collection(self, datastore: str) -> BackupTaskDict:
return self.run_backup_job("gc", datastore)
def run_backup_job(self, kind: BackupJobKind, job_id: str) -> BackupTaskDict:
if kind not in _RUN_PATHS:
raise ValueError(f"unknown job kind {kind!r}; one of {', '.join(_RUN_PATHS)}")
path = _RUN_PATHS[kind].format(id=job_id)
return self._task(self._pbs("post", path), f"{kind} job {job_id}")
def verify_backup_snapshot(self, snapshot: BackupSnapshotRefDict) -> BackupTaskDict:
path = f"admin/datastore/{snapshot['datastore']}/verify"
return self._task(self._pbs("post", path, **_snapshot_params(snapshot)), "verify")
def set_backup_snapshot_protected(
self, snapshot: BackupSnapshotRefDict, protected: bool
) -> None:
path = f"admin/datastore/{snapshot['datastore']}/protected"
self._pbs("put", path, **_snapshot_params(snapshot), protected=int(protected))
def delete_backup_snapshot(self, snapshot: BackupSnapshotRefDict) -> None:
path = f"admin/datastore/{snapshot['datastore']}/snapshots"
self._pbs("delete", path, **_snapshot_params(snapshot))
def get_backup_task(self, task: str) -> BackupTaskStatusDict:
base = f"nodes/{self._node_name}/tasks/{task}"
status = self._pbs("get", f"{base}/status") or {}
log = self._pbs("get", f"{base}/log", limit=0) or []
running = status.get("status") == "running"
state, error = ("running", None) if running else run_state(status.get("exitstatus"))
return {
"task": task,
"running": running,
"state": state,
"error": error,
"log": [str(line.get("t", "")) for line in log],
}
+168
View File
@@ -0,0 +1,168 @@
"""The Proxmox Backup Server as a host: facts, interfaces, environment, and the
disks and ZFS pools it manages (``StorageDriver``, its second role).
The API answers most of it; the shell over SSH adds what the API does not
carry (the host name, DMI, the interfaces' MAC addresses). Without an SSH login
those stay at what the API alone can say.
"""
from __future__ import annotations
import logging
from typing import TYPE_CHECKING, Any
from napalm.base.exceptions import ConnectionException
from napalm_device_types.models import DiskPoolDict, PhysicalDiskDict
from napalm_proxmox import utils
logger = logging.getLogger(__name__)
_JsonDict = dict[str, Any]
#: Each interface's name and MAC address, one per line.
LINK_ADDRESSES_COMMAND = (
'for i in /sys/class/net/*; do echo "${i##*/} $(cat "$i/address" 2>/dev/null)"; done'
)
_HEALTH = {"passed": "healthy", "failed": "failed"}
_POOL_STATUS = {"online": "online", "degraded": "degraded", "faulted": "faulted"}
class ProxmoxPbsHostMixin:
"""Host getters of the PBS driver."""
hostname: str
if TYPE_CHECKING: # pragma: no cover - the driver's (napalm_proxmox._base)
def _exec_ssh_command(self, command: str) -> str: ...
def _node_api(self) -> Any: ...
def _hardware(self) -> tuple[str, str, str]: ...
def _get_node_status(self) -> _JsonDict: ...
def _get_node_network(self) -> list[_JsonDict]: ...
def _get_version_info(self) -> _JsonDict: ...
def _shell(self, command: str) -> str | None:
"""*command*'s output, or None where the server has no SSH login for us."""
try:
return self._exec_ssh_command(command)
except ConnectionException as exc:
logger.debug("%s: no shell for %r: %s", self.hostname, command, exc)
return None
def get_facts(self) -> _JsonDict:
"""Facts from the API, with the host name and DMI over SSH where it is open."""
status = self._get_node_status()
version = self._get_version_info().get("version", "")
fqdn = (self._shell("hostname -f") or "").strip() or self.hostname
vendor, model, serial = self._hardware()
return {
"uptime": float(status.get("uptime", 0)),
"vendor": vendor or "Proxmox Server Solutions GmbH",
"model": model or "Proxmox Backup Server",
"hostname": fqdn.split(".")[0] if fqdn != self.hostname else fqdn,
"fqdn": fqdn,
"os_version": f"Proxmox Backup Server {version}".strip(),
"serial_number": serial,
"interface_list": sorted(
i["iface"] for i in self._get_node_network() if i.get("iface")
),
"running_kernel": (status.get("current-kernel") or {}).get("release", ""),
}
def _link_macs(self) -> dict[str, str]:
out = self._shell(LINK_ADDRESSES_COMMAND) or ""
macs: dict[str, str] = {}
for line in out.splitlines():
name, _, mac = line.strip().partition(" ")
if name and mac:
macs[name] = utils.normalize_mac(mac.strip())
return macs
def get_interfaces(self) -> dict[str, _JsonDict]:
"""The interfaces the server's network configuration lists. The API carries
no MAC address; it is read over SSH, and empty without it."""
macs = self._link_macs()
result: dict[str, _JsonDict] = {}
for iface in self._get_node_network():
name = iface.get("iface", "")
if not name:
continue
result[name] = {
"is_up": bool(iface.get("active")),
"is_enabled": bool(iface.get("autostart")) or bool(iface.get("active")),
"description": (iface.get("comments") or "").strip(),
"last_flapped": -1.0,
"speed": utils.speed_mbps(iface),
"mtu": int(iface.get("mtu") or 1500),
"mac_address": macs.get(name, utils.normalize_mac(iface.get("hwaddr", ""))),
}
return result
def get_interfaces_ip(self) -> dict[str, _JsonDict]:
result: dict[str, _JsonDict] = {}
for iface in self._get_node_network():
addrs = utils.addresses_from_node_network(iface)
if iface.get("iface") and addrs:
result[iface["iface"]] = addrs
return result
def get_environment(self) -> _JsonDict:
"""CPU and memory from the node status; a PBS reports no sensors."""
status = self._get_node_status()
mem = status.get("memory") or {}
return {
"fans": {},
"temperature": {},
"power": {},
"cpu": {"0": {"%usage": round(float(status.get("cpu") or 0.0) * 100, 2)}},
"memory": {
"available_ram": int(mem.get("total") or 0),
"used_ram": int(mem.get("used") or 0),
},
}
# ------------------------------------------------------------------ #
# Storage role
# ------------------------------------------------------------------ #
def get_disks(self) -> list[PhysicalDiskDict]:
"""The server's disks (``/nodes/{node}/disks/list``), as it classifies them."""
disks: list[PhysicalDiskDict] = []
for raw in self._node_api().disks.list.get() or []:
kind = raw.get("disk-type")
disks.append(
{
"slot": raw.get("name") or raw.get("devpath", ""),
"model": raw.get("model") or "",
"serial": raw.get("serial") or "",
"vendor": raw.get("vendor") or "",
"type": kind if kind in ("ssd", "nvme") else "hdd",
"size": int(raw.get("size") or 0),
"rpm": int(raw.get("rpm") or 0),
"temperature": -1,
"health": _HEALTH.get(str(raw.get("status", "")).lower(), "unknown"),
"pool": "",
}
)
return disks
def get_disk_pools(self) -> dict[str, DiskPoolDict]:
"""The ZFS pools the server manages (``/nodes/{node}/disks/zfs``)."""
pools: dict[str, DiskPoolDict] = {}
for raw in self._node_api().disks.zfs.get() or []:
name = raw["name"]
pools[name] = {
"name": name,
"type": "zfs",
"level": "",
"status": _POOL_STATUS.get(str(raw.get("health", "")).lower(), "unknown"),
"total": int(raw.get("size") or 0),
"used": int(raw.get("alloc") or 0),
"available": int(raw.get("free") or 0),
"disks": [],
"auto_expand": False,
"dedup": float(raw.get("dedup") or 1.0) > 1.0,
"compression": "",
}
return pools
+2 -239
View File
@@ -14,20 +14,18 @@
"""System-level NAPALM getters for Proxmox VE nodes.
Environment, NTP, SNMP, users, packages, services, updates and disk SMART.
Environment, NTP, SNMP, users, hostname and disk SMART. What a PVE node shares
with a Proxmox Backup Server as a Debian host is in ``host_mixin``.
"""
from __future__ import annotations
import base64
import ipaddress
import logging
import re
import shlex
from typing import Any
from napalm.base.exceptions import CommandErrorException
from napalm_device_types import APT_UPGRADABLE_COMMAND, parse_apt_upgradable
from napalm_proxmox.ntp import NTP_STATS_COMMAND, parse_ntp_stats
@@ -35,17 +33,6 @@ logger = logging.getLogger(__name__)
_JsonDict = dict[str, Any]
#: The subscription API's status (lower-cased) as a napalm-device-types
#: ``LicenseStatus``; "new" and "suspended" have none of their own.
_SUBSCRIPTION_STATUS = {
"notfound": "none",
"active": "active",
"invalid": "invalid",
"expired": "expired",
}
_PACKAGE_NAME = re.compile(r"^[a-zA-Z0-9][a-zA-Z0-9_.+:~-]*$")
#: What may stand in an snmpd.conf value: no whitespace, no quote, no comment.
_SNMP_WORD = re.compile(r'^[^\s"#]+$')
#: The net-snmp agent's configuration file.
@@ -303,200 +290,26 @@ class ProxmoxSystemMixin:
# Kernel facts (KernelFactsMixin supplies get_kernel_facts)
# ------------------------------------------------------------------ #
def _run_kernel_facts_command(self, command: str) -> str:
"""The transport for ``KernelFactsMixin.get_kernel_facts``: the exec path."""
return self._exec_ssh_command(command)
def _run_service_packages_command(self, command: str) -> str:
"""The transport for ``ServicePackagesMixin.get_service_packages``: the exec path."""
return str(self._exec_ssh_command(command))
# ------------------------------------------------------------------ #
# Listening sockets (ListeningSocketsMixin supplies get_listening_sockets)
# ------------------------------------------------------------------ #
def _run_system_identity_command(self, command: str, *, privileged: bool) -> str:
"""The transport for ``SystemIdentityMixin.get_system_identity``: the exec path,
which runs as root either way."""
return str(self._exec_ssh_command(command))
def _run_listening_sockets_command(self, command: str, *, privileged: bool) -> str:
"""The transport for ``ListeningSocketsMixin.get_listening_sockets``:
the exec path, which runs as root either way."""
return str(self._exec_ssh_command(command))
# ------------------------------------------------------------------ #
# Packages (Debian APT)
# ------------------------------------------------------------------ #
def get_packages(self) -> list[_JsonDict]:
"""Return installed Debian packages with available-update info.
Installed list comes from ``dpkg-query`` via SSH (the Proxmox API
``/apt/installed`` endpoint is not implemented on PVE 8.x).
Available updates come from the Proxmox API ``/apt/update``.
"""
# Available updates from Proxmox API (keyed by package name)
upgradable: dict[str, str] = {}
try:
for upd in self._api.nodes(self._node_name).apt.update.get():
pkg = upd.get("Package", "")
if pkg:
upgradable[pkg] = upd.get("Version", "")
except Exception as exc:
logger.debug("Failed to fetch available APT updates: %s", exc)
# Installed packages via SSH dpkg-query
raw = self._exec_ssh_command(
"dpkg-query -W -f="
"'${Package}\\t${Version}\\t${db:Status-Status}\\t${Installed-Size}"
"\\t${source:Package}\\t${source:Version}\\n'"
" 2>/dev/null"
)
result: list[_JsonDict] = []
for line in raw.splitlines():
parts = line.strip().split("\t")
if len(parts) < 2:
continue
name = parts[0]
version = parts[1] if len(parts) > 1 else ""
status = parts[2] if len(parts) > 2 else "installed"
size_kb = parts[3] if len(parts) > 3 else "0"
# Debian source package (differs from the binary for split packages,
# e.g. openssh-server → openssh). Needed for accurate OSV matching.
source_package = parts[4] if len(parts) > 4 and parts[4] else name
# And its version, which is a different number from this package's.
#
# OSV states Debian ranges in *source* versions, so a consumer given
# the source package and only the binary version compares two
# unrelated numbers: libldb2 is 2:2.11.0+samba4.22.11+dfsg-… while
# its source, samba, is 2:4.22.11+dfsg-…. Reporting the source
# without its version is worse than reporting neither, because it
# looks usable. Every package on all three Proxmox nodes was in that
# state — the format string never asked for the field.
#
# dpkg leaves it empty when it equals `Version`, and so does an
# older dpkg that does not know the field at all.
source_version = parts[5] if len(parts) > 5 and parts[5] else version
if not name or status != "installed":
continue
size_bytes = int(size_kb) * 1024 if size_kb.isdigit() else 0
result.append(
{
"name": name,
"version": version,
"installed": True,
"description": "",
"size": size_bytes,
"source": "pve",
"source_package": source_package,
"source_version": source_version,
"upgrade_version": upgradable.get(name, ""),
}
)
return result
# ------------------------------------------------------------------ #
# Licenses (napalm-device-types' LicenseMixin)
# ------------------------------------------------------------------ #
def get_licenses(self) -> list[_JsonDict]:
"""The node's subscription, as the API reports it.
A fact for the caller: whether running without one is a problem is not
the driver's to say. An unreadable subscription reports nothing.
"""
sub = self._get_node_subscription()
raw = str(sub.get("status", ""))
if not raw:
return []
lic: _JsonDict = {
"name": "pve-subscription",
"status": _SUBSCRIPTION_STATUS.get(raw.lower(), "unknown"),
"detail": sub.get("message") or raw,
}
if sub.get("productname"):
lic["edition"] = sub["productname"]
if sub.get("nextduedate"):
lic["expires"] = sub["nextduedate"]
return [lic]
# ------------------------------------------------------------------ #
# Services (systemd, through napalm-device-types' SystemdServicesMixin)
# ------------------------------------------------------------------ #
def _run_service_command(self, command: str, *, privileged: bool, timeout: int) -> str:
"""The transport for ``SystemdServicesMixin``: the exec path, as root.
*privileged* needs nothing more on a node the driver reaches as root, and
the exec path keeps its own timeout.
"""
return str(self._exec_ssh_command(command))
# ------------------------------------------------------------------ #
# Available updates
# ------------------------------------------------------------------ #
def get_available_updates(self) -> list[_JsonDict]:
"""Return the node's upgradable packages, with origin and security status.
``apt list --upgradable`` over the exec path names each candidate's suite
(``trixie-security``); the APT API names only an Origin ("Debian",
"Proxmox") and is the fallback, with the security status unknown.
:raises Exception: when neither answers -- never an empty list for
"could not read".
"""
try:
updates = parse_apt_upgradable(self._exec_ssh_command(APT_UPGRADABLE_COMMAND) or "")
except ValueError as exc:
logger.debug("apt list over the exec path failed, using the API: %s", exc)
updates = self._updates_from_api()
return sorted(updates, key=lambda u: u["name"])
def _updates_from_api(self) -> list[_JsonDict]:
return [
{
"name": upd["Package"],
"current_version": upd.get("OldVersion", ""),
"new_version": upd.get("Version", ""),
"origin": upd.get("Origin"),
"security": None,
}
for upd in self._api.nodes(self._node_name).apt.update.get() # type: ignore[union-attr]
if upd.get("Package")
]
def refresh_available_updates(self) -> _JsonDict:
"""Resynchronise the node's package index (``POST nodes/{n}/apt/update``)."""
try:
task = self._api.nodes(self._node_name).apt.update.post() # type: ignore[union-attr]
except Exception as exc:
return {"success": False, "output": str(exc)}
return {"success": True, "output": f"Package index refresh started ({task})"}
def _run_host_status_command(self, command: str) -> str:
"""The transport for ``HostStatusMixin.get_host_status``: the exec path."""
return str(self._exec_ssh_command(command))
def apply_updates(self, packages: list[str]) -> _JsonDict:
"""Upgrade the given packages via ``apt-get install`` over SSH."""
for pkg in packages:
if not re.match(r"^[a-zA-Z0-9_\-\+\.]+$", pkg):
raise ValueError(f"Invalid package name: {pkg!r}")
pkg_args = " ".join(packages)
output, status = self._rc(
f"DEBIAN_FRONTEND=noninteractive apt-get install --only-upgrade -y {pkg_args}"
)
if status == 0:
return {"success": True, "output": output}
error = (
f"apt-get exited with status {status}"
if status >= 0
else "apt-get's exit status could not be read"
)
return {"success": False, "output": output, "error": error}
# ------------------------------------------------------------------ #
# SNMP / Health
# ------------------------------------------------------------------ #
@@ -548,59 +361,9 @@ class ProxmoxSystemMixin:
# caller's values. The node's shell runs as root.
# ------------------------------------------------------------------
def _rc(self, command: str) -> tuple[str, int]:
"""Run *command* and return its output and exit status (-1 when unknown)."""
out = self._exec_ssh_command(f"{{ {command}; }} 2>&1; echo __PVE_RC=$?")
text, _, tail = out.rpartition("__PVE_RC=")
try:
return text.strip(), int(tail.strip())
except ValueError:
return out.strip(), -1
def _step(self, command: str, done: str) -> dict:
output, status = self._rc(command)
return {"ok": status == 0, "output": done if status == 0 else output[-300:]}
# apt's proxy (napalm-device-types AptProxyMixin): the transports. The node
# is logged in as root, so every command runs as root.
def _run_apt_command(self, command: str) -> str:
return str(self._exec_ssh_command(command))
def _run_apt_root_command(self, command: str, done: str) -> dict:
return self._step(command, done)
def _write_apt_file(self, path: str, content: str) -> dict:
return self._write_root_file(path, content)
def _write_root_file(self, path: str, content: str) -> dict:
"""Write *content* to *path* as root, mode 644, through a temporary file."""
encoded = base64.b64encode(content.encode()).decode()
# No ``exit`` here: it would end the shell before ``_rc`` reads the status.
return self._step(
f'umask 077 && t=$(mktemp) && echo {encoded} | base64 -d > "$t" && '
f'install -m 644 -o root -g root "$t" {shlex.quote(path)}; '
's=$?; rm -f "$t"; [ "$s" -eq 0 ]',
f"wrote {path}",
)
def install_package(self, name: str, version: str = "") -> None:
"""apt-get install *name* (``PackageManagementMixin``); raises when apt fails."""
if not _PACKAGE_NAME.match(name) or (version and not _PACKAGE_NAME.match(version)):
raise ValueError(f"not a package name: {name!r}")
target = f"{name}={version}" if version else name
output, status = self._rc(f"DEBIAN_FRONTEND=noninteractive apt-get install -y {target}")
if status != 0:
raise RuntimeError(f"apt-get install {target} failed: {output[-300:]}")
def get_management_peer(self) -> str | None:
"""The address the node sees the SSH connection come from (``SSH_CONNECTION``)."""
out = self._exec_ssh_command('printf "%s\\n" "$SSH_CONNECTION"').split()
try:
return str(ipaddress.ip_address(out[0])) if out else None
except ValueError:
return None
def _install_snmp_agent(self) -> dict:
return self._step(
"DEBIAN_FRONTEND=noninteractive apt-get install -y snmpd snmp", "net-snmp installed"
-14
View File
@@ -194,17 +194,3 @@ class ProxmoxVMContractMixin:
return parse_vm_config(str(vmid), vm_type, cfg)
# -- the node itself --------------------------------------------------------
def reboot_host(self) -> None:
"""Restart this Proxmox node through the API (no SSH involved)."""
try:
self._node_api().status.post(command="reboot")
except Exception as exc:
raise RuntimeError(f"Reboot of node {self._node_name!r} refused: {exc}") from exc
def shutdown_host(self) -> None:
"""Shut this Proxmox node down through the API; its guests stop as on any shutdown."""
try:
self._node_api().status.post(command="shutdown")
except Exception as exc:
raise RuntimeError(f"Shutdown of node {self._node_name!r} refused: {exc}") from exc
+4 -3
View File
@@ -5,7 +5,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "napalm-proxmox"
version = "0.8.0"
description = "NAPALM driver for Proxmox VE (classic Linux networking, SDN, OVS)"
description = "NAPALM drivers for Proxmox VE (classic Linux networking, SDN, OVS) and Proxmox Backup Server"
readme = "README.md"
license = "Apache-2.0"
authors = [{ name = "Christian Manivong" }]
@@ -21,8 +21,8 @@ classifiers = [
requires-python = ">=3.12"
dependencies = [
"napalm>=5.0.0",
"napalm_device_types>=4.24.0",
"paramiko>=5.0.0", # CVE-2026-44405; imported directly for SSH fallback (driver.py)
"napalm_device_types>=4.25.0", # BackupServerDriver
"paramiko>=5.0.0", # CVE-2026-44405; imported directly for the node's SSH session (_base.py)
"proxmoxer>=2.0.0",
"netaddr>=0.9.0",
"requests>=2.31.0",
@@ -33,6 +33,7 @@ dev = ["pytest>=8", "pytest-cov>=5", "mypy>=1.11", "ruff>=0.6"]
[project.entry-points."napalm.drivers"]
proxmox = "napalm_proxmox:ProxmoxDriver"
proxmox_backup = "napalm_proxmox:ProxmoxBackupDriver"
[project.urls]
Repository = "https://git.netork.io/NAPALM/napalm-proxmox"
+2159
View File
File diff suppressed because it is too large. Load diff
+96
View File
@@ -0,0 +1,96 @@
"""A stand-in for proxmoxer's PBS client, answering from recorded responses.
``tests/fixtures/pbs_api.json`` holds what a real Proxmox Backup Server 4.2
answered (sanitised). Paths a test adds on top -- backup groups, snapshots and
jobs, which that server did not have yet -- follow the shapes of the PBS API
schema (``pbs.proxmox.com/docs/api-viewer``).
"""
from __future__ import annotations
import copy
import json
from pathlib import Path
from typing import Any
from proxmoxer.core import ResourceException
RECORDED: dict[str, Any] = json.loads(
(Path(__file__).parent / "fixtures" / "pbs_api.json").read_text()
)
def _key(path: str, params: dict[str, Any]) -> str:
query = "&".join(f"{k}={params[k]}" for k in sorted(params))
return f"{path.strip('/')}?{query}" if query else path.strip("/")
class _Resource:
"""proxmoxer's chained form: ``api.nodes("localhost").status.get()``."""
def __init__(self, api: FakePbsApi, path: str) -> None:
self._api, self._path = api, path
def __getattr__(self, name: str) -> _Resource:
return _Resource(self._api, f"{self._path}/{name}")
def __call__(self, segment: str) -> _Resource:
return _Resource(self._api, f"{self._path}/{segment}")
def get(self, **params: Any) -> Any:
return self._api.get(self._path, **params)
def post(self, **params: Any) -> Any:
return self._api.post(self._path, **params)
class FakePbsApi:
"""``get``/``post``/``put``/``delete`` by path, like ``ProxmoxAPI.get("admin/gc")``,
or chained, like ``ProxmoxAPI.nodes("localhost").status.get()``.
A response is looked up as ``"path?k=v&..."`` first, then as ``"path"``. A
value that is an exception is raised. Every call is recorded in ``calls``.
"""
def __init__(self, **overrides: Any) -> None:
self.responses: dict[str, Any] = copy.deepcopy(RECORDED)
self.responses.update(overrides)
self.calls: list[tuple[str, str, dict[str, Any]]] = []
def _answer(self, method: str, path: str, params: dict[str, Any]) -> Any:
self.calls.append((method, path.strip("/"), params))
for key in (f"{method} {_key(path, params)}", f"{method} {path.strip('/')}"):
if key in self.responses:
return self._value(self.responses[key])
if method == "GET":
for key in (_key(path, params), path.strip("/")):
if key in self.responses:
return self._value(self.responses[key])
raise ResourceException(404, "Not Found", f"no recorded answer for {method} {path}")
@staticmethod
def _value(value: Any) -> Any:
if isinstance(value, Exception):
raise value
return copy.deepcopy(value)
def __getattr__(self, name: str) -> _Resource:
if name.startswith("_"):
raise AttributeError(name)
return _Resource(self, name)
def get(self, path: str, **params: Any) -> Any:
return self._answer("GET", path, params)
def post(self, path: str, **params: Any) -> Any:
return self._answer("POST", path, params)
def put(self, path: str, **params: Any) -> Any:
return self._answer("PUT", path, params)
def delete(self, path: str, **params: Any) -> Any:
return self._answer("DELETE", path, params)
def forbidden() -> ResourceException:
return ResourceException(403, "Forbidden", "permission check failed")
-4
View File
@@ -2,7 +2,6 @@
"coverage_floor": 68.0,
"function_complexity": {
"napalm_proxmox/driver.py::ProxmoxDriver._resolve_node": 13,
"napalm_proxmox/driver.py::ProxmoxDriver.get_facts": 16,
"napalm_proxmox/interfaces_mixin.py::ProxmoxInterfaceMixin.get_interfaces_counters": 14,
"napalm_proxmox/interfaces_mixin.py::ProxmoxInterfaceMixin.get_mac_address_table": 22,
"napalm_proxmox/routing_mixin.py::ProxmoxRoutingMixin.get_route_to": 13,
@@ -15,7 +14,6 @@
"napalm_proxmox/sdn_mixin.py::ProxmoxSDNMixin.get_vlans": 16,
"napalm_proxmox/sdn_mixin.py::ProxmoxSDNMixin.set_vlan": 13,
"napalm_proxmox/system_mixin.py::ProxmoxSystemMixin.get_disk_smart": 22,
"napalm_proxmox/system_mixin.py::ProxmoxSystemMixin.get_packages": 16,
"napalm_proxmox/utils.py::addresses_from_node_network": 15,
"napalm_proxmox/vm_contract_mixin.py::parse_vm_config": 16,
"napalm_proxmox/vm_mixin.py::ProxmoxVMMixin._get_vm_disk_and_boot": 16,
@@ -25,14 +23,12 @@
"napalm_proxmox/vm_provision_mixin.py::ProxmoxVMProvisionMixin.get_vm_status": 16
},
"function_lines": {
"napalm_proxmox/driver.py::ProxmoxDriver.get_facts": 82,
"napalm_proxmox/interfaces_mixin.py::ProxmoxInterfaceMixin.get_mac_address_table": 71,
"napalm_proxmox/routing_mixin.py::ProxmoxRoutingMixin.ping": 78,
"napalm_proxmox/sdn_mixin.py::ProxmoxSDNMixin._get_vm_vlan_tags": 54,
"napalm_proxmox/sdn_mixin.py::ProxmoxSDNMixin.get_vlans": 66,
"napalm_proxmox/sdn_mixin.py::ProxmoxSDNMixin.set_vlan": 70,
"napalm_proxmox/system_mixin.py::ProxmoxSystemMixin.get_disk_smart": 74,
"napalm_proxmox/system_mixin.py::ProxmoxSystemMixin.get_packages": 66,
"napalm_proxmox/system_mixin.py::ProxmoxSystemMixin.set_hostname": 114,
"napalm_proxmox/vm_mixin.py::ProxmoxVMMixin.get_vm_interfaces": 179,
"napalm_proxmox/vm_mixin.py::ProxmoxVMMixin.get_vms": 114,
+54
View File
@@ -0,0 +1,54 @@
"""What the PVE driver hands proxmoxer, pinned before the connection moved into a
base it shares with the Proxmox Backup Server driver (#52)."""
from __future__ import annotations
from unittest.mock import patch
import pytest
from napalm_proxmox.driver import ProxmoxDriver
from tests.conftest import _build_mock_api
def _kwargs(username: str = "root", password: str = "secret", **optional_args) -> dict:
drv = ProxmoxDriver("pve1", username, password, optional_args=optional_args)
with patch("napalm_proxmox.driver.ProxmoxAPI", return_value=_build_mock_api()) as api:
drv.open()
return api.call_args.kwargs
def test_password_login_on_the_pve_port():
assert _kwargs() == {
"host": "pve1",
"user": "root@pam",
"password": "secret",
"port": 8006,
"verify_ssl": True,
"backend": "https",
"service": "PVE",
}
def test_port_and_realm_from_the_arguments():
kwargs = _kwargs(port=443, realm="pve")
assert (kwargs["port"], kwargs["user"]) == (443, "root@pve")
def test_a_username_with_a_realm_keeps_it():
assert _kwargs("admin@ldap")["user"] == "admin@ldap"
@pytest.mark.parametrize("arg", ["verify_ssl", "ssl_verify", "verify"])
def test_tls_verification_under_each_name(arg):
assert _kwargs(**{arg: False})["verify_ssl"] is False
def test_token_without_a_user_part_is_passed_as_given():
kwargs = _kwargs(token_name="mytoken", token_value="s3cret")
assert (kwargs["user"], kwargs["token_name"], kwargs["token_value"]) == (
"root@pam",
"mytoken",
"s3cret",
)
assert "password" not in kwargs
+2 -2
View File
@@ -60,8 +60,8 @@ def test_the_query_asks_for_the_field():
not, so no amount of parsing could have produced one."""
import inspect
from napalm_proxmox import system_mixin
from napalm_proxmox import host_mixin
source = inspect.getsource(system_mixin.ProxmoxSystemMixin.get_packages)
source = inspect.getsource(host_mixin.ProxmoxHostMixin.get_packages)
assert "source:Version" in source
+553
View File
@@ -0,0 +1,553 @@
"""The backup-server role of the PBS driver: datastores, groups, jobs, remotes and
the actions on them (#52)."""
from __future__ import annotations
import pytest
from proxmoxer.core import ResourceException
from napalm_proxmox import ProxmoxBackupDriver
from tests.pbs_fake import FakePbsApi, forbidden
# --------------------------------------------------------------------------- #
# What the recorded server did not have yet, in the shapes of the PBS API schema
# --------------------------------------------------------------------------- #
T0 = 1793487600 # 2026-11-01
GROUPS_ROOT = [
{
"backup-type": "vm",
"backup-id": "101",
"backup-count": 3,
"last-backup": T0 + 2 * 86400,
"owner": "backup@pbs",
"files": ["drive-scsi0.img.fidx", "index.json.blob"],
"comment": "web-01",
},
{
"backup-type": "ct",
"backup-id": "200",
"backup-count": 1,
"last-backup": T0,
"owner": "backup@pbs",
"files": ["root.pxar.didx"],
},
]
SNAPSHOTS_ROOT = [
{
"backup-type": "vm",
"backup-id": "101",
"backup-time": T0,
"size": 1000,
"protected": False,
"files": [],
"verification": {"state": "ok", "upid": "UPID:pbs:1"},
},
{
"backup-type": "vm",
"backup-id": "101",
"backup-time": T0 + 86400,
"size": 1100,
"protected": False,
"files": [],
"verification": {"state": "failed", "upid": "UPID:pbs:2"},
},
{
"backup-type": "vm",
"backup-id": "101",
"backup-time": T0 + 2 * 86400,
"size": 1200,
"protected": True,
"files": [],
"comment": "before upgrade",
},
{
"backup-type": "ct",
"backup-id": "200",
"backup-time": T0,
"protected": False,
"files": [],
},
]
GROUPS_PROD = [
{"backup-type": "host", "backup-id": "db-01", "backup-count": 0, "last-backup": T0, "files": []}
]
GC = [
{
"store": "backup",
"schedule": "daily",
"last-run-state": "OK",
"last-run-endtime": T0 + 3600,
"next-run": T0 + 86400,
"upid": "UPID:pbs:gc",
"disk-bytes": 0,
"disk-chunks": 0,
"index-data-bytes": 0,
"index-file-count": 0,
"pending-bytes": 0,
"pending-chunks": 0,
"removed-bad": 0,
"removed-bytes": 0,
"removed-chunks": 0,
"still-bad": 0,
}
]
VERIFY = [
{
"id": "v-weekly",
"store": "backup",
"schedule": "sat 03:00",
"last-run-state": "WARNINGS: 2",
"last-run-endtime": T0,
"next-run": T0 + 7 * 86400,
}
]
PRUNE = [{"id": "p-daily", "store": "backup", "ns": "prod", "schedule": "daily", "disable": True}]
SYNC = [
{
"id": "offsite",
"store": "backup",
"remote": "pbs-offsite",
"remote-store": "mirror",
"sync-direction": "push",
"schedule": "hourly",
"last-run-state": "connection refused",
"last-run-endtime": T0,
"next-run": T0 + 3600,
"comment": "second site",
}
]
TAPE = [
{
"id": "t-monthly",
"store": "backup",
"pool": "monthly",
"drive": "lto8",
"schedule": "monthly",
"last-run-upid": "UPID:pbs:tape",
}
]
REMOTES = [
{
"name": "pbs-offsite",
"host": "pbs2.example.com",
"port": 8007,
"auth-id": "sync@pbs",
"fingerprint": "bb:" * 31 + "bb",
"password": "never-reported",
}
]
ALL_RIGHTS = {"/": {"Datastore.Audit": True, "Sys.Audit": True, "Remote.Audit": True}}
def _driver(**responses) -> tuple[ProxmoxBackupDriver, FakePbsApi]:
drv = ProxmoxBackupDriver("pbs1", "root", "secret")
api = FakePbsApi(**responses)
drv._api = api
drv._node_name = "localhost"
return drv, api
def _with_groups(**more):
return _driver(
**{
"admin/datastore/backup/namespace": [{"ns": ""}, {"ns": "prod"}],
"admin/datastore/backup/groups": GROUPS_ROOT,
"admin/datastore/backup/groups?ns=prod": GROUPS_PROD,
"admin/datastore/backup/snapshots": SNAPSHOTS_ROOT,
"admin/datastore/backup/snapshots?ns=prod": [],
**more,
}
)
# --------------------------------------------------------------------------- #
# Rights: a list the token may not see comes back empty, not refused
# --------------------------------------------------------------------------- #
class TestRights:
def test_a_token_without_datastore_audit_is_refused_not_read_as_empty(self):
"""A privilege-separated token without an ACL entry gets [] for the
datastore list; that is no answer, not a server without datastores."""
drv, _ = _driver(**{"access/permissions": {}, "admin/datastore": []})
with pytest.raises(PermissionError, match="Datastore.Audit"):
drv.get_backup_datastores()
def test_remotes_need_remote_audit(self):
drv, _ = _driver(**{"config/remote": []}) # recorded rights lack Remote.Audit
with pytest.raises(PermissionError, match="Remote.Audit"):
drv.get_backup_remotes()
def test_a_right_on_a_parent_path_counts(self):
drv, _ = _driver(**{"access/permissions": ALL_RIGHTS, "config/remote": REMOTES})
assert [r["name"] for r in drv.get_backup_remotes()] == ["pbs-offsite"]
def test_a_refused_path_raises(self):
drv, _ = _driver(**{"status/datastore-usage": forbidden()})
with pytest.raises(Exception, match="403"):
drv.get_backup_datastores()
# --------------------------------------------------------------------------- #
# Datastores (recorded)
# --------------------------------------------------------------------------- #
class TestDatastores:
def test_recorded_removable_datastore(self):
drv, _ = _driver()
assert drv.get_backup_datastores() == [
{
"name": "backup",
# a removable datastore's path is relative to its mount point
"path": None,
"backend": "removable",
"total": 999715438592,
"used": 34838138880,
"available": 964877299712,
"estimated_full": None,
"maintenance": None,
"comment": None,
}
]
def test_s3_backend_estimate_and_maintenance(self):
drv, _ = _driver(
**{
"config/datastore": [
{
"name": "cold",
"path": "/var/cache/pbs/cold",
"backend": "type=s3,client=minio,bucket=pbs",
"maintenance-mode": "type=read-only,message=migrating",
"comment": "object store",
}
],
"status/datastore-usage": [
{
"store": "cold",
"backend-type": "s3",
"mount-status": "nonremovable",
"total": 100,
"used": 40,
"avail": 60,
"estimated-full-date": 1800000000,
}
],
}
)
(store,) = drv.get_backup_datastores()
assert (store["backend"], store["path"], store["estimated_full"]) == (
"s3",
"/var/cache/pbs/cold",
1800000000,
)
assert (store["maintenance"], store["comment"]) == ("read-only", "object store")
def test_an_unmounted_removable_datastore_is_offline(self):
drv, _ = _driver(
**{
"status/datastore-usage": [
{"store": "backup", "backend-type": "filesystem", "mount-status": "notmounted"}
]
}
)
(store,) = drv.get_backup_datastores()
assert (store["maintenance"], store["total"], store["used"]) == ("unmounted", None, None)
# --------------------------------------------------------------------------- #
# Groups and snapshots
# --------------------------------------------------------------------------- #
class TestGroups:
def test_recorded_empty_datastore_has_no_groups(self):
drv, _ = _driver(
**{"admin/datastore/backup/groups": [], "admin/datastore/backup/snapshots": []}
)
assert drv.get_backup_groups() == []
def test_each_group_with_its_newest_snapshot(self):
drv, _ = _with_groups()
groups = {(g["namespace"], g["type"], g["id"]): g for g in drv.get_backup_groups()}
assert groups[("", "vm", "101")] == {
"datastore": "backup",
"namespace": "",
"type": "vm",
"id": "101",
"owner": "backup@pbs",
"count": 3,
"newest": {
"time": T0 + 2 * 86400,
"size": 1200,
"verify": None, # the newest was never verified
"protected": True,
"comment": "before upgrade",
},
"oldest_time": T0,
"comment": "web-01",
}
assert groups[("", "ct", "200")]["newest"]["size"] is None
def test_namespaces_are_walked(self):
drv, _ = _with_groups()
prod = [g for g in drv.get_backup_groups() if g["namespace"] == "prod"]
assert [(g["type"], g["id"], g["count"], g["newest"]) for g in prod] == [
("host", "db-01", 0, None)
]
def test_one_datastore_only(self):
drv, api = _with_groups()
drv.get_backup_groups("backup")
assert all("/other/" not in path for _, path, _ in api.calls)
def test_an_unmounted_removable_datastore_is_left_out_of_the_listing(self):
"""Recorded: the USB disk behind the datastore was unplugged. Its content
cannot be read (400), which must not fail the listing of every datastore;
get_backup_datastores reports it as "unmounted"."""
not_mounted = ResourceException(400, "Bad Request", "datastore 'backup' is not mounted")
drv, _ = _driver(
**{
"admin/datastore": [
{"backend-type": "filesystem", "mount-status": "notmounted", "store": "backup"}
],
"admin/datastore/backup/namespace": not_mounted,
}
)
assert drv.get_backup_groups() == []
with pytest.raises(ValueError, match="not mounted"):
drv.get_backup_groups("backup")
def test_snapshots_of_one_group_newest_first(self):
drv, api = _with_groups(
**{
"admin/datastore/backup/snapshots?backup-id=101&backup-type=vm": [
s for s in SNAPSHOTS_ROOT if s["backup-id"] == "101"
]
}
)
snaps = drv.get_backup_snapshots("backup", "", "vm", "101")
assert [(s["time"], s["verify"]) for s in snaps] == [
(T0 + 2 * 86400, None),
(T0 + 86400, "failed"),
(T0, "ok"),
]
def test_snapshots_in_a_namespace_name_it(self):
drv, api = _with_groups(
**{"admin/datastore/backup/snapshots?backup-id=db-01&backup-type=host&ns=prod": []}
)
assert drv.get_backup_snapshots("backup", "prod", "host", "db-01") == []
# --------------------------------------------------------------------------- #
# Jobs
# --------------------------------------------------------------------------- #
class TestJobs:
def test_recorded_server_has_only_its_unscheduled_gc(self):
drv, _ = _driver()
assert drv.get_backup_jobs() == [
{
"id": "backup",
"kind": "gc",
"datastore": "backup",
"namespace": None,
"schedule": None,
"enabled": False,
"last_run": None,
"last_state": None,
"last_error": None,
"next_run": None,
"remote": None,
"direction": None,
"comment": None,
}
]
def test_every_kind_and_how_its_last_run_ended(self):
drv, _ = _driver(
**{
"admin/gc": GC,
"admin/verify": VERIFY,
"admin/prune": PRUNE,
"admin/sync": SYNC,
"tape/backup": TAPE,
}
)
jobs = {(j["kind"], j["id"]): j for j in drv.get_backup_jobs()}
assert set(jobs) == {
("gc", "backup"),
("verify", "v-weekly"),
("prune", "p-daily"),
("sync", "offsite"),
("tape", "t-monthly"),
}
gc = jobs[("gc", "backup")]
assert (gc["enabled"], gc["last_state"], gc["last_run"], gc["next_run"]) == (
True,
"ok",
T0 + 3600,
T0 + 86400,
)
verify = jobs[("verify", "v-weekly")]
assert (verify["last_state"], verify["last_error"]) == ("warning", "WARNINGS: 2")
prune = jobs[("prune", "p-daily")]
assert (prune["enabled"], prune["namespace"], prune["last_state"]) == (False, "prod", None)
sync = jobs[("sync", "offsite")]
assert (sync["last_state"], sync["last_error"]) == ("error", "connection refused")
assert (sync["remote"], sync["direction"], sync["comment"]) == (
"pbs-offsite",
"push",
"second site",
)
tape = jobs[("tape", "t-monthly")]
assert (tape["last_state"], tape["last_run"]) == ("running", None)
def test_a_sync_without_a_remote_is_local(self):
local = [{**SYNC[0], "remote": None, "sync-direction": None, "id": "local"}]
drv, _ = _driver(
**{"admin/sync": local, "admin/verify": [], "admin/prune": [], "tape/backup": []}
)
sync = next(j for j in drv.get_backup_jobs() if j["kind"] == "sync")
assert (sync["remote"], sync["direction"]) == (None, "pull")
def test_a_failed_listing_fails_the_whole_read(self):
drv, _ = _driver(**{"admin/sync": forbidden()})
with pytest.raises(Exception, match="403"):
drv.get_backup_jobs()
# --------------------------------------------------------------------------- #
# Remotes
# --------------------------------------------------------------------------- #
def test_remotes_never_carry_their_secret():
drv, _ = _driver(**{"access/permissions": ALL_RIGHTS, "config/remote": REMOTES})
assert drv.get_backup_remotes() == [
{
"name": "pbs-offsite",
"host": "pbs2.example.com",
"port": 8007,
"fingerprint": "bb:" * 31 + "bb",
"auth_id": "sync@pbs",
"comment": None,
}
]
# --------------------------------------------------------------------------- #
# Actions and tasks
# --------------------------------------------------------------------------- #
SNAPSHOT = {
"datastore": "backup",
"namespace": "",
"group_type": "vm",
"group_id": "101",
"time": T0,
}
class TestActions:
def test_garbage_collection(self):
drv, api = _driver(**{"POST admin/datastore/backup/gc": "UPID:pbs:gc-now"})
assert drv.start_garbage_collection("backup") == {"task": "UPID:pbs:gc-now"}
@pytest.mark.parametrize(
("kind", "method", "path"),
[
("verify", "POST", "admin/verify/j1/run"),
("prune", "POST", "admin/prune/j1/run"),
("sync", "POST", "admin/sync/j1/run"),
("tape", "POST", "tape/backup/j1"),
("gc", "POST", "admin/datastore/j1/gc"),
],
)
def test_run_a_job_now(self, kind, method, path):
drv, api = _driver(**{f"{method} {path}": "UPID:pbs:run"})
assert drv.run_backup_job(kind, "j1") == {"task": "UPID:pbs:run"}
assert api.calls[-1][:2] == (method, path)
def test_an_unknown_kind_is_refused(self):
drv, _ = _driver()
with pytest.raises(ValueError, match="kind"):
drv.run_backup_job("restore", "j1") # type: ignore[arg-type]
def test_a_run_without_a_task_id_is_an_error(self):
drv, _ = _driver(**{"POST admin/sync/j1/run": None})
with pytest.raises(RuntimeError, match="task"):
drv.run_backup_job("sync", "j1")
def test_verify_one_snapshot(self):
drv, api = _driver(**{"POST admin/datastore/backup/verify": "UPID:pbs:verify"})
assert drv.verify_backup_snapshot(SNAPSHOT) == {"task": "UPID:pbs:verify"}
assert api.calls[-1][2] == {"backup-type": "vm", "backup-id": "101", "backup-time": T0}
def test_namespace_is_passed_where_given(self):
drv, api = _driver(**{"POST admin/datastore/backup/verify": "UPID:pbs:verify"})
drv.verify_backup_snapshot({**SNAPSHOT, "namespace": "prod"})
assert api.calls[-1][2]["ns"] == "prod"
def test_protect_and_unprotect(self):
drv, api = _driver(**{"PUT admin/datastore/backup/protected": None})
drv.set_backup_snapshot_protected(SNAPSHOT, True)
assert api.calls[-1] == (
"PUT",
"admin/datastore/backup/protected",
{"backup-type": "vm", "backup-id": "101", "backup-time": T0, "protected": 1},
)
drv.set_backup_snapshot_protected(SNAPSHOT, False)
assert api.calls[-1][2]["protected"] == 0
def test_delete(self):
drv, api = _driver(**{"DELETE admin/datastore/backup/snapshots": None})
drv.delete_backup_snapshot(SNAPSHOT)
assert api.calls[-1] == (
"DELETE",
"admin/datastore/backup/snapshots",
{"backup-type": "vm", "backup-id": "101", "backup-time": T0},
)
class TestTasks:
UPID = "UPID:pbs:000003E8:0000A000:00000002:6AE8E898:garbage_collection:backup:root@pam!netork:"
def _task(self, status, log=()):
return _driver(
**{
f"nodes/localhost/tasks/{self.UPID}/status": status,
f"nodes/localhost/tasks/{self.UPID}/log?limit=0": [
{"n": i + 1, "t": t} for i, t in enumerate(log)
],
}
)[0]
def test_running(self):
drv = self._task({"upid": self.UPID, "status": "running"}, ["starting garbage collection"])
assert drv.get_backup_task(self.UPID) == {
"task": self.UPID,
"running": True,
"state": "running",
"error": None,
"log": ["starting garbage collection"],
}
@pytest.mark.parametrize(
("exitstatus", "state", "error"),
[
("OK", "ok", None),
("WARNINGS: 1", "warning", "WARNINGS: 1"),
("unable to acquire lock", "error", "unable to acquire lock"),
],
)
def test_finished(self, exitstatus, state, error):
drv = self._task({"upid": self.UPID, "status": "stopped", "exitstatus": exitstatus})
task = drv.get_backup_task(self.UPID)
assert (task["running"], task["state"], task["error"]) == (False, state, error)
+96
View File
@@ -0,0 +1,96 @@
"""Proxmox Backup Server: the second driver of this package (#52).
Its API and web UI listen on 8007, not on PVE's 8006, and its node is always
``localhost``: a PBS has no cluster to look itself up in.
"""
from __future__ import annotations
import tomllib
from pathlib import Path
from unittest.mock import MagicMock, patch
import pytest
from napalm.base.exceptions import ConnectionException
from napalm_device_types.roles import role_keys_of
from napalm_proxmox import ProxmoxBackupDriver
def _open(username: str = "root", password: str = "secret", **optional_args):
drv = ProxmoxBackupDriver("pbs1", username, password, optional_args=optional_args)
api = MagicMock()
api.version.get.return_value = {"version": "4.0.14", "release": "4.0", "repoid": "abc"}
with patch("napalm_proxmox.backup_driver.ProxmoxAPI", return_value=api) as cls:
drv.open()
return drv, api, cls.call_args.kwargs
class TestOpen:
def test_password_login_on_the_pbs_port(self):
_, _, kwargs = _open()
assert kwargs == {
"host": "pbs1",
"user": "root@pam",
"password": "secret",
"port": 8007,
"verify_ssl": True,
"backend": "https",
"service": "PBS",
}
def test_token_is_split_for_proxmoxer(self):
_, _, kwargs = _open(token_name="root@pam!netork", token_value="s3cret")
assert (kwargs["user"], kwargs["token_name"], kwargs["token_value"]) == (
"root@pam",
"netork",
"s3cret",
)
assert "password" not in kwargs
def test_port_from_the_arguments(self):
assert _open(port=443)[2]["port"] == 443
def test_node_is_localhost_without_asking_a_cluster(self):
drv, api, _ = _open()
assert drv._node_name == "localhost"
api.cluster.status.get.assert_not_called()
def test_open_proves_the_login(self):
"""proxmoxer logs in lazily for a token; open() must fail on a bad one,
not the first getter."""
drv = ProxmoxBackupDriver("pbs1", "root", "", optional_args={"token_name": "a!b"})
api = MagicMock()
api.version.get.side_effect = Exception("401 Unauthorized")
with patch("napalm_proxmox.backup_driver.ProxmoxAPI", return_value=api):
with pytest.raises(ConnectionException, match="401"):
drv.open()
def test_alive_and_closed(self):
drv, api, _ = _open()
assert drv.is_alive() == {"is_alive": True}
drv.close()
assert drv.is_alive() == {"is_alive": False}
class TestDriverFacts:
def test_backup_server_first_storage_second(self):
assert role_keys_of(ProxmoxBackupDriver)[:2] == ["backup_server", "storage"]
def test_found_on_8007_by_its_own_banner(self):
assert [(p.scheme, p.port) for p in ProxmoxBackupDriver.PORT_SPECS] == [("https", 8007)]
mandatory = [r.pattern for r in ProxmoxBackupDriver.HTTP_FINGERPRINT if r.mandatory]
assert mandatory == ["proxmox backup server"]
def test_names(self):
assert ProxmoxBackupDriver.DRIVER_NAME == "proxmox_backup"
assert ProxmoxBackupDriver.platform == "proxmox_backup"
assert ProxmoxBackupDriver.VENDOR == "Proxmox"
assert ProxmoxBackupDriver.FULL_UPGRADE is True
def test_entry_point(self):
project = tomllib.loads((Path(__file__).parents[1] / "pyproject.toml").read_text())
assert project["project"]["entry-points"]["napalm.drivers"] == {
"proxmox": "napalm_proxmox:ProxmoxDriver",
"proxmox_backup": "napalm_proxmox:ProxmoxBackupDriver",
}
+193
View File
@@ -0,0 +1,193 @@
"""The PBS driver as a host: facts, interfaces, environment, disks and pools for
the storage role, subscription and packages (#52)."""
from __future__ import annotations
import pytest
from napalm.base.exceptions import ConnectionException
from napalm_proxmox import ProxmoxBackupDriver
from tests.pbs_fake import FakePbsApi
DMI = "QEMU\nStandard PC (Q35 + ICH9, 2009)\npc-q35-10.0\n\n"
LINKS = "lo 00:00:00:00:00:00\nnic0 02:00:00:00:00:80\n"
def _driver(ssh: dict[str, str] | None = None, **responses) -> ProxmoxBackupDriver:
"""A PBS driver on recorded answers; *ssh* maps a command's start to its output,
and without it the server has no SSH login."""
drv = ProxmoxBackupDriver("192.0.2.80", "root", "", optional_args={"token_name": "a!b"})
api = FakePbsApi(**responses)
drv._api = api
drv._node_name = "localhost"
def exec_ssh(command: str) -> str:
if ssh is None:
raise ConnectionException("no SSH credential for 192.0.2.80")
for start, output in ssh.items():
if start in command:
return output
return ""
drv._exec_ssh_command = exec_ssh # type: ignore[method-assign]
return drv
SSH = {"hostname -f": "pbs.example.com", "dmi": DMI, "/sys/class/net": LINKS}
class TestFacts:
def test_from_the_api_and_ssh(self):
facts = _driver(SSH).get_facts()
assert facts == {
"uptime": 5234.0,
"vendor": "QEMU",
"model": "Standard PC (Q35 + ICH9, 2009)",
"hostname": "pbs",
"fqdn": "pbs.example.com",
"os_version": "Proxmox Backup Server 4.2",
"serial_number": "",
"interface_list": ["nic0"],
"running_kernel": "7.0.0-3-pve",
}
def test_without_ssh_the_api_alone(self):
"""A token-only server: no shell, so no DMI and no host name of its own."""
facts = _driver().get_facts()
assert (facts["hostname"], facts["fqdn"]) == ("192.0.2.80", "192.0.2.80")
assert (facts["vendor"], facts["model"]) == (
"Proxmox Server Solutions GmbH",
"Proxmox Backup Server",
)
assert facts["running_kernel"] == "7.0.0-3-pve"
class TestInterfaces:
def test_recorded_interface_with_its_mac_over_ssh(self):
assert _driver(SSH).get_interfaces() == {
"nic0": {
"is_up": True,
"is_enabled": True,
"description": "",
"last_flapped": -1.0,
"speed": -1.0,
"mtu": 1500,
"mac_address": "02:00:00:00:00:80",
}
}
def test_without_ssh_no_mac(self):
assert _driver().get_interfaces()["nic0"]["mac_address"] == ""
def test_addresses(self):
assert _driver().get_interfaces_ip() == {
"nic0": {"ipv4": {"192.0.2.80": {"prefix_length": 24}}}
}
def test_environment_from_the_node_status():
env = _driver().get_environment()
assert env["cpu"] == {"0": {"%usage": 1.49}}
assert env["memory"] == {"available_ram": 4103909376, "used_ram": 541790208}
assert (env["fans"], env["temperature"], env["power"]) == ({}, {}, {})
class TestStorageRole:
def test_recorded_disks(self):
disks = _driver().get_disks()
assert [d["slot"] for d in disks] == ["sda", "sdc"]
assert disks[1] == {
"slot": "sdc",
"model": "Samsung_SSD_850_EVO_1TB",
"serial": "S0000000000000X",
"vendor": "ASMT",
# as the server reports it: behind a USB bridge it took the SSD for a disk
"type": "hdd",
"size": 1000204886016,
"rpm": 0,
"temperature": -1,
"health": "healthy",
"pool": "",
}
assert disks[0]["type"] == "ssd"
@pytest.mark.parametrize(
("status", "health"), [("passed", "healthy"), ("failed", "failed"), ("unknown", "unknown")]
)
def test_disk_health(self, status, health):
drv = _driver(
**{
"nodes/localhost/disks/list": [
{"name": "sdc", "devpath": "/dev/sdc", "status": status}
]
}
)
assert drv.get_disks()[0]["health"] == health
def test_recorded_server_has_no_zfs_pool(self):
assert _driver().get_disk_pools() == {}
def test_zfs_pool(self):
drv = _driver(
**{
"nodes/localhost/disks/zfs": [
{
"name": "tank",
"size": 1000,
"alloc": 400,
"free": 600,
"health": "DEGRADED",
"dedup": 1.0,
"frag": 3,
}
]
}
)
assert drv.get_disk_pools() == {
"tank": {
"name": "tank",
"type": "zfs",
"level": "",
"status": "degraded",
"total": 1000,
"used": 400,
"available": 600,
"disks": [],
"auto_expand": False,
"dedup": False,
"compression": "",
}
}
class TestHost:
def test_subscription_is_the_pbs_one(self):
assert _driver().get_licenses() == [
{"name": "pbs-subscription", "status": "none", "detail": "There is no subscription key"}
]
def test_reboot_and_shutdown_through_the_api(self):
drv = _driver(**{"POST nodes/localhost/status": None})
drv.reboot_host()
drv.shutdown_host()
calls = [c for c in drv._api.calls if c[0] == "POST"]
assert calls == [
("POST", "nodes/localhost/status", {"command": "reboot"}),
("POST", "nodes/localhost/status", {"command": "shutdown"}),
]
def test_packages_name_their_source(self):
drv = _driver({"dpkg-query": "proxmox-backup-server\t4.2.0-1\tinstalled\t100\t\t\n"})
(pkg,) = drv.get_packages()
assert (pkg["name"], pkg["source"]) == ("proxmox-backup-server", "pbs")
def test_os_capabilities_present(self):
for name in (
"get_kernel_facts",
"get_listening_sockets",
"get_available_updates",
"apply_updates",
"get_ssh_server_algorithms",
"get_services",
):
assert hasattr(ProxmoxBackupDriver, name), name