- CI moves to .gitea/workflows/ci.yml and calls the shared driver workflow
of NAPALM/driver-standard: conformance, ruff, strict mypy, tests on
3.10-3.13 and at the lowest declared versions, bandit, pip-audit, build,
and on a release tag the upload to PyPI.
- tests/quality_baseline.json records today's long and complex functions
and the one file over 1000 lines; they may shrink, not grow.
- Coverage floor 95 % (measured 95.15 %).
- CHANGELOG.md starts here; project links point at git.netork.io; the
license is the SPDX string PEP 639 wants.
- Long docstring and command lines are wrapped; recorded device output in
tests/ keeps its lines (E501 is off there).
- host_firewall: the negation token is the named constant _NEGATION. bandit
read token == "!" as a hardcoded password.
- ssh_host_keys: _known_hosts_file takes the expected key as an argument
instead of asserting it; both callers had already returned when it is None.
- ssh_server: the staged path is on the device, not a local temp file
(nosec B108).
- tests: raise ... from exc, zip(strict=True).
Mechanical: ruff format, and ruff check --fix with the rule set every NAPALM
driver uses (E, F, I, UP, B at line length 100). typing.List/Dict/Optional
become builtin generics and X | None, imports are sorted, the coding cookies
are gone. Two tests that read the source text follow the new spelling.
What ruff cannot fix comes in the next commit.
get_host_firewall() reads `nft -j list ruleset` as root and returns every
chain with its family, table, hook, priority and policy, and every rule as
matches and a verdict. Rules iptables-nft wrote show their extensions in nft's
JSON only as an opaque `xt`, so the tables iptables manages are taken from
iptables-save instead; a legacy iptables is kept beside nft's tables, since
both apply. A condition the parse does not model is an `unknown` match with
its text, never left out. Without root neither tool can be read, and
HostFirewallUnavailable is raised rather than a host without rules.
A driver supplies only _run_host_firewall_command(command, privileged=).
Version 4.21.0.
get_service_packages() returns, per systemd service, the installed packages
it is made of: the one shipping its unit file and the one shipping the
binary its ExecStart runs -- not the libraries that binary links against.
One read-only command: systemctl show for every loaded service unit, then
one dpkg -S for all paths (each also across the /usr merge), or rpm -qf per
path. The report is framed; one cut short raises. A driver supplies
_run_service_packages_command, as for KernelFactsMixin.
Version 4.20.0.
Drivers get the host keys to expect in optional_args["ssh_host_keys"] (by port)
and report what they saw through optional_args["ssh_host_key_observer"].
With a known key, any other key is refused before authentication
(SshHostKeyMismatch); without one, the first key is accepted. One
implementation for paramiko (HostKeyCheck.connect) and netmiko
(HostKeyCheck.netmiko + observe). A caller or driver that ignores both keys
keeps working as before.
Version 4.19.0.
feat/service-clusters (#47) and feat/opnsense-public-contracts (#46) both
called themselves 4.17.0 and were merged one after the other: two trees
under one version. 4.17.0 stays the public contracts (83cd7db, pinned by
netOrk for #1113); the service clusters on top of it are 4.18.0.
ServiceClustersMixin.get_service_clusters() -> ServiceClustersDict
{patroni, adguardhome_sync, bunkerweb}: per mechanism the clusters the
host's configuration says it belongs to, each with its members (address,
port, name, role as the mechanism words it, this_host, config_from) and the
unit or container it was read from (source).
- Patroni: scope, name and restapi.connect_address from its YAML, then the
members and roles /cluster lists from the DCS.
- adguardhome-sync: ORIGIN_URL and REPLICA*_URL; the origin pushes to each
replica.
- BunkerWeb: a worker's API_WHITELIST_IP names the manager it admits, a
manager's BUNKERWEB_INSTANCES its workers.
Two hosts answering at one virtual IP prove nothing; this is the evidence
that they run one service (NetOrk/netork#1125). One generic command for
every Linux host, filtered on the host like the virtual IPs' one:
allowlisted fields only, URLs without user:password@, DCS endpoints stay
there. Root first, then without it; None per mechanism is "did not look".
An application followed a firewall's upgrade, edited its resolver's
forwards and read its OSPF state through the driver's private REST
helpers (NetOrk/netork#1113). Declared contracts, neutral shapes:
- UpdateMixin.get_update_progress() -> UpdateProgressDict {state, log}
for an update apply_updates only starts.
- DnsForwardMixin: get_dns_forwards(), set_dns_forward(domain, server,
port), remove_dns_forward(domain); the driver applies.
- OspfMixin.get_ospf() -> OspfDict {router_id, areas, interfaces}; the
routing daemon's format is the driver's to translate.
All declared only.
FirmwareOfferMixin.get_firmware_offer() -> FirmwareOfferDict
{current_version, latest_version, newer, download_url?}: what runs and
the newest firmware the vendor offers, from its download catalogue or
the device's own update check. Comparing vendor version strings stays
with the driver, so it says whether the offer is newer; whether that is a
warning is the application's call (NetOrk/netork#1111). Declared only.
LicenseDict gets withholds: NotRequired[List[str]]: what a device keeps back
for want of a license while the rest of it answers. A Supermicro X10 with
only SFT-OOB-LIC answers Redfish but withholds its drives and firmware
inventory until DCMS is activated. That makes it partly licensed, not
unlicensed, and the caller can tell the two apart (NetOrk/netork#1111).
Drivers report facts and the application decides what is a warning
(docs/STRUCTURE.md, NetOrk/netork#1111). Three facts drivers so far
only reported as warning codes get contracts:
- LicenseMixin.get_licenses(): licenses and subscriptions with their
status (a Proxmox subscription, a VMware edition, a BMC license a
feature needs).
- HypervisorDriver.get_hypervisor_hosts(): each host connected, in
maintenance, with configuration issues, under a central manager.
- PackageProxyMixin.get_package_proxy(): the proxy apt fetches through.
All declared only.
run_device_action was a string dispatch drivers filled with an
application's remediations, values hard-coded inside (community
"public", the caller's address, a /24 firewall rule).
napalm_device_types.fixes declares them as templates instead. A
template runs declared steps in order and stops at the first failure
(run_fix_steps); the driver implements the steps, or overrides the
whole method where its device takes the change in one call; every
value comes from the caller.
- SnmpAgentMixin.configure_snmp_agent(settings): install, configure,
firewall, start. The caller verifies with its own SNMP client.
get_management_peer() (optional) says the address the device sees the
management connection come from, for choosing allowed_sources.
- EnsurePackageMixin.ensure_package(name): install only what is missing.
- PackageProxyMixin.configure_package_proxy(url): declared.
- ContainerEngineMixin.grant_container_engine_access(user): declared.
None is mixed into a role base, so a capability check stays truthful.
Closes#33
Drivers no longer collect over SNMP (docs/STRUCTURE.md). A driver whose
device publishes its health values under OIDs of its own declares them
as SNMP_HEALTH_PROFILE: per value either scalars asked in order or a
table folded by first, average or max, optionally with a pattern for
text answers; memory as total with used or free, or a percentage, in
KiB or bytes; and how to read the interfaces. The application polls
and computes.
HealthMetricsMixin.get_health_metrics now warns that it is deprecated;
it goes in 5.0.
Closes#32
get_dhcp_ha (declared on DhcpServerMixin, optional) is what a DHCP
server's own HA configuration says: on or off, this server's peer name,
each peer's name, role as the server words it, and HA URL.
ConfigSyncMixin.get_config_sync is where a host pushes its configuration
and for which services, by the names get_services reports; sync items
without a service name stay visible under `unmapped`. A driver mixes it
in, no role promises it. Both raise when they could not read.
What the facts mean for a pair -- hot standby, which side to write to --
is the caller's (NetOrk/netork#1125).
carp_virtual_ip() and its state table are CARP protocol knowledge; under the
agreed structure (PR #37, docs/STRUCTURE.md) the base package carries only
contracts, data shapes and I/O-free reconcilers. They move to the driver that
speaks CARP (napalm-opnsense). The contract stays: VirtualIpsDict.carp and
VirtualIpDict.holds.
Nothing used the helper yet; it was released in 4.10.0 an hour ago.
netOrk #1121.
A firewall pair's CARP addresses are a third virtual IP mechanism. Unlike
vip-manager and keepalived, a CARP reading says who holds the address: a
firewall's interface list need not show it, but CARP's state names the MASTER.
- VirtualIpsDict gains carp (NotRequired), VirtualIpDict gains holds
(NotRequired; None leaves the holder to the interfaces).
- carp_virtual_ip() shapes one address bound to a VHID: MASTER holds, BACKUP
stands by, INIT and DISABLED neither run nor hold, an unknown state says
nothing. The same for OPNsense and pfSense; a driver supplies the fields.
netOrk #1121.
What ties a server to its BMC: the BMC reports the server's serial
number and system UUID, the host reads the same values from its DMI
tables. Both drivers that read DMI (napalm-linux, napalm-proxmox) had a
copy of the shell for it, and neither read the board serial.
SystemIdentityMixin.get_system_identity() reads product_uuid,
product_serial and board_serial in one framed command and returns them
with vendor placeholders empty: a Supermicro board's product serial is
0123456789 on every one of them, others read "To be filled by O.E.M."
or "Default string", and many cheap boards share the UUID
03000200-0400-0500-0006-000700080009. is_placeholder_identity() is the
same filter for a BMC driver's side. A driver supplies only
_run_system_identity_command(command, privileged=); the files are
root's.
Discovery compares drivers by score divided by the most a driver could
score, and that maximum summed every positive rule. Rules that never
match together, such as one vendor's landing page or another's, made a
driver that knows several vendors lose to one that knows none:
napalm-redfish's Supermicro landing page scored 6 of 41 and lost to a
bare "linux" in the same BMC's SNMP sysDescr, 5 of 18.
FingerprintRule gets an optional group. rules_max_weight() sums the
positive rules and counts each group once, with its strongest rule.
Existing rules have no group and count as before.
The package declares "Typing :: Typed" but shipped no PEP 561 marker, so
mypy in driver repositories treated it as untyped. Add the marker, include
it as package data, and release 4.7.1.
## feat!: settle the open review questions on roles, diffs, SSH and SNMP
- roles: a driver that sets ROLE itself replaces the role it inherits,
so a Windows driver on OSDriver is "windows", not also "linux".
- os: OSDriver's account listing is get_os_users; NAPALM's get_users has
another shape and a runtime stub, so hasattr could not tell.
- README: NAPALM's own getters must come from the driver or a mixin
listed before the first role base; its stubs win the MRO otherwise.
- firewall/DHCP diffs: an empty key, a key given twice, or one more than
one live entry holds raises ValueError instead of overwriting by guess;
live entries without a key are never matched.
- ssh_server: only host key algorithms the host has a key for count as
staying; when the keys cannot be read, the host key list is not
touched. sshd_config and its copy keep their file mode (GNU and BSD
stat), 644 only for a new file.
- SNMP: 64-bit ifXTable octets and ifHighSpeed, with the 32-bit counters
as fallback for agents without ifXTable.
- pyproject: declare typing-extensions, which models.py imports.
BREAKING CHANGE: OSDriver.get_users is now get_os_users; diffs raise
ValueError on keys that cannot name one entry; plan_ssh_hardening
removes no host key algorithm without host_keys.
## fix: tell "could not read" and "does not install" apart in update and VIP readers
- package_updates: the dnf security reader records dnf's exit status and
raises when it is missing or non-zero, as the apt reader does; a failed
dnf no longer reads as "no security advisories".
- host_status: an enabled dnf-automatic.timer counts as auto updates only
with apply_updates = yes (dnf5's automatic.conf wins over dnf4's);
dnf-automatic-install.timer always does. Unattended-upgrades counts only
while the program is installed, not from a conffile left behind.
- virtual_ips: a host with systemctl but without systemd as PID 1 reads as
"did not look" (None), not as "has none". keepalived blocks whose brace
is on the next line are read.
BREAKING CHANGE: parse_dnf_security expects DNF_SECURITY_COMMAND's
__DNF_RC marker and raises ValueError without it.
A role base for baseboard management controllers (Supermicro, Dell
iDRAC, HPE iLO, Lenovo XCC), reached over Redfish or IPMI, not SSH.
It declares, under TYPE_CHECKING only: get_power_state, set_power_state
(Redfish ResetType actions), get_hardware_inventory (CPUs, DIMMs,
drives, PSUs, NICs, and the server's serial, system UUID and NIC MACs),
get_event_log, get_firmware_inventory and get_management_protocols.
reboot_host restarts the BMC itself (Manager.Reset), never the managed
server, which goes through set_power_state. No HealthMetricsMixin and no
command channel; sensors come through NAPALM's get_environment.
Squashes three changes:
## chore: support Python 3.13 and make mypy --strict pass in CI
Add 3.13 to the CI matrix and classifiers, run mypy in CI, fix the
14 strict errors it reported, and correct the README's minimum Python.
## fix: eight bugs found in a full review, each with a regression test
- channel: run_on_transport lost output sshd sent after the exit status;
it now reads stdout and stderr to EOF before returning.
- ssh_server: a failed way back (restore or reload) raised out of
harden_ssh_server after the host was touched; it is now reported in
the result, with rolled_back False.
- host_status: a newer RPM kernel (RHEL, Fedora) was never seen, because
its release, not its arch, followed the last "-".
- package_updates: an apt line wrapped at a blank lost the blank, and
with it the package.
- kernel: an empty payload (gzip or base64 missing) read as a report.
- listening: "[fe80::1]%eth0:546", as ss prints it, kept its brackets.
- ping_sweep: a reply whose "success" was not a dict aborted the sweep.
- lag: a member of unknown speed (-1) lowered the LAG's speed.
## docs: bring the README and docstrings in line with the code
The firewall worked example showed the runtime stubs the README forbids;
the class table missed PhoneDriver and MediaDriver; ConfigLifecycleMixin
is mixed in by drivers, not by a role base; services, mac_acl and
packages named a class, a key and an exception that do not exist.
A driver gains root with sudo -S and puts the password in front of stdin.
A sudo that does not prompt -- NOPASSWD, a cached timestamp -- leaves it
there, and the cat under sudo wrote it into the drop-in as its first line.
Seen on the first real run (NetOrk/netork#1027): sshd -t rejected the file
and the rollback removed it, but the password was in a 0644 file for a
moment and in the error the caller showed.
The content now goes without privilege to a mktemp file only the login user
can read; root installs that file (mode 644, root-owned) and removes it.
The fake host in the tests prepends the password to every privileged stdin,
as a non-prompting sudo does.
Bumps to 4.5.1. NetOrk/netork#1027, NetOrk/netork#1029
ssh-audit names algorithms an SSH server should stop offering. Taking them
out is the same on every OpenSSH host, so it lives here once: a driver
mixes in OpenSshServerMixin and supplies run_command.
harden_ssh_server reads the lists sshd really uses (sshd -T), removes the
requested algorithms and writes the rest to
/etc/ssh/sshd_config.d/00-netork-ssh-hardening.conf -- or, where
sshd_config has no Include of that directory, rewrites sshd_config after
a copy. It never removes the last algorithm of a list, nor, given the
managing client's lists, the last one that client speaks. sshd -t must
accept the result and sshd -T must show it; after the reload the caller's
verify opens a new session, and if that fails the still-open one puts the
previous state back and reloads again.
Bumps to 4.5.0. NetOrk/netork#1027
A Proxmox host's root filesystem stood at 94 % with about 400 MB free for a
week; a kernel upgrade failed on it with "No space left on device" and left
dpkg halfway, and nothing had said the disk was full (NetOrk/netork#862).
HOST_STATUS_COMMAND now also runs POSIX `df -Pk / /boot /var`, without root.
get_host_status() returns `filesystems`: each filesystem once (a directory
that is not a mount of its own is the filesystem it lives on), with its
mountpoint, total, used and available bytes, or None when df said nothing.
`available` is what df says an ordinary user can still write, which on ext4
is less than total minus used. How full is too full is the caller's decision.
Version 4.4.0.
On every Ubuntu host with unattended-upgrades, get_host_status() read the
package manager as busy, so an interrupted dpkg was never reported there.
Ubuntu keeps unattended-upgrade-shutdown --wait-for-signal running all the
time; its process name is cut to 15 characters, unattended-upgr, the same
as a real run's.
BUSY_CHECK now matches dpkg, apt, apt-get and aptitude by name and
unattended-upgrade by its command line. The tests run it through sh with a
stand-in ps, and check that the command does not match itself: over an exec
channel it is the command line of the shell that runs it.
Closes#26
christianmanivong
Christian Manivong