UpdateMixin declares get_installed_advisory_cves(): per package, the CVEs
the installed security advisories close -- the distribution's word that a
backport is in although the upstream version did not move. The command
(DNF_INSTALLED_CVES_COMMAND, dnf 4, from the cache) and its parser
(parse_dnf_installed_cves) live here for every dnf driver, as the
security read's do. netOrk reads it where its knowledge base holds no
records of the host's release (NetOrk/netork#2394).
_version_key compared the text between numbers as plain strings, so
'.el' outranked '.' and 6.12.0-208.112.5.el9uek counted as newer than
the running 6.12.0-208.112.5.1.el9uek: an Oracle Linux host asked to
reboot into an older kernel, and a reboot never cleared it. The
releases now compare like rpmvercmp: separators only separate, a
number is newer than letters in its place, and '~' comes before
anything. Checked against rpm.labelCompare on Oracle Linux 9.
Closes#109
The installonly packages were dnf's upstream defaults plus dnf.conf's.
Oracle builds kernel-uek and its parts into libdnf's defaults, and the
UEK packages provide nothing of upstream's list, so every host with more
than one UEK kernel was reported as left halfway. The defaults are now
also asked of the host's libdnf through its Python binding; upstream's
list stays the floor for dnf 5 and yum.
Closes#107
A device that gets the site's time giver through DHCP option 42 is
aligned (NetOrk/netork#2380): most clients already learn it that way.
4.35.0 switched that off on every write -- chrony's DHCP sourcedir was
commented out, and systemd-timesyncd's link servers failed the verify
step -- and folded DHCP's servers into ntp_servers, so an application
could not tell configured from DHCP-provided servers.
TimeSyncDict gains dhcp_ntp_servers (not required): the servers the
device got by DHCP and uses, in its order; [] for none, absent or None
where the driver cannot tell. ntp_servers keeps only the configured ones.
PosixTimeSyncMixin reads chrony's sources from the DHCP client's
sourcedir (CHRONY_DHCP_DIRS, /run/chrony-dhcp by default) as chronyd
reads them, and systemd-timesyncd's LinkNTPServers; ntpd and OpenNTPD
keep DHCP's servers in a configuration of their own, so there it is
unknown. A chrony write now comments out only server, pool and peer
lines and leaves every sourcedir and the DHCP client's files alone. A
DHCP sourcedir that 4.35.0 commented out behind the mark is given back
on the next write, with a restart, so hosts it aligned hear DHCP again.
Version 4.36.0.
netOrk gets one list of NTP servers and a time zone per site and aligns
every device that differs (NetOrk/netork#2360). The contract comes first,
so the drivers can be built against it at the same time (#2361):
TimeSyncMixin declares get_time_sync() -> TimeSyncDict (servers in the
device's order, the zone as an IANA name, synchronised, the client) and
carries configure_time_sync(ntp_servers=None, timezone=None) as a fix
template: None leaves a value alone, the device ends up with exactly the
servers given, and a verify step reads it back. time_sync_writes says
which of the two values a driver writes; anything else, an unknown zone,
a malformed server or an empty list is refused before anything is
written.
PosixTimeSyncMixin implements it over run_command for Linux-like hosts:
one reading as the login user (timedatectl, chrony's configuration and
chronyc, timedatectl show-timesync, ntpd and OpenNTPD), the zone through
timedatectl set-timezone, chrony's servers in a file of their own with
every other source commented out, systemd-timesyncd's in a drop-in that
resets NTP= and empties FallbackNTP=. ntpd and OpenNTPD are read only.
posix_tz moves here from netOrk, so OpenWrt can write the POSIX rule and
a switch the UTC offset and summer-time rule (parse_posix_tz). It reads
the tzdata package, now a dependency.
Release 4.35.0: the drivers that implement it need it on PyPI.
A restart aborts every backup that runs on the device it restarts, and a
guest's restart breaks the backup its hypervisor is taking of it. netOrk
asks right before it restarts a host after patching (NetOrk/netork#2318),
so the reading belongs in the contract first: RunningBackupsMixin declares
get_running_backups(), opt-in like VmStopRequestMixin, so hasattr tells a
caller whether the driver can tell at all.
Release 4.34.0: the drivers that implement it need it on PyPI.
get_log_sources() reads, in the same command, whether rsyslog runs, its
forwarding rules (legacy @/@@/:omrelp: and RainerScript omfwd/omrelp,
with selector, ruleset and whether they take every message), its imfile
inputs, and whether the journal reaches it (imjournal, or imuxsock with
journald's ForwardToSyslog; failing a setting, the default the
distribution ships commented out). Returned as `forwarding`;
parse_log_forwarding and parse_rsyslog are public (NetOrk/netork#2053).
LogSourcesMixin.get_log_sources() reads, in one sh -c call as root, the
units journald holds entries for, the log files running processes hold
open (/proc/<pid>/fd: *.log, *.log.N, anything under /var/log) with size
and last write from stat -L on the descriptor, and the engine's own file
of each container's output as that container's stream. Each file names
the service or container of its first writer from /proc/<pid>/cgroup; a
file a container writes keeps the container's own path.
journald's store, deleted files and a database's write-ahead log
(LevelDB's 000004.log) are left out. Without root it reads again and says
so (attributed: false); a host whose find cannot list open files (busybox)
raises LogSourcesUnavailable rather than reading as no files.
Refs NetOrk/netork#2049
Scheduled jobs (NetOrk/netork#1933): ScheduledJobsMixin, its POSIX
implementation and napalm_device_types.cron. 4.26.0 went out meanwhile
with the sshd drop-in change, so this is 4.27.0.
What a host runs on its own schedule as one shape: cron lines, run-parts
scripts, anacron, systemd timers, periodic, a platform's own jobs, Windows
tasks. Every job says who owns its definition and what the driver can do
to it; every write names the digest the caller saw and is refused when the
host's definition changed since.
PosixScheduledJobsMixin reads everything in one framed report as root,
switches a cron line behind #napalm-disabled:, a timer by systemctl and a
run-parts script by its execute bit, runs a job now bounded by timeout, and
writes an application's job as a timer where systemd runs, else in
/etc/cron.d or the user's crontab. napalm_device_types.cron parses cron
expressions, gives their next runs and writes them as OnCalendar lines.
get_cron_jobs and CronJobDict go: nothing called them.
Version 4.26.0, so drivers can require it before the release is tagged.
Closes#86
Devices ship self-signed certificates, so a driver either fails the CA
check on every one of them or turns verification off and sends the
login to whatever answers on the address. tls_certificates pins the
certificate by its SHA-256 fingerprint instead, shaped like
ssh_host_keys: the caller passes the fingerprint to expect by port
(optional_args["tls_fingerprints"]), the driver refuses any other
certificate before a request carries credentials, and every certificate
seen goes to optional_args["tls_certificate_observer"]. Without a
fingerprint the first certificate is accepted.
CertificateCheck checks the certificate in a handshake of its own, then
pins a requests session to it (urllib3's assert_fingerprint, direct and
through a proxy), so no other host can step in between the check and
the login.
Refs NAPALM/napalm-hpe-aruba-procurve#17
netOrk runs every driver in its own containers on Python 3.12, so 3.10
and 3.11 are no longer supported or tested. requires-python, mypy and
ruff move to 3.12, along with what ruff's pyupgrade rules then rewrite.
The NAPALM driver standard landed on main (#52, 4.21.1). apt_proxy follows
it: builtin generics, strict mypy, complexity 10 at most (parse_apt_proxy,
same_proxy_url and the comment-out step split into helpers), 100 % covered.
Its TypedDicts live in apt_proxy.py, since models.py is over 1000 lines and
may not grow. Version 4.22.0 with its CHANGELOG section.
apt reads apt.conf.d in order and apt.conf last; the last value wins. The
Debian installer writes its proxy into apt.conf, Ubuntu's into
90curtin-aptproxy: reading 00proxy alone missed both, and a fix that wrote
00proxy and read it back reported success while apt fetched elsewhere.
- get_package_proxy_report(): apt-config dump's proxy, per-host overrides,
auto-detect, every line of apt's files that sets it (in reading order,
editable or inside a block), the http and https sources.
- check_package_proxy(url, expect): a fetch with apt-helper, past any proxy;
an HTTP error counts as an answer.
- configure_package_proxy(url): write 00proxy, comment out other proxy
lines, read apt's proxy back, fetch one InRelease through it.
- same_proxy_url(): scheme, host and port, a trailing slash aside.
A driver supplies _run_apt_command, _run_apt_root_command and
_write_apt_file. Version 4.22.0.
- CI moves to .gitea/workflows/ci.yml and calls the shared driver workflow
of NAPALM/driver-standard: conformance, ruff, strict mypy, tests on
3.10-3.13 and at the lowest declared versions, bandit, pip-audit, build,
and on a release tag the upload to PyPI.
- tests/quality_baseline.json records today's long and complex functions
and the one file over 1000 lines; they may shrink, not grow.
- Coverage floor 95 % (measured 95.15 %).
- CHANGELOG.md starts here; project links point at git.netork.io; the
license is the SPDX string PEP 639 wants.
- Long docstring and command lines are wrapped; recorded device output in
tests/ keeps its lines (E501 is off there).
- host_firewall: the negation token is the named constant _NEGATION. bandit
read token == "!" as a hardcoded password.
- ssh_host_keys: _known_hosts_file takes the expected key as an argument
instead of asserting it; both callers had already returned when it is None.
- ssh_server: the staged path is on the device, not a local temp file
(nosec B108).
- tests: raise ... from exc, zip(strict=True).
Mechanical: ruff format, and ruff check --fix with the rule set every NAPALM
driver uses (E, F, I, UP, B at line length 100). typing.List/Dict/Optional
become builtin generics and X | None, imports are sorted, the coding cookies
are gone. Two tests that read the source text follow the new spelling.
What ruff cannot fix comes in the next commit.
get_host_firewall() reads `nft -j list ruleset` as root and returns every
chain with its family, table, hook, priority and policy, and every rule as
matches and a verdict. Rules iptables-nft wrote show their extensions in nft's
JSON only as an opaque `xt`, so the tables iptables manages are taken from
iptables-save instead; a legacy iptables is kept beside nft's tables, since
both apply. A condition the parse does not model is an `unknown` match with
its text, never left out. Without root neither tool can be read, and
HostFirewallUnavailable is raised rather than a host without rules.
A driver supplies only _run_host_firewall_command(command, privileged=).
Version 4.21.0.
get_service_packages() returns, per systemd service, the installed packages
it is made of: the one shipping its unit file and the one shipping the
binary its ExecStart runs -- not the libraries that binary links against.
One read-only command: systemctl show for every loaded service unit, then
one dpkg -S for all paths (each also across the /usr merge), or rpm -qf per
path. The report is framed; one cut short raises. A driver supplies
_run_service_packages_command, as for KernelFactsMixin.
Version 4.20.0.
Drivers get the host keys to expect in optional_args["ssh_host_keys"] (by port)
and report what they saw through optional_args["ssh_host_key_observer"].
With a known key, any other key is refused before authentication
(SshHostKeyMismatch); without one, the first key is accepted. One
implementation for paramiko (HostKeyCheck.connect) and netmiko
(HostKeyCheck.netmiko + observe). A caller or driver that ignores both keys
keeps working as before.
Version 4.19.0.
feat/service-clusters (#47) and feat/opnsense-public-contracts (#46) both
called themselves 4.17.0 and were merged one after the other: two trees
under one version. 4.17.0 stays the public contracts (83cd7db, pinned by
netOrk for #1113); the service clusters on top of it are 4.18.0.
An application followed a firewall's upgrade, edited its resolver's
forwards and read its OSPF state through the driver's private REST
helpers (NetOrk/netork#1113). Declared contracts, neutral shapes:
- UpdateMixin.get_update_progress() -> UpdateProgressDict {state, log}
for an update apply_updates only starts.
- DnsForwardMixin: get_dns_forwards(), set_dns_forward(domain, server,
port), remove_dns_forward(domain); the driver applies.
- OspfMixin.get_ospf() -> OspfDict {router_id, areas, interfaces}; the
routing daemon's format is the driver's to translate.
All declared only.
FirmwareOfferMixin.get_firmware_offer() -> FirmwareOfferDict
{current_version, latest_version, newer, download_url?}: what runs and
the newest firmware the vendor offers, from its download catalogue or
the device's own update check. Comparing vendor version strings stays
with the driver, so it says whether the offer is newer; whether that is a
warning is the application's call (NetOrk/netork#1111). Declared only.
LicenseDict gets withholds: NotRequired[List[str]]: what a device keeps back
for want of a license while the rest of it answers. A Supermicro X10 with
only SFT-OOB-LIC answers Redfish but withholds its drives and firmware
inventory until DCMS is activated. That makes it partly licensed, not
unlicensed, and the caller can tell the two apart (NetOrk/netork#1111).
Drivers report facts and the application decides what is a warning
(docs/STRUCTURE.md, NetOrk/netork#1111). Three facts drivers so far
only reported as warning codes get contracts:
- LicenseMixin.get_licenses(): licenses and subscriptions with their
status (a Proxmox subscription, a VMware edition, a BMC license a
feature needs).
- HypervisorDriver.get_hypervisor_hosts(): each host connected, in
maintenance, with configuration issues, under a central manager.
- PackageProxyMixin.get_package_proxy(): the proxy apt fetches through.
All declared only.
run_device_action was a string dispatch drivers filled with an
application's remediations, values hard-coded inside (community
"public", the caller's address, a /24 firewall rule).
napalm_device_types.fixes declares them as templates instead. A
template runs declared steps in order and stops at the first failure
(run_fix_steps); the driver implements the steps, or overrides the
whole method where its device takes the change in one call; every
value comes from the caller.
- SnmpAgentMixin.configure_snmp_agent(settings): install, configure,
firewall, start. The caller verifies with its own SNMP client.
get_management_peer() (optional) says the address the device sees the
management connection come from, for choosing allowed_sources.
- EnsurePackageMixin.ensure_package(name): install only what is missing.
- PackageProxyMixin.configure_package_proxy(url): declared.
- ContainerEngineMixin.grant_container_engine_access(user): declared.
None is mixed into a role base, so a capability check stays truthful.
Closes#33
Drivers no longer collect over SNMP (docs/STRUCTURE.md). A driver whose
device publishes its health values under OIDs of its own declares them
as SNMP_HEALTH_PROFILE: per value either scalars asked in order or a
table folded by first, average or max, optionally with a pattern for
text answers; memory as total with used or free, or a percentage, in
KiB or bytes; and how to read the interfaces. The application polls
and computes.
HealthMetricsMixin.get_health_metrics now warns that it is deprecated;
it goes in 5.0.
Closes#32
get_dhcp_ha (declared on DhcpServerMixin, optional) is what a DHCP
server's own HA configuration says: on or off, this server's peer name,
each peer's name, role as the server words it, and HA URL.
ConfigSyncMixin.get_config_sync is where a host pushes its configuration
and for which services, by the names get_services reports; sync items
without a service name stay visible under `unmapped`. A driver mixes it
in, no role promises it. Both raise when they could not read.
What the facts mean for a pair -- hot standby, which side to write to --
is the caller's (NetOrk/netork#1125).
carp_virtual_ip() and its state table are CARP protocol knowledge; under the
agreed structure (PR #37, docs/STRUCTURE.md) the base package carries only
contracts, data shapes and I/O-free reconcilers. They move to the driver that
speaks CARP (napalm-opnsense). The contract stays: VirtualIpsDict.carp and
VirtualIpDict.holds.
Nothing used the helper yet; it was released in 4.10.0 an hour ago.
netOrk #1121.
A firewall pair's CARP addresses are a third virtual IP mechanism. Unlike
vip-manager and keepalived, a CARP reading says who holds the address: a
firewall's interface list need not show it, but CARP's state names the MASTER.
- VirtualIpsDict gains carp (NotRequired), VirtualIpDict gains holds
(NotRequired; None leaves the holder to the interfaces).
- carp_virtual_ip() shapes one address bound to a VHID: MASTER holds, BACKUP
stands by, INIT and DISABLED neither run nor hold, an unknown state says
nothing. The same for OPNsense and pfSense; a driver supplies the fields.
netOrk #1121.
What ties a server to its BMC: the BMC reports the server's serial
number and system UUID, the host reads the same values from its DMI
tables. Both drivers that read DMI (napalm-linux, napalm-proxmox) had a
copy of the shell for it, and neither read the board serial.
SystemIdentityMixin.get_system_identity() reads product_uuid,
product_serial and board_serial in one framed command and returns them
with vendor placeholders empty: a Supermicro board's product serial is
0123456789 on every one of them, others read "To be filled by O.E.M."
or "Default string", and many cheap boards share the UUID
03000200-0400-0500-0006-000700080009. is_placeholder_identity() is the
same filter for a BMC driver's side. A driver supplies only
_run_system_identity_command(command, privileged=); the files are
root's.
Discovery compares drivers by score divided by the most a driver could
score, and that maximum summed every positive rule. Rules that never
match together, such as one vendor's landing page or another's, made a
driver that knows several vendors lose to one that knows none:
napalm-redfish's Supermicro landing page scored 6 of 41 and lost to a
bare "linux" in the same BMC's SNMP sysDescr, 5 of 18.
FingerprintRule gets an optional group. rules_max_weight() sums the
positive rules and counts each group once, with its strongest rule.
Existing rules have no group and count as before.
The package declares "Typing :: Typed" but shipped no PEP 561 marker, so
mypy in driver repositories treated it as untyped. Add the marker, include
it as package data, and release 4.7.1.
## feat!: settle the open review questions on roles, diffs, SSH and SNMP
- roles: a driver that sets ROLE itself replaces the role it inherits,
so a Windows driver on OSDriver is "windows", not also "linux".
- os: OSDriver's account listing is get_os_users; NAPALM's get_users has
another shape and a runtime stub, so hasattr could not tell.
- README: NAPALM's own getters must come from the driver or a mixin
listed before the first role base; its stubs win the MRO otherwise.
- firewall/DHCP diffs: an empty key, a key given twice, or one more than
one live entry holds raises ValueError instead of overwriting by guess;
live entries without a key are never matched.
- ssh_server: only host key algorithms the host has a key for count as
staying; when the keys cannot be read, the host key list is not
touched. sshd_config and its copy keep their file mode (GNU and BSD
stat), 644 only for a new file.
- SNMP: 64-bit ifXTable octets and ifHighSpeed, with the 32-bit counters
as fallback for agents without ifXTable.
- pyproject: declare typing-extensions, which models.py imports.
BREAKING CHANGE: OSDriver.get_users is now get_os_users; diffs raise
ValueError on keys that cannot name one entry; plan_ssh_hardening
removes no host key algorithm without host_keys.
## fix: tell "could not read" and "does not install" apart in update and VIP readers
- package_updates: the dnf security reader records dnf's exit status and
raises when it is missing or non-zero, as the apt reader does; a failed
dnf no longer reads as "no security advisories".
- host_status: an enabled dnf-automatic.timer counts as auto updates only
with apply_updates = yes (dnf5's automatic.conf wins over dnf4's);
dnf-automatic-install.timer always does. Unattended-upgrades counts only
while the program is installed, not from a conffile left behind.
- virtual_ips: a host with systemctl but without systemd as PID 1 reads as
"did not look" (None), not as "has none". keepalived blocks whose brace
is on the next line are read.
BREAKING CHANGE: parse_dnf_security expects DNF_SECURITY_COMMAND's
__DNF_RC marker and raises ValueError without it.
A role base for baseboard management controllers (Supermicro, Dell
iDRAC, HPE iLO, Lenovo XCC), reached over Redfish or IPMI, not SSH.
It declares, under TYPE_CHECKING only: get_power_state, set_power_state
(Redfish ResetType actions), get_hardware_inventory (CPUs, DIMMs,
drives, PSUs, NICs, and the server's serial, system UUID and NIC MACs),
get_event_log, get_firmware_inventory and get_management_protocols.
reboot_host restarts the BMC itself (Manager.Reset), never the managed
server, which goes through set_power_state. No HealthMetricsMixin and no
command channel; sensors come through NAPALM's get_environment.
Squashes three changes:
## chore: support Python 3.13 and make mypy --strict pass in CI
Add 3.13 to the CI matrix and classifiers, run mypy in CI, fix the
14 strict errors it reported, and correct the README's minimum Python.
## fix: eight bugs found in a full review, each with a regression test
- channel: run_on_transport lost output sshd sent after the exit status;
it now reads stdout and stderr to EOF before returning.
- ssh_server: a failed way back (restore or reload) raised out of
harden_ssh_server after the host was touched; it is now reported in
the result, with rolled_back False.
- host_status: a newer RPM kernel (RHEL, Fedora) was never seen, because
its release, not its arch, followed the last "-".
- package_updates: an apt line wrapped at a blank lost the blank, and
with it the package.
- kernel: an empty payload (gzip or base64 missing) read as a report.
- listening: "[fe80::1]%eth0:546", as ss prints it, kept its brackets.
- ping_sweep: a reply whose "success" was not a dict aborted the sweep.
- lag: a member of unknown speed (-1) lowered the LAG's speed.
## docs: bring the README and docstrings in line with the code
The firewall worked example showed the runtime stubs the README forbids;
the class table missed PhoneDriver and MediaDriver; ConfigLifecycleMixin
is mixed in by drivers, not by a role base; services, mac_acl and
packages named a class, a key and an exception that do not exist.