Compare commits

...
22 Commits
Author SHA1 Message Date
christianmanivong 2080ccbfd7 Merge pull request 'chore(release): 4.38.0' (#118) from chore/release-4.38.0 into main
driver / types (push) Successful in 56s
driver / standard (push) Successful in 16s
driver / lint (push) Successful in 13s
driver / test (3.12) (push) Successful in 1m10s
driver / test (3.13) (push) Successful in 1m1s
driver / lowest (push) Successful in 37s
driver / build (push) Successful in 28s
driver / security (push) Successful in 53s
driver / publish (push) Successful in 27s
CI / driver (push) Successful in 3m1s
2026-11-13 21:56:00 +00:00
christianmanivong ce1fe18405 chore(release): 4.38.0
driver / standard (pull_request) Successful in 16s
driver / lint (pull_request) Successful in 14s
driver / types (pull_request) Successful in 1m2s
driver / test (3.12) (pull_request) Successful in 1m18s
driver / test (3.13) (pull_request) Successful in 1m9s
driver / lowest (pull_request) Successful in 46s
driver / build (pull_request) Successful in 31s
driver / security (pull_request) Successful in 1m11s
CI / driver (pull_request) Successful in 2m59s
driver / publish (pull_request) Skipped
2026-11-13 22:39:54 +01:00
christianmanivong e43926f1d5 Merge pull request 'feat(config-revisions): declare a device's own configuration history' (#117) from feat/config-revisions into main
driver / lint (push) Successful in 15s
driver / types (push) Successful in 1m11s
driver / test (3.13) (push) Successful in 1m6s
driver / test (3.12) (push) Successful in 1m27s
driver / lowest (push) Successful in 37s
driver / build (push) Successful in 29s
driver / security (push) Successful in 1m5s
driver / publish (push) Skipped
driver / standard (push) Successful in 16s
CI / driver (push) Successful in 2m58s
2026-11-13 21:39:38 +00:00
christianmanivong 43e0210ff6 feat(config-revisions): declare a device's own configuration history
driver / lint (pull_request) Successful in 13s
driver / standard (pull_request) Successful in 16s
driver / types (pull_request) Successful in 1m3s
driver / test (3.13) (pull_request) Successful in 1m12s
driver / test (3.12) (pull_request) Successful in 1m18s
driver / lowest (pull_request) Successful in 47s
driver / build (pull_request) Successful in 34s
driver / security (pull_request) Successful in 3m34s
CI / driver (pull_request) Successful in 5m2s
driver / publish (pull_request) Skipped
Some devices keep a copy of their configuration for every save and can
go back to one. An application restores a configuration there by
reverting to the revision that holds it; on OPNsense that is the only
way, since its API takes no uploaded config.xml (NetOrk/netork#2511).

ConfigRevisionsMixin declares get_config_revisions() (newest first, as
ConfigRevisionDict) and get_config_revision(revision_id).
revert_config_revision(revision_id) is a template over the driver's
_revert_to_config_revision step: it refuses an id that is not in the
history (UnknownConfigRevision), reads the revision before the revert
can rotate it out, and afterwards checks that the newest revision holds
its configuration (ConfigRevertFailed otherwise), since a device may
report a revert it did not make. config_revision_content(text) is the
hook where a driver leaves out what its device stamps into every save,
so revisions with the same configuration compare equal.

Refs NetOrk/netork#2511
2026-11-13 22:28:30 +01:00
christianmanivong 241acef6c1 Merge pull request 'feat(accounts): add_service_account takes the keys the new one replaces' (#115) from fix/service-account-replaces into main
driver / standard (push) Successful in 16s
driver / lint (push) Successful in 13s
driver / types (push) Successful in 55s
driver / test (3.13) (push) Successful in 59s
driver / test (3.12) (push) Successful in 1m10s
driver / lowest (push) Successful in 35s
driver / build (push) Successful in 28s
driver / security (push) Successful in 57s
driver / publish (push) Skipped
CI / driver (push) Successful in 2m35s
2026-11-13 08:55:08 +00:00
christianmanivong 4c0ba5b2c0 refactor(accounts): the account step of add_service_account in a helper
driver / standard (pull_request) Successful in 16s
driver / lint (pull_request) Successful in 14s
driver / types (pull_request) Successful in 52s
driver / test (3.12) (pull_request) Successful in 1m8s
driver / lowest (pull_request) Successful in 39s
driver / test (3.13) (pull_request) Successful in 1m7s
driver / build (pull_request) Successful in 26s
driver / security (pull_request) Successful in 54s
CI / driver (pull_request) Successful in 2m32s
driver / publish (pull_request) Skipped
replaces= made add_service_account 55 lines long, over the quality
ratchet's 50. Its account step -- none to take when the driver logs in
as the account already, else the create or reopen -- moves to
_accounts_service_account_step. Behaviour is unchanged; the tests of
add_service_account cover both branches.
2026-11-13 09:00:43 +01:00
christianmanivong e9da399568 Merge remote-tracking branch 'origin/main' into fix/service-account-replaces 2026-11-13 08:59:48 +01:00
christianmanivong 8673d1eda1 Merge pull request 'feat(log-forwarding): each forwarding rule names its template' (#116) from feat/forward-rule-template into main
driver / lint (push) Successful in 16s
driver / standard (push) Successful in 17s
driver / types (push) Successful in 1m8s
driver / test (3.13) (push) Successful in 1m13s
driver / test (3.12) (push) Successful in 1m22s
driver / lowest (push) Successful in 43s
driver / build (push) Successful in 31s
driver / security (push) Successful in 1m3s
driver / publish (push) Skipped
CI / driver (push) Successful in 2m33s
2026-11-12 22:42:45 +00:00
christianmanivong 46d7125aa6 refactor(log-forwarding): a legacy rule's template is read on its own
driver / standard (pull_request) Successful in 17s
driver / lint (pull_request) Successful in 14s
driver / types (pull_request) Successful in 58s
driver / test (3.12) (pull_request) Successful in 1m21s
driver / test (3.13) (pull_request) Successful in 1m8s
driver / lowest (pull_request) Successful in 40s
driver / build (pull_request) Successful in 29s
driver / security (pull_request) Successful in 1m4s
CI / driver (pull_request) Successful in 2m57s
driver / publish (pull_request) Skipped
The quality ratchet holds _Reader.legacy at complexity 10.
2026-11-12 23:38:28 +01:00
christianmanivong 8e43fe2ed9 feat(log-forwarding): each forwarding rule names its template
driver / lint (pull_request) Successful in 16s
driver / standard (pull_request) Failing after 17s
driver / types (pull_request) Successful in 1m9s
driver / test (3.13) (pull_request) Successful in 1m15s
driver / test (3.12) (pull_request) Successful in 1m30s
driver / lowest (pull_request) Successful in 40s
driver / build (pull_request) Successful in 32s
driver / security (pull_request) Successful in 1m5s
CI / driver (pull_request) Failing after 2m38s
driver / publish (pull_request) Skipped
A rule's template decides the timestamp it sends: rsyslog's default, RFC 3164,
carries the host's local time without a zone, which a collector such as Graylog
files shifted by the host's UTC offset (NetOrk/netork#493). The rule now says
which template it uses: its own (`@host;Name`, `action(... template="...")`),
else the default set before it (`$ActionForwardDefaultTemplate`, or
`module(load="builtin:omfwd" template="...")` for omfwd actions), else None.
2026-11-12 23:36:38 +01:00
christianmanivong 679584766f feat(accounts): add_service_account takes the keys the new one replaces
driver / lint (pull_request) Successful in 15s
driver / standard (pull_request) Failing after 18s
driver / types (pull_request) Successful in 1m1s
driver / test (3.13) (pull_request) Successful in 1m11s
driver / test (3.12) (pull_request) Successful in 1m19s
driver / lowest (pull_request) Successful in 46s
driver / build (pull_request) Successful in 38s
driver / security (pull_request) Successful in 1m16s
CI / driver (pull_request) Failing after 2m45s
driver / publish (pull_request) Skipped
netOrk moves a host from its installation-wide key to its site's
(NetOrk/netork#2462): the old key has to go from its own account, which
remove_local_account_keys refuses (the login's own account, a system
account). With replaces, the old keys go once the new key is in -- never
the new key itself, and not while it could not be added.
2026-11-12 21:55:13 +01:00
christianmanivong 1b04818982 Merge pull request 'fix(dhcp): a refused subnet no longer stops the rest of the set' (#114) from fix/dhcp-subnet-apply-tolerance into main
driver / standard (push) Successful in 14s
driver / lint (push) Successful in 13s
driver / types (push) Successful in 56s
driver / test (3.12) (push) Successful in 1m12s
driver / lowest (push) Successful in 43s
driver / test (3.13) (push) Successful in 1m6s
driver / build (push) Successful in 29s
driver / security (push) Successful in 1m5s
driver / publish (push) Skipped
CI / driver (push) Successful in 2m44s
2026-11-12 19:06:13 +00:00
christianmanivong 65991d0930 fix(dhcp): a refused subnet no longer stops the rest of the set
driver / standard (pull_request) Successful in 16s
driver / lint (pull_request) Successful in 16s
driver / types (pull_request) Successful in 1m19s
driver / test (3.12) (pull_request) Successful in 1m32s
driver / test (3.13) (pull_request) Successful in 1m16s
driver / lowest (pull_request) Successful in 41s
driver / build (pull_request) Successful in 35s
driver / security (pull_request) Successful in 1m13s
CI / driver (pull_request) Successful in 3m15s
driver / publish (pull_request) Skipped
apply_dhcp_subnetset called apply_dhcp_subnet for each add and update
and committed only at the end, so one refused write raised before the
commit: the subnets written ahead of it stayed saved but inactive, and
as the device reports them as configured, no later diff committed them.

It now behaves like apply_dhcp_reservationset since 4.35.1: a refused
subnet gets a [failed] line, the others are written, any non-empty diff
is committed, then DhcpApplyError is raised. DhcpApplyError takes an
optional kind ("reservation" by default) for its message.

Closes #113
2026-11-12 19:19:03 +01:00
christianmanivong 9dfb5596bc Merge pull request 'fix(dhcp): release an address before another reservation takes it' (#112) from fix/dhcp-reservation-apply-order into main
driver / standard (push) Successful in 17s
driver / lint (push) Successful in 14s
driver / types (push) Successful in 1m1s
driver / test (3.12) (push) Successful in 1m17s
driver / test (3.13) (push) Successful in 1m9s
driver / lowest (push) Successful in 40s
driver / build (push) Successful in 30s
driver / security (push) Successful in 1m9s
driver / publish (push) Skipped
CI / driver (push) Successful in 2m59s
2026-11-12 18:14:11 +00:00
christianmanivong b4cdcfd347 fix(dhcp): release an address before another reservation takes it
driver / standard (pull_request) Successful in 17s
driver / lint (pull_request) Successful in 14s
driver / types (pull_request) Successful in 1m3s
driver / test (3.12) (pull_request) Successful in 1m25s
driver / test (3.13) (pull_request) Successful in 1m13s
driver / lowest (pull_request) Successful in 45s
driver / build (pull_request) Successful in 32s
driver / security (pull_request) Successful in 1m10s
CI / driver (pull_request) Successful in 3m6s
driver / publish (pull_request) Skipped
apply_dhcp_reservationset wrote adds first, then updates in the order of
the desired set. Kea keeps addresses unique at each write, so when an
address moved from one reservation to another, the reservation taking it
was refused as a duplicate whenever it came first.

The changes are now ordered (_dhcp_order): an update that moves a
reservation off its address goes before the change that takes that
address, along a chain from its free end, and adds come after every
update. A swap or rotation of addresses has no such order; its changes
are still tried, so a server that checks only at the commit takes them,
and a refused one's [failed] line says it was a swap and with which
reservations. Nothing is moved to a temporary address: the contract
knows no free one, and a failed second write would leave it there.

The diff is computed against one read of the reservations, shared with
diff_dhcp_reservations. Refused rows still do not stop the others, the
rest is committed and DhcpApplyError follows, as since 4.35.1.

Closes #106
2026-11-12 19:07:31 +01:00
christianmanivong 5258a43dc9 Merge pull request 'feat(updates): the CVEs a host's installed advisories close (4.37.0)' (#111) from feature/installed-advisories into main
driver / types (push) Successful in 59s
driver / standard (push) Successful in 15s
driver / lint (push) Successful in 13s
driver / test (3.12) (push) Successful in 1m12s
driver / test (3.13) (push) Successful in 1m16s
driver / lowest (push) Successful in 39s
driver / build (push) Successful in 28s
driver / security (push) Successful in 1m6s
driver / publish (push) Successful in 28s
CI / driver (push) Successful in 6m6s
2026-11-12 11:12:53 +00:00
christianmanivong f3cb3601a8 feat(updates): the CVEs a host's installed advisories close (4.37.0)
driver / standard (pull_request) Successful in 16s
driver / lint (pull_request) Successful in 15s
driver / types (pull_request) Successful in 1m2s
driver / test (3.13) (pull_request) Successful in 1m11s
driver / test (3.12) (pull_request) Successful in 1m24s
driver / lowest (pull_request) Successful in 40s
driver / build (pull_request) Successful in 28s
driver / security (pull_request) Successful in 1m0s
CI / driver (pull_request) Successful in 2m29s
driver / publish (pull_request) Skipped
UpdateMixin declares get_installed_advisory_cves(): per package, the CVEs
the installed security advisories close -- the distribution's word that a
backport is in although the upstream version did not move. The command
(DNF_INSTALLED_CVES_COMMAND, dnf 4, from the cache) and its parser
(parse_dnf_installed_cves) live here for every dnf driver, as the
security read's do. netOrk reads it where its knowledge base holds no
records of the host's release (NetOrk/netork#2394).
2026-11-12 12:09:37 +01:00
christianmanivong a8fedf299a Merge pull request 'fix(host-status): kernel releases compare the way rpm does (4.36.2)' (#110) from fix/rpm-kernel-version-order into main
driver / types (push) Successful in 1m4s
driver / standard (push) Successful in 15s
driver / lint (push) Successful in 14s
driver / test (3.12) (push) Successful in 1m20s
driver / test (3.13) (push) Successful in 1m13s
driver / lowest (push) Successful in 46s
driver / build (push) Successful in 40s
driver / security (push) Successful in 1m16s
driver / publish (push) Successful in 30s
CI / driver (push) Successful in 3m36s
2026-11-12 10:08:48 +00:00
christianmanivong c736fb30d0 refactor(host-status): compare one version segment in a helper of its own
driver / standard (pull_request) Successful in 15s
driver / lint (pull_request) Successful in 14s
driver / types (pull_request) Successful in 56s
driver / test (3.12) (pull_request) Successful in 1m8s
driver / test (3.13) (pull_request) Successful in 1m8s
driver / lowest (pull_request) Successful in 40s
driver / build (pull_request) Successful in 26s
driver / security (pull_request) Successful in 57s
CI / driver (pull_request) Successful in 2m30s
driver / publish (pull_request) Skipped
The quality ratchet refused _version_cmp at complexity 12. Comparing one
segment moves to _segment_cmp, which also takes 01 and 1 as equal, as
rpm does.
2026-11-12 11:05:49 +01:00
christianmanivong cf555d006d fix(host-status): kernel releases compare the way rpm does (4.36.2)
driver / lint (pull_request) Successful in 13s
driver / standard (pull_request) Failing after 15s
driver / types (pull_request) Successful in 1m5s
driver / test (3.13) (pull_request) Successful in 1m14s
driver / test (3.12) (pull_request) Successful in 1m19s
driver / lowest (pull_request) Successful in 42s
driver / build (pull_request) Successful in 28s
driver / security (pull_request) Successful in 1m3s
CI / driver (pull_request) Failing after 2m32s
driver / publish (pull_request) Skipped
_version_key compared the text between numbers as plain strings, so
'.el' outranked '.' and 6.12.0-208.112.5.el9uek counted as newer than
the running 6.12.0-208.112.5.1.el9uek: an Oracle Linux host asked to
reboot into an older kernel, and a reboot never cleared it. The
releases now compare like rpmvercmp: separators only separate, a
number is newer than letters in its place, and '~' comes before
anything. Checked against rpm.labelCompare on Oracle Linux 9.

Closes #109
2026-11-12 11:03:46 +01:00
christianmanivong 4e79cd1fc8 Merge pull request 'fix(host-status): an Oracle Linux host's UEK kernels are not an interrupted dnf (4.36.1)' (#108) from fix/dnf-installonly-defaults into main
driver / types (push) Successful in 1m0s
driver / standard (push) Successful in 16s
driver / lint (push) Successful in 13s
driver / test (3.12) (push) Successful in 1m17s
driver / test (3.13) (push) Successful in 1m12s
driver / lowest (push) Successful in 41s
driver / build (push) Successful in 30s
driver / security (push) Successful in 1m3s
driver / publish (push) Successful in 30s
CI / driver (push) Successful in 3m21s
2026-11-12 09:37:34 +00:00
christianmanivong 304f404567 fix(host-status): an Oracle Linux host's UEK kernels are not an interrupted dnf (4.36.1)
driver / lint (pull_request) Successful in 16s
driver / standard (pull_request) Successful in 18s
driver / types (pull_request) Successful in 59s
driver / test (3.13) (pull_request) Successful in 1m12s
driver / test (3.12) (pull_request) Successful in 1m17s
driver / lowest (pull_request) Successful in 41s
driver / build (pull_request) Successful in 30s
driver / security (pull_request) Successful in 56s
CI / driver (pull_request) Successful in 2m26s
driver / publish (pull_request) Skipped
The installonly packages were dnf's upstream defaults plus dnf.conf's.
Oracle builds kernel-uek and its parts into libdnf's defaults, and the
UEK packages provide nothing of upstream's list, so every host with more
than one UEK kernel was reported as left halfway. The defaults are now
also asked of the host's libdnf through its Python binding; upstream's
list stays the floor for dnf 5 and yum.

Closes #107
2026-11-12 10:30:48 +01:00
22 changed files with 1292 additions and 94 deletions

No files matched your search

+67
View File
@@ -7,6 +7,73 @@ recorded here; their history is the git log.
## [Unreleased]
## [4.38.0] - 2026-11-13
### Added
- `ConfigRevisionsMixin`: a device's own configuration history (NetOrk/netork#2511).
`get_config_revisions()` (newest first, the first in place now, as
`ConfigRevisionDict`: id, UTC time, description, user, size) and
`get_config_revision(revision_id)` are declared. `revert_config_revision(revision_id)`
is a template over the driver's `_revert_to_config_revision`: it refuses an id that is
not in the history (`UnknownConfigRevision`), and after the revert checks that the
newest revision holds the revision's configuration, else `ConfigRevertFailed`; it
returns that revision. `config_revision_content(text)` leaves out what a device stamps
into every save, so an application can compare revisions with a configuration it has.
- `add_service_account(..., replaces=[...])`: netOrk's keys the new one takes the place of
go from its account once the new key is in it (step `old keys`), never the new key
itself; while the new key could not be added they stay, so netOrk keeps its way in.
netOrk calls it through a login with the new key, after trying that login, to move a
host from its installation-wide key to its site's (NetOrk/netork#2462).
- Each forwarding rule of `LogForwardingDict` names its `template`: the one the rule
gives (`@host;RSYSLOG_SyslogProtocol23Format`, `action(… template="…")`), else the
default set before it (`$ActionForwardDefaultTemplate`, or
`module(load="builtin:omfwd" template="…")` for omfwd actions), else None -- rsyslog's
own default, RFC 3164, which sends the host's local time without a time zone. A
reader can tell a rule whose timestamps a collector files shifted (NetOrk/netork#493).
### Fixed
- `apply_dhcp_reservationset` writes its changes so an address is released before it is
taken: an update that moves a reservation off its address goes before the change that
takes that address (along a chain, from its free end), and adds come after every
update. A server that keeps addresses unique at each write, such as Kea, no longer
refuses the second reservation as a duplicate when an address moves from one to
another. A swap -- reservations trading addresses, or a longer rotation -- has no such
order: its changes are still tried, and where the server refuses them the `[failed]`
line says it was a swap and with which reservations. Everything else is still written
and committed, and `DhcpApplyError` follows (#106).
- `apply_dhcp_subnetset` no longer stops at the first subnet the device refuses. As
`apply_dhcp_reservationset` has since 4.35.1, a refused subnet gets a `[failed]` line,
the others are still written, any non-empty diff is committed -- even one whose every
write was refused -- and `DhcpApplyError` follows. Before, the subnets written ahead of
the refused one stayed saved but inactive, and since the device reports them as
configured, no later apply committed them. `DhcpApplyError` takes an optional `kind`
(`"reservation"` by default, `"subnet"` here) that its message names (#113).
## [4.37.0] - 2026-11-12
### Added
- `UpdateMixin.get_installed_advisory_cves()` (declared): the CVEs a host's installed
security advisories close, per package. With `DNF_INSTALLED_CVES_COMMAND` (dnf 4,
read from the cache) and `parse_dnf_installed_cves()`, the parser every dnf driver
shares. netOrk reads it where its knowledge base holds no records of the host's
release, as the distribution's word that a backport is in (netOrk #2394).
## [4.36.2] - 2026-11-12
### Fixed
- `kernel_reboot_pending` compares kernel releases the way rpm does (`rpmvercmp`). A
number is newer than letters in the same place, so an Oracle Linux host running
`6.12.0-208.112.5.1.el9uek` no longer asks to reboot into the older
`6.12.0-208.112.5.el9uek`. A `~` (release candidate) comes before its release (#109).
## [4.36.1] - 2026-11-12
### Fixed
- The host status no longer reports an Oracle Linux host's UEK kernels as an interrupted
dnf transaction. The installonly packages are now also the ones the host's libdnf was
built with, read from its Python binding: Oracle adds `kernel-uek` and its parts there,
and their packages provide nothing of upstream's defaults (#107).
## [4.36.0] - 2026-11-11
### Added
+21 -1
View File
@@ -374,6 +374,23 @@ that 4.35.0 commented out comes back on the next write -- and systemd-timesyncd'
`LinkNTPServers` stay. ntpd and OpenNTPD are read, not written; what DHCP gives them is
unknown.
`ConfigRevisionsMixin` (`get_config_revisions`, `get_config_revision`,
`revert_config_revision`) is a device's own configuration history: a copy of the
configuration for every save, which the device can go back to. An application restores a
configuration there by reverting to the revision that holds it -- on OPNsense the only way,
since its API takes no uploaded config.xml (NetOrk/netork#2511). Listing (newest first, the
first in place now: id, UTC time, description, user, size as `ConfigRevisionDict`) and
reading one revision are declared only. `revert_config_revision(revision_id)` is a
template: it refuses an id that is not in the history (`UnknownConfigRevision`, a
`ValueError`) before anything changes, reads the revision -- before the revert, which may
rotate it out -- lets the driver revert (`_revert_to_config_revision`), and checks that the
newest revision afterwards holds the revision's configuration (`ConfigRevertFailed`
otherwise: a device may say it reverted when it did not). It returns that newest revision.
`config_revision_content(text)` is what of a revision is its configuration; a driver whose
device stamps every save (who, when) strips the stamp there, and an application that looks
for the revision holding a configuration compares through it too. A driver that can list
but not revert sets `revert_config_revision = None`.
### Access channels: why there is no container model here
`CommandChannelMixin` declares a driver's public channel to its host:
@@ -493,7 +510,10 @@ The same split applies to `DhcpServerMixin`: `get_dhcp_reservations`/
`apply_dhcp_reservation`/`commit_dhcp_reservations` are abstract (Kea REST on
OPNsense, dnsmasq/odhcpd UCI on OpenWrt), while `diff_dhcp_reservations` and
`apply_dhcp_reservationset` are concrete — matching by normalised MAC and the
apply-then-commit orchestration are identical for every DHCP server.
apply-then-commit orchestration are identical for every DHCP server. The apply writes
its changes so an address is released before another reservation takes it (an update
that frees an address first, adds last); a swap of addresses has no such order and is
reported as one where the server refuses it.
A new driver (FortiGate, pfSense, …) gets `diff_firewall_rules`/
`apply_firewall_ruleset` for free the moment it implements the three abstract
+1 -1
View File
@@ -39,7 +39,7 @@ application SNMP polling, ping sweeps, discovery scoring, guest
| **Capability** | Declares only. | `packages`, `services`, `updates`, `nat_vpn`, `mac_acl`, `config_lifecycle`, `interface_filter`, `host_reboot` |
| **Template** | A concrete method that runs a fixed sequence of abstract steps. A driver implements the steps, or overrides the whole method when its device does it in one go. All values come from the caller. | the fix routines below |
| **Contract types** | Interfaces and data shapes other packages implement or fill. | `CommandChannelMixin`, `CommandResult`, `ByteStream`; the declaration of `ContainerEngineMixin`; the reader mixins' public method declarations; the SNMP profile type; `models` |
| **Reconciler** | Concrete and vendor-neutral, without I/O of its own. | `firewall_rules` and `dhcp` diff/apply, `_matching`, `lag` |
| **Reconciler** | Concrete and vendor-neutral, without I/O of its own. | `firewall_rules` and `dhcp` diff/apply, `_matching`, `_dhcp_order`, `lag` |
| **Introspection** | Concrete, without I/O. | `roles`; `base` (`FingerprintRule`, `PortSpec`, the fingerprint attributes) |
| **Driver facts** | Class attributes that describe the device or how it is reached, not what an application does with it. | `USES_SSH`, `FULL_UPGRADE`, `platform`, `SNMP_HEALTH_PROFILE` |
+15
View File
@@ -38,6 +38,7 @@ instead of being restated on every role that happens to need it:
* :class:`~napalm_device_types.channel.CommandChannelMixin`
* :class:`~napalm_device_types.config_lifecycle.ConfigLifecycleMixin`
* :class:`~napalm_device_types.config_revisions.ConfigRevisionsMixin`
* :class:`~napalm_device_types.config_sync.ConfigSyncMixin`
* :class:`~napalm_device_types.container_engine.ContainerEngineMixin`
* :class:`~napalm_device_types.dhcp.DhcpServerMixin`
@@ -121,6 +122,12 @@ from napalm_device_types.clusters import (
HAResourceDict,
)
from napalm_device_types.config_lifecycle import ConfigLifecycleMixin
from napalm_device_types.config_revisions import (
ConfigRevertFailed,
ConfigRevisionDict,
ConfigRevisionsMixin,
UnknownConfigRevision,
)
from napalm_device_types.config_sync import ConfigSyncMixin
from napalm_device_types.container_engine import (
ContainerEngineConnection,
@@ -217,8 +224,10 @@ from napalm_device_types.os import OSDriver
from napalm_device_types.ospf import OspfMixin
from napalm_device_types.package_updates import (
APT_UPGRADABLE_COMMAND,
DNF_INSTALLED_CVES_COMMAND,
DNF_SECURITY_COMMAND,
parse_apt_upgradable,
parse_dnf_installed_cves,
parse_dnf_security,
)
from napalm_device_types.packages import PackageManagementMixin
@@ -399,6 +408,10 @@ __all__ = [
"open_stream_on_transport",
"run_on_transport",
"ConfigLifecycleMixin",
"ConfigRevisionsMixin",
"ConfigRevisionDict",
"ConfigRevertFailed",
"UnknownConfigRevision",
"ConfigSyncDict",
"ConfigSyncMixin",
"DhcpHaDict",
@@ -432,8 +445,10 @@ __all__ = [
"OSDriver",
"PackageManagementMixin",
"APT_UPGRADABLE_COMMAND",
"DNF_INSTALLED_CVES_COMMAND",
"DNF_SECURITY_COMMAND",
"parse_apt_upgradable",
"parse_dnf_installed_cves",
"parse_dnf_security",
"parse_host_status",
"strip_terminal_codes",
+116
View File
@@ -0,0 +1,116 @@
"""The order an applied reservation set is written in (#106).
A DHCP server such as Kea keeps every address unique at each write, not only at the
commit: a reservation that takes an address another one still holds is refused as a
duplicate. So the changes are ordered for an address to be released before it is taken:
* An update that moves a reservation off its address goes before the change that takes
that address -- along a chain, from its free end.
* Adds come after every update: they release nothing, and an update may free their
address.
A *swap* -- two reservations trading addresses, or a longer rotation -- has no such
order. Its changes are still tried, in the order of the desired set: a server that checks
only at the commit takes them. A server that refuses them refuses each with its own
message, and the progress line says it was a swap. Nothing is moved to a temporary
address: the contract knows no free one (pools, leases and reservations nobody manages
all hold addresses), and a second write that failed would leave the reservation on it.
Without I/O: a reconciler's part, see ``docs/STRUCTURE.md``.
"""
from __future__ import annotations
import ipaddress
from dataclasses import dataclass
from napalm_device_types.models import DhcpReservationDict, DhcpReservationDiffDict
@dataclass(frozen=True)
class ReservationChange:
"""One write of an applied set, in the order it is made."""
reservation: DhcpReservationDict
#: The live reservation's id for an update, None for an add.
uuid: str | None
#: The changed fields of an update (``"ip, hostname"``), empty for an add.
fields: str = ""
#: The other reservations' MACs of the swap this change is part of, if any.
swap: tuple[str, ...] = ()
def _address(ip: str | None) -> str:
"""*ip* in one spelling, so ``192.0.2.010``-style differences cannot hide a match."""
text = (ip or "").strip()
try:
return str(ipaddress.ip_address(text))
except ValueError:
return text.lower()
def _waits_on(updates: list[tuple[str, str]]) -> list[int | None]:
"""For each ``(old, new)`` address pair, the update whose release it waits for."""
released = {old: i for i, (old, new) in enumerate(updates) if old and old != new}
return [released.get(new) if new != old else None for old, new in updates]
def _cycles(waits_on: list[int | None]) -> dict[int, list[int]]:
"""Each update that is part of a cycle, with the cycle's members in order."""
members: dict[int, list[int]] = {}
for start in range(len(waits_on)):
path: list[int] = []
node: int | None = start
while node is not None and node not in path and node not in members:
path.append(node)
node = waits_on[node]
if node is not None and node in path:
cycle = sorted(path[path.index(node) :])
members.update(dict.fromkeys(cycle, cycle))
return members
def _update_order(waits_on: list[int | None], cycles: dict[int, list[int]]) -> list[int]:
"""Every update after the one it waits on; stable otherwise. Where only cycles are
left, the first of them goes as it is, and so on."""
order: list[int] = []
done: set[int] = set()
pending = list(range(len(waits_on)))
while pending:
ready = [i for i in pending if waits_on[i] is None or waits_on[i] in done]
if not ready:
ready = [next(i for i in pending if i in cycles)]
order += ready
done.update(ready)
pending = [i for i in pending if i not in done]
return order
def order_reservation_changes(
diff: DhcpReservationDiffDict, live: list[DhcpReservationDict]
) -> list[ReservationChange]:
"""The changes of *diff* in the order they are written; see the module.
:param diff: What ``diff_dhcp_reservations`` found.
:param live: The reservations it compared against, for the addresses updates release.
"""
held = {r.get("uuid"): _address(r.get("ip")) for r in live if r.get("uuid")}
updates = diff["update"]
waits_on = _waits_on(
[(held.get(e["uuid"], ""), _address(e["reservation"].get("ip"))) for e in updates]
)
cycles = _cycles(waits_on)
def swap(i: int) -> tuple[str, ...]:
return tuple(updates[j]["reservation"]["mac"] for j in cycles.get(i, []) if j != i)
ordered = [
ReservationChange(
updates[i]["reservation"],
updates[i]["uuid"],
", ".join(updates[i]["changed_fields"]),
swap(i),
)
for i in _update_order(waits_on, cycles)
]
return ordered + [ReservationChange(r, None) for r in diff["add"]]
+140
View File
@@ -0,0 +1,140 @@
"""A device's own configuration history: list it, read a revision, revert to one.
Some devices keep their earlier configurations themselves: a copy of the
configuration for every save, which the device can go back to. An application
restores a configuration there by reverting to the revision that holds it -- on a
device whose API takes no uploaded configuration (OPNsense), the only way
(NetOrk/netork#2511). :class:`ConfigRevisionsMixin` is the contract:
* ``get_config_revisions()`` and ``get_config_revision(revision_id)`` -- declared
only, so ``callable(getattr(driver, "get_config_revisions", None))`` is the
truthful answer to "does this device keep a history";
* ``revert_config_revision(revision_id)`` -- a template: it refuses an id that is not
in the history, reads the revision, lets the driver revert
(``_revert_to_config_revision``), and reads the configuration in place afterwards
to check that it holds the revision's. A device may answer that it reverted when it
did not; the check does not take its word;
* ``config_revision_content(text)`` -- what of a revision is its configuration. Every
save may stamp a revision anew (OPNsense writes who saved it and when into
config.xml), so two revisions with the same configuration differ as text. A driver
whose device does that strips the stamp here; by default the whole text counts. An
application that looks for the revision holding a configuration compares through
it too.
A driver mixes it in when its device keeps a history; no role base carries it. One
that can list the history but not revert to it sets ``revert_config_revision`` to
``None``.
"""
from __future__ import annotations
from typing import TYPE_CHECKING, TypedDict
class ConfigRevisionDict(TypedDict):
"""One revision of the device's configuration (``get_config_revisions``)."""
id: str # what get_config_revision and revert_config_revision take, as the device names it
time: str | None # when it was saved, UTC ISO 8601; None when the device does not say
description: str | None # what made the change, as the device records it; None: none
user: str | None # who saved it, as the device names them; None when it names nobody
size: int | None # its size in bytes; None when not given
class UnknownConfigRevision(ValueError):
"""The device's configuration history has no revision of that id."""
def __init__(self, revision_id: object) -> None:
super().__init__(f"no configuration revision {revision_id!r} in the device's history")
class ConfigRevertFailed(RuntimeError):
"""The device refused the revert, or its configuration afterwards is not the revision's."""
class ConfigRevisionsMixin:
"""List, read and revert to the device's configuration revisions; see the module."""
if TYPE_CHECKING: # pragma: no cover - declared for type checkers only
def get_config_revisions(self) -> list[ConfigRevisionDict]:
"""The device's configuration history, newest first.
The first revision is the configuration in place now.
Example::
[
{
"id": "config-1794603885.1239.xml",
"time": "2026-11-13T21:04:45+00:00",
"description": "/api/firewall/filter/apply made changes",
"user": "apiuser@192.0.2.10",
"size": 58213,
},
]
:raises Exception: when the history could not be read; never an empty
list for a failed read.
"""
...
def get_config_revision(self, revision_id: str) -> str:
"""The configuration of revision *revision_id*, as the device keeps it.
:raises UnknownConfigRevision: for an id that is not in the history.
:raises Exception: when the device could not be read.
"""
...
# The step a driver implements:
def _revert_to_config_revision(self, revision_id: str) -> None:
"""Make revision *revision_id* the device's configuration; return once it is
in place, as the newest revision of the history.
:raises UnknownConfigRevision: when the device no longer has it.
:raises ConfigRevertFailed: when the device refuses.
"""
...
def config_revision_content(self, text: str) -> str:
"""What of a revision's *text* is its configuration: the whole text by default.
A driver whose device stamps every save into the configuration (who, when)
returns the text without that stamp, so that two revisions with the same
configuration compare equal.
"""
return text
def revert_config_revision(self, revision_id: str) -> ConfigRevisionDict:
"""Make revision *revision_id* the device's configuration again.
Returns the revision in place afterwards: the newest of the history, which
holds the configuration of *revision_id* (on most devices a new revision the
revert saved). What the device does beyond that -- reload its services,
restart, nothing -- is the driver's to document.
:raises UnknownConfigRevision: for an id that is not in the history now;
nothing has changed then.
:raises ConfigRevertFailed: when the device refuses, or when the configuration
in place afterwards is not the revision's.
:raises Exception: when the device could not be reached.
"""
if revision_id not in {r["id"] for r in self.get_config_revisions()}:
raise UnknownConfigRevision(revision_id)
# Read before the revert: saving it may rotate the revision out of the history.
wanted = self.config_revision_content(self.get_config_revision(revision_id))
self._revert_to_config_revision(revision_id)
revisions = self.get_config_revisions()
if not revisions:
raise ConfigRevertFailed(
f"the device has no configuration after reverting to {revision_id!r}"
)
now = revisions[0]
if self.config_revision_content(self.get_config_revision(now["id"])) != wanted:
raise ConfigRevertFailed(
f"the configuration in place ({now['id']!r}) is not revision {revision_id!r}"
" after the revert"
)
return now
+113 -57
View File
@@ -26,6 +26,7 @@ import re
from collections.abc import Iterator
from typing import TYPE_CHECKING, Any
from napalm_device_types._dhcp_order import ReservationChange, order_reservation_changes
from napalm_device_types._matching import check_desired, index_live
from napalm_device_types.models import (
DhcpHaDict,
@@ -57,18 +58,26 @@ _DHCP_SUBNET_COMPARE_FIELDS = (
_HEX_ONLY = re.compile(r"[^0-9a-f]")
#: Added to a refused change that is part of an address swap (#106).
_SWAP_NOTE = (
"an address swap with {}: no order frees an address first; "
"move one of them to a free address in a run of its own"
)
class DhcpApplyError(RuntimeError):
"""Some reservations of an applied set were refused by the device.
"""Some changes of an applied reservation or subnet set were refused by the device.
Raised after the rest were written and committed: ``failed`` of ``attempted``
changes did not go through, and each was named in a ``[failed]`` progress line.
``kind`` is what was applied: ``"reservation"`` or ``"subnet"``.
"""
def __init__(self, failed: int, attempted: int) -> None:
def __init__(self, failed: int, attempted: int, kind: str = "reservation") -> None:
self.failed = failed
self.attempted = attempted
super().__init__(f"{failed} of {attempted} DHCP reservation change(s) were refused")
self.kind = kind
super().__init__(f"{failed} of {attempted} DHCP {kind} change(s) were refused")
def normalize_mac(mac: str | None) -> str:
@@ -119,6 +128,43 @@ def _subnet_field_differs(field: str, live: DhcpSubnetDict, desired: DhcpSubnetD
return any(live_options.get(option) != value for option, value in desired_options.items())
def _reservation_key(reservation: DhcpReservationDict) -> str:
return normalize_mac(reservation.get("mac"))
def _diff_reservations(
live: list[DhcpReservationDict], desired: list[DhcpReservationDict]
) -> DhcpReservationDiffDict:
"""``diff_dhcp_reservations`` against reservations already read."""
live_by_mac, shared = index_live(live, _reservation_key)
check_desired(desired, _reservation_key, shared, "MAC")
add: list[DhcpReservationDict] = []
update: list[DhcpReservationUpdateDict] = []
for desired_reservation in desired:
current = live_by_mac.get(_reservation_key(desired_reservation))
if current is None:
add.append(desired_reservation)
continue
changed_fields = [
field
for field in _DHCP_RESERVATION_COMPARE_FIELDS
if current.get(field) != desired_reservation.get(field)
]
if changed_fields:
update.append(
{
"uuid": current["uuid"],
"reservation": desired_reservation,
"changed_fields": changed_fields,
}
)
return {"add": add, "update": update}
class DhcpServerMixin:
"""Mixin providing DHCP reservation and subnet read/diff/apply.
@@ -271,36 +317,7 @@ class DhcpServerMixin:
:raises ValueError: When a desired reservation cannot name a single live one.
"""
def key(reservation: DhcpReservationDict) -> str:
return normalize_mac(reservation.get("mac"))
live_by_mac, shared = index_live(self.get_dhcp_reservations(), key)
check_desired(desired, key, shared, "MAC")
add: list[DhcpReservationDict] = []
update: list[DhcpReservationUpdateDict] = []
for desired_reservation in desired:
live = live_by_mac.get(key(desired_reservation))
if live is None:
add.append(desired_reservation)
continue
changed_fields = [
field
for field in _DHCP_RESERVATION_COMPARE_FIELDS
if live.get(field) != desired_reservation.get(field)
]
if changed_fields:
update.append(
{
"uuid": live["uuid"],
"reservation": desired_reservation,
"changed_fields": changed_fields,
}
)
return {"add": add, "update": update}
return _diff_reservations(self.get_dhcp_reservations(), desired)
def apply_dhcp_reservationset(self, desired: list[DhcpReservationDict]) -> Iterator[str]:
"""
@@ -311,6 +328,9 @@ class DhcpServerMixin:
committing reloads the DHCP daemon and drops in-flight requests, which
is too high a price for a no-op run.
The changes are written so an address is released before it is taken; a
swap of addresses has no such order and is reported as one (``_dhcp_order``).
A change the device refuses (say, an address in a subnet its DHCP server
does not manage) does not stop the others: it gets a ``[failed]`` line,
the rest are written and committed, and ``DhcpApplyError`` is raised at
@@ -322,7 +342,8 @@ class DhcpServerMixin:
plus a final commit line.
:raises DhcpApplyError: after the commit, when a change was refused.
"""
diff = self.diff_dhcp_reservations(desired)
live = self.get_dhcp_reservations()
diff = _diff_reservations(live, desired)
attempted = len(diff["add"]) + len(diff["update"])
if not attempted:
yield "[commit] no changes"
@@ -330,17 +351,13 @@ class DhcpServerMixin:
added = updated = refused = 0
for reservation in diff["add"]:
line, ok = self._write_reservation(reservation, None, "[add]")
added, refused = added + ok, refused + (not ok)
for change in order_reservation_changes(diff, live):
line, ok = self._write_change(change)
refused += not ok
added += ok and change.uuid is None
updated += ok and change.uuid is not None
yield line
for entry in diff["update"]:
fields = ", ".join(entry["changed_fields"])
line, ok = self._write_reservation(entry["reservation"], entry["uuid"], "[update]")
updated, refused = updated + ok, refused + (not ok)
yield f"{line} ({fields})" if ok else line
# Committed even when every write was refused: a run that stopped before its
# commit left saved writes behind, which the device reports as configured, so
# no later diff would ever commit them.
@@ -350,6 +367,16 @@ class DhcpServerMixin:
if refused:
raise DhcpApplyError(refused, attempted)
def _write_change(self, change: ReservationChange) -> tuple[str, bool]:
"""Write one change of an ordered set; its progress line, and whether it went in."""
tag = "[add]" if change.uuid is None else "[update]"
line, ok = self._write_reservation(change.reservation, change.uuid, tag)
if ok and change.fields:
line = f"{line} ({change.fields})"
elif not ok and change.swap:
line = f"{line} -- {_SWAP_NOTE.format(', '.join(change.swap))}"
return line, ok
def _write_reservation(
self, reservation: DhcpReservationDict, uuid: str | None, tag: str
) -> tuple[str, bool]:
@@ -430,24 +457,53 @@ class DhcpServerMixin:
As with ``apply_dhcp_reservationset``, an empty diff does **not**
commit: reconfiguring the DHCP daemon is too expensive for a no-op.
A subnet the device refuses (say, one that overlaps a subnet it serves)
does not stop the others: it gets a ``[failed]`` line, the rest are
written and committed, and ``DhcpApplyError`` is raised at the end. Any
non-empty diff commits, even one whose every write was refused, so that
writes an earlier, interrupted run saved take effect (#113).
:param desired: The complete desired subnet set.
:yields: Progress lines, one per applied add/update, plus a final
commit line.
:yields: Progress lines, one per applied, updated or refused change,
plus a final commit line.
:raises DhcpApplyError: after the commit, when a change was refused.
"""
diff = self.diff_dhcp_subnets(desired)
for subnet in diff["add"]:
self.apply_dhcp_subnet(subnet)
yield f"[add] {subnet['subnet']}"
for entry in diff["update"]:
self.apply_dhcp_subnet(entry["subnet"], uuid=entry["uuid"])
fields = ", ".join(entry["changed_fields"])
yield f"[update] {entry['subnet']['subnet']} ({fields})"
if not diff["add"] and not diff["update"]:
attempted = len(diff["add"]) + len(diff["update"])
if not attempted:
yield "[commit] no changes"
return
changes: list[tuple[DhcpSubnetDict, str | None, str]] = [
(subnet, None, "") for subnet in diff["add"]
]
changes += [
(e["subnet"], e["uuid"], ", ".join(e["changed_fields"])) for e in diff["update"]
]
added = updated = refused = 0
for subnet, uuid, fields in changes:
line, ok = self._write_subnet(subnet, uuid, fields)
refused += not ok
added += ok and uuid is None
updated += ok and uuid is not None
yield line
# Committed even when every write was refused; see apply_dhcp_reservationset.
self.commit_dhcp_subnets()
yield (f"[commit] applied {len(diff['add'])} add(s), {len(diff['update'])} update(s)")
yield f"[commit] applied {added} add(s), {updated} update(s)"
if refused:
raise DhcpApplyError(refused, attempted, "subnet")
def _write_subnet(
self, subnet: DhcpSubnetDict, uuid: str | None, fields: str
) -> tuple[str, bool]:
"""Write one subnet; its progress line, and whether the device took it."""
try:
self.apply_dhcp_subnet(subnet, uuid=uuid)
except Exception as exc:
return f"[failed] {subnet['subnet']}: {exc}", False
if uuid is None:
return f"[add] {subnet['subnet']}", True
return f"[update] {subnet['subnet']} ({fields})", True
+37 -5
View File
@@ -60,6 +60,7 @@ too full is not decided here.
from __future__ import annotations
import re
from functools import cmp_to_key
from typing import TYPE_CHECKING
from napalm_device_types.models import (
@@ -171,9 +172,39 @@ def _sections(output: str) -> dict[str, list[str]]:
return sections
def _version_key(version: str) -> tuple[object, ...]:
"""Natural order: 6.8.0-142 after 6.8.0-87, 7.0.14 after 7.0.2."""
return tuple(int(part) if part.isdigit() else part for part in re.split(r"(\d+)", version))
#: A version's segments as rpm compares them: runs of digits, runs of letters, and
#: ``~``; whatever else stands between them only separates.
_VERSION_SEGMENT = re.compile(r"~|\d+|[A-Za-z]+")
def _segment_cmp(x: str, y: str) -> int:
"""One segment against another, as :func:`_version_cmp` takes them; 0 when equal."""
if "~" in (x, y):
return (y == "~") - (x == "~")
if x.isdigit() and y.isdigit():
return (int(x) > int(y)) - (int(x) < int(y))
if x.isdigit() or y.isdigit():
return 1 if x.isdigit() else -1
return (x > y) - (x < y)
def _version_cmp(a: str, b: str) -> int:
"""rpm's ``rpmvercmp``: below 0 when ``a`` is older, above when newer.
Numbers compare as numbers (6.8.0-142 after 6.8.0-87), letters as text, and a
number is newer than letters in its place: ``112.5.1.el9uek`` after
``112.5.el9uek`` (#109). A version that has more segments where the other
ends is the newer one, unless its next is ``~``, which comes before anything:
``6.13~rc5`` before ``6.13``, as dpkg sorts it too.
"""
left, right = _VERSION_SEGMENT.findall(a), _VERSION_SEGMENT.findall(b)
for x, y in zip(left, right, strict=False):
if order := _segment_cmp(x, y):
return order
if len(left) == len(right):
return 0
longer, sign = (left, 1) if len(left) > len(right) else (right, -1)
return -sign if longer[min(len(left), len(right))] == "~" else sign
def kernel_reboot_pending(running: str, installed: list[str]) -> str | None:
@@ -188,8 +219,9 @@ def kernel_reboot_pending(running: str, installed: list[str]) -> str | None:
same = [k for k in installed if _kernel_flavour(k)[1] == flavour]
if not same:
return None
newest = max(same, key=lambda k: _version_key(_kernel_flavour(k)[0]))
if _version_key(_kernel_flavour(newest)[0]) > _version_key(version):
order = cmp_to_key(_version_cmp)
newest = max(same, key=lambda k: order(_kernel_flavour(k)[0]))
if order(_kernel_flavour(newest)[0]) > order(version):
return newest
return None
+44 -16
View File
@@ -481,7 +481,12 @@ class LocalAccountsMixin:
return self._accounts_result(name, parts)
def add_service_account(
self, name: str, public_key: str, *, uid: int | None = None
self,
name: str,
public_key: str,
*,
uid: int | None = None,
replaces: Sequence[str] = (),
) -> LocalAccountChangeDict:
"""Make *name* netOrk's own account: it logs in with *public_key*, and gains
root without a password.
@@ -495,36 +500,59 @@ class LocalAccountsMixin:
once no other account has it; an account that is there keeps its own,
as a uid is never changed under files that carry it.
*replaces* are netOrk's keys *public_key* takes the place of: they go
from the account once *public_key* is in it, matched as in
:meth:`add_local_account`, and never *public_key* itself. Call it so
through a login with *public_key*, tried before: netOrk keeps its way
in (NetOrk/netork#2462).
Steps as in :meth:`add_local_account`: ``account``, then ``sudo`` and
``keys``. The new login itself is for the caller to try.
``keys``, and ``old keys`` with *replaces* when ``keys`` went. The new
login itself is for the caller to try.
An account that is there and cannot log in (``nologin``, ``false``) is
another service's -- a satellite's ``netork``, say -- and is left alone.
:raises ValueError: for a name or key no host takes, root, a uid of 0 or
less, or another service's account.
:raises ValueError: for a name or key no host takes, a replaced key that
is none, root, a uid of 0 or less, or another service's account.
:raises RuntimeError: when the login user cannot be told.
"""
check_public_key(public_key)
old = [key for key in _checked_keys(replaces) if _key_blob(key) != _key_blob(public_key)]
login, existing = self._accounts_service_target(name, uid)
first = self._accounts_service_account_step(name, login, existing, uid)
parts = [first]
if first["success"]:
parts += self._accounts_service_keys(name, public_key, old)
return self._accounts_result(name, parts)
def _accounts_service_account_step(
self, name: str, login: str, existing: list[str] | None, uid: int | None
) -> FixResultDict:
"""The ``account`` step of netOrk's own account; nothing to do when the driver
logs in as *name* already."""
if name == login:
step: FixStepDict = {
"step": "account",
"ok": True,
"output": f"this driver logs in as {name} already",
}
first: FixResultDict = _joined([step])
else:
first = run_fix_steps(
[("account", lambda: self._accounts_service_account(name, existing, uid))]
)
parts = [first]
if first["success"]:
parts += [
run_fix_steps([("sudo", lambda: self._accounts_grant_service_sudo(name))]),
run_fix_steps([("keys", lambda: self._accounts_add_key(name, public_key))]),
]
return self._accounts_result(name, parts)
return _joined([step])
return run_fix_steps(
[("account", lambda: self._accounts_service_account(name, existing, uid))]
)
def _accounts_service_keys(
self, name: str, public_key: str, old: list[str]
) -> list[FixResultDict]:
"""``sudo`` and ``keys`` of netOrk's own account, then ``old keys`` once the key is in."""
parts = [
run_fix_steps([("sudo", lambda: self._accounts_grant_service_sudo(name))]),
run_fix_steps([("keys", lambda: self._accounts_add_key(name, public_key))]),
]
if old and parts[-1]["success"]:
parts.append(run_fix_steps([("old keys", lambda: self._accounts_drop_keys(name, old))]))
return parts
def _accounts_service_target(self, name: str, uid: int | None) -> tuple[str, list[str] | None]:
"""The login user and *name*'s passwd line, once *name* may become netOrk's.
+39 -4
View File
@@ -35,6 +35,9 @@ class ForwardRuleDict(TypedDict):
selector: str # "*.*", "authpriv.*", or the condition around it
complete: bool # every message of its ruleset goes
ruleset: str | None # None: the default ruleset, which gets the journal and syslog
#: The template its messages are sent in; None: rsyslog's default, RFC 3164,
#: which carries the sender's local time without a zone (NetOrk/netork#493).
template: str | None
class FileInputDict(TypedDict):
@@ -165,6 +168,10 @@ class _Reader:
self.pending: tuple[str, str | None] | None = None
self.legacy_ruleset: str | None = None
self.last_selector = "*.*"
#: ``$ActionForwardDefaultTemplate``, for the legacy rules after it.
self.legacy_template: str | None = None
#: ``module(load="builtin:omfwd" template=…)``, for every omfwd action.
self.module_template: str | None = None
def ruleset(self) -> str | None:
names = [value for kind, value in self.stack if kind == "ruleset"]
@@ -175,7 +182,13 @@ class _Reader:
return once or (conditions[-1] if conditions else None)
def forward(
self, protocol: str, target: str, port: int | None, selector: str, once: str | None
self,
protocol: str,
target: str,
port: int | None,
selector: str,
once: str | None,
template: str | None,
) -> None:
condition = self.condition(once)
self.parsed["rules"].append(
@@ -186,6 +199,7 @@ class _Reader:
"selector": condition or selector,
"complete": condition is None and selector in _EVERYTHING,
"ruleset": self.ruleset(),
"template": template,
}
)
@@ -225,6 +239,8 @@ class _Reader:
def call(self, name: str, params: dict[str, str], before: str, once: str | None) -> None:
if name == "module":
self.module(params.get("load", ""))
if params.get("load", "").removeprefix("builtin:") == "omfwd":
self.module_template = params.get("template") or self.module_template
elif name == "ruleset":
self.pending = ("ruleset", params.get("name"))
elif name == "input" and params.get("type") == "imfile" and params.get("file"):
@@ -240,7 +256,10 @@ class _Reader:
kind = params.get("type", "")
if kind in _FORWARD_TYPES and params.get("target"):
protocol = "relp" if kind == "omrelp" else params.get("protocol", "udp").lower()
self.forward(protocol, params["target"], _port(params.get("port")), selector, once)
template = params.get("template") or (self.module_template if kind == "omfwd" else None)
self.forward(
protocol, params["target"], _port(params.get("port")), selector, once, template
)
def module(self, name: str) -> None:
if name == "imjournal":
@@ -257,6 +276,8 @@ class _Reader:
self.parsed["file_inputs"].append({"path": value, "ruleset": self.legacy_ruleset})
elif name == "$ruleset":
self.legacy_ruleset = None if value == "RSYSLOG_DefaultRuleset" else value
elif name == "$actionforwarddefaulttemplate":
self.legacy_template = value or None
def legacy(self, text: str, once: str | None) -> None:
match = _PROPERTY_RE.match(text) or _LEGACY_RE.match(text)
@@ -264,13 +285,27 @@ class _Reader:
return
selector = self.last_selector if match["sel"] == "&" else match["sel"]
self.last_selector = selector
forward = _FORWARD_RE.match(match["action"].strip())
action = match["action"].strip()
forward = _FORWARD_RE.match(action)
if forward:
protocol = {"@": "udp", "@@": "tcp"}.get(forward["at"] or "", "relp")
self.forward(
protocol, forward["v6"] or forward["host"], _port(forward["port"]), selector, once
protocol,
forward["v6"] or forward["host"],
_port(forward["port"]),
selector,
once,
self.legacy_template_of(action[forward.end() :], protocol),
)
def legacy_template_of(self, rest: str, protocol: str) -> str | None:
"""The template after a legacy rule's target (``;Name``), else the default
set before it; RELP takes no forwarding default."""
_, semicolon, named = rest.partition(";")
if semicolon and named.strip():
return named.strip()
return self.legacy_template if protocol != "relp" else None
def parse_rsyslog(text: str) -> RsyslogDict:
"""rsyslog's configuration -- ``rsyslog.conf`` and ``rsyslog.d`` as one text --
+19 -6
View File
@@ -13,9 +13,13 @@ what ``dnf check --duplicates`` and ``package-cleanup --dupes`` report. Packages
dnf installs side by side on purpose do not count: its default ``installonlypkgs``
(``kernel``, ``installonlypkg(kernel)`` and the rest), plus any the host adds in
``dnf.conf``, asked of rpm by what they provide; and ``gpg-pubkey``, one entry
per imported key. Also interrupted is a transaction that never ended: dnf 4's
history (libdnf's ``trans`` table) holds its row with ``state`` 0 until
``finish()`` writes 1 (done) or 2 (error), so the newest row at 0 is one cut off;
per imported key. The defaults are the ones the host's libdnf was built with, asked
of its Python binding, as a distribution may add its own: Oracle Linux adds
``kernel-uek`` and its parts, whose packages provide nothing of upstream's list.
Upstream's list is the floor where libdnf cannot be asked (dnf 5, yum). Also
interrupted is a transaction that never ended: dnf 4's history (libdnf's ``trans``
table) holds its row with ``state`` 0 until ``finish()`` writes 1 (done) or 2
(error), so the newest row at 0 is one cut off;
yum leaves a ``transaction-all.*`` journal in ``/var/lib/yum``, which
``yum-complete-transaction`` reads (one renamed ``.disabled`` is given up). The
history is read with Python's sqlite3 module in read-only mode, as a stock EL host
@@ -81,6 +85,13 @@ _DNF_HISTORY_PY = (
"r = c.execute('SELECT id, state FROM trans ORDER BY id DESC LIMIT 1').fetchone(); "
"r and print(*r)"
)
#: The ``installonlypkgs`` the host's libdnf (dnf 4) was built with, space-separated;
#: nothing where there is no libdnf. ``ConfigMain`` is held in a name: an option
#: taken from a temporary one outlives it, and reading it then crashes Python.
_LIBDNF_INSTALLONLY_PY = (
"import libdnf.conf; c = libdnf.conf.ConfigMain(); "
"print(c.installonlypkgs().getValueString().replace(',', ' '))"
)
_RUNNING_PART = (
"echo '[pm-running]'; cat /proc/[0-9]*/comm 2>/dev/null | grep -xE '"
@@ -90,7 +101,10 @@ _RUNNING_PART = (
_RPM_PART = (
"if command -v rpm >/dev/null 2>&1 && { command -v dnf || command -v yum; } >/dev/null 2>&1; "
"then echo '[rpm-frontend]'; { command -v dnf || command -v yum; } | sed 's#.*/##'; "
"rpmpy=; for p in /usr/libexec/platform-python python3; do "
"if command -v $p >/dev/null 2>&1; then rpmpy=$p; break; fi; done; "
f"echo '[rpm-installonly]'; io=\"{' '.join(_INSTALLONLY)} "
f'$([ -n "$rpmpy" ] && $rpmpy -c "{_LIBDNF_INSTALLONLY_PY}" 2>/dev/null) '
f"$(sed -n 's/^[[:space:]]*installonlypkgs[[:space:]]*=//p' {_DNF_CONF} 2>/dev/null "
"| tr ',' ' ')\"; rpm -q --whatprovides $io --qf '%{NAME}\\n' 2>/dev/null "
"| grep -v ' ' | sort -u; "
@@ -98,9 +112,8 @@ _RPM_PART = (
"| grep -v '^gpg-pubkey ' | sort | uniq -d | head -n 50; "
f"echo '[yum-journal]'; ls -1 {YUM_DIR} 2>/dev/null | grep '^transaction-all\\.' "
"| head -n 20; "
"echo '[dnf-history]'; for py in /usr/libexec/platform-python python3; do "
f'if command -v $py >/dev/null 2>&1; then $py -c "{_DNF_HISTORY_PY}" 2>/dev/null; '
"break; fi; done; fi; "
"echo '[dnf-history]'; "
f'[ -n "$rpmpy" ] && $rpmpy -c "{_DNF_HISTORY_PY}" 2>/dev/null; fi; '
)
_APK_PART = (
f"if command -v apk >/dev/null 2>&1 && [ -r {APK_INSTALLED} ]; then echo '[apk-audit]'; "
+31
View File
@@ -36,11 +36,24 @@ DNF_SECURITY_COMMAND = (
" echo __DNF_RC=$?; } | cat"
)
#: Read-only. Every advisory whose fix the installed packages carry, one line per
#: CVE it names (``CVE SEVERITY/Sec. NEVRA``): the distribution's own word that a
#: backport is in, where netOrk's knowledge base holds no records of the release
#: (netOrk #2394). dnf 4 only: dnf 5 reads ``--with-cve`` as a filter, and
#: Fedora's advisories reference bugs rather than CVEs. From the cache, as
#: :data:`DNF_SECURITY_COMMAND` reads it; the output is long (90 000 lines on an
#: Oracle Linux 9 host), so a caller reads it seldom.
DNF_INSTALLED_CVES_COMMAND = (
"{ LC_ALL=C dnf --cacheonly updateinfo list --installed --with-cve --quiet 2>/dev/null;"
" echo __DNF_RC=$?; } | cat"
)
# openssl/noble-updates,noble-security 3.0.13-0ubuntu3.6 amd64 [upgradable from: 3.0.13-0ubuntu3.5]
_APT_LINE = re.compile(r"^(\S+)/(\S+)\s+(\S+)\s+\S+\s+\[upgradable from:\s+(\S+)\]")
_SECURITY_SUITE = "-security"
_APT_STATUS = re.compile(r"^__APT_RC=(\d+)\s*$", re.MULTILINE)
_DNF_STATUS = re.compile(r"^__DNF_RC=(\d+)\s*$", re.MULTILINE)
_CVE_ID = re.compile(r"^CVE-\d{4}-\d{4,}$")
def _joined_lines(output: str) -> list[str]:
@@ -118,3 +131,21 @@ def parse_dnf_security(output: str) -> set[str]:
if len(parts) >= 3 and parts[1].endswith("/Sec."):
names.add(nevra_name(parts[-1]))
return names
def parse_dnf_installed_cves(output: str) -> dict[str, list[str]]:
"""The CVEs the installed advisories close, per package name, each once and sorted.
Parses :data:`DNF_INSTALLED_CVES_COMMAND`'s ``CVE SEVERITY/Sec. NEVRA`` lines;
a line that names no CVE (a key prompt, an advisory id) is no statement.
:raises ValueError: when dnf failed or its exit status never arrived -- "could
not read" must not look like "no advisory installed".
"""
by_name: dict[str, set[str]] = {}
listing = _listing(output, _DNF_STATUS, "dnf updateinfo list --installed --with-cve")
for line in listing.splitlines():
parts = line.split()
if len(parts) >= 3 and _CVE_ID.match(parts[0]):
by_name.setdefault(nevra_name(parts[-1]), set()).add(parts[0])
return {name: sorted(cves) for name, cves in by_name.items()}
+22
View File
@@ -50,6 +50,28 @@ class UpdateMixin:
"""
...
def get_installed_advisory_cves(self) -> dict[str, list[str]]:
"""
The CVEs the host's installed security advisories close, per package
name: the distribution's own word that a fix is in the installed build,
although its upstream version did not move (a backport).
netOrk asks for it only where its knowledge base holds no records of
the host's release (netOrk #2394), since the answer can be long. A
package manager whose advisories name no CVEs has nothing to say here.
**An empty dict means no installed advisory names a CVE.** A reader
that cannot read raises, as :meth:`get_available_updates` does.
``NotImplementedError`` where the host's package manager keeps no such
advisories (:data:`~napalm_device_types.package_updates.DNF_INSTALLED_CVES_COMMAND`
is dnf 4's).
Example::
{"glibc-common": ["CVE-2023-4911"], "curl": ["CVE-2023-38545"]}
"""
...
def refresh_available_updates(self) -> dict[str, Any]:
"""
Refreshes the host's package index, so that :meth:`get_available_updates`
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "napalm-device-types"
version = "4.36.0"
version = "4.38.0"
description = "Abstract device-type base classes for NAPALM drivers"
readme = "README.md"
requires-python = ">=3.12"
-1
View File
@@ -20,7 +20,6 @@
},
"function_lines": {
"napalm_device_types/_ucd_metrics.py::collect_ucd_metrics": 119,
"napalm_device_types/dhcp.py::DhcpServerMixin.diff_dhcp_reservations": 54,
"napalm_device_types/dhcp.py::DhcpServerMixin.diff_dhcp_subnets": 60,
"napalm_device_types/firewall_rules.py::FirewallRuleMixin.diff_firewall_rules": 56,
"napalm_device_types/hypervisor.py::HypervisorDriver.create_vm_from_cloud_init": 84,
+172
View File
@@ -0,0 +1,172 @@
"""ConfigRevisionsMixin: a device's own configuration history (NetOrk/netork#2511).
Some devices keep their earlier configurations themselves. An application restores a
configuration there by reverting to the revision that holds it -- on OPNsense the only
way, since no API takes an uploaded config.xml. Listing and reading are declared only,
so ``callable(getattr(driver, "get_config_revisions", None))`` is the truthful answer;
``revert_config_revision`` is a template: it refuses an id that is not in the history,
lets the driver revert, and checks that the configuration in place now is the
revision's.
"""
from __future__ import annotations
import inspect
from typing import get_type_hints
import pytest
from napalm_device_types import (
ConfigRevertFailed,
ConfigRevisionDict,
ConfigRevisionsMixin,
DeviceTypeDriver,
FirewallDriver,
OSDriver,
SwitchDriver,
UnknownConfigRevision,
)
def test_the_shape():
assert get_type_hints(ConfigRevisionDict) == {
"id": str,
"time": str | None,
"description": str | None,
"user": str | None,
"size": int | None,
}
def test_no_role_demands_it():
for role in (FirewallDriver, OSDriver, SwitchDriver, DeviceTypeDriver):
assert not issubclass(role, ConfigRevisionsMixin)
def test_listing_and_reading_are_declared_not_implemented():
assert not hasattr(ConfigRevisionsMixin, "get_config_revisions")
assert not hasattr(ConfigRevisionsMixin, "get_config_revision")
source = inspect.getsource(ConfigRevisionsMixin)
assert "def get_config_revisions(self) -> list[ConfigRevisionDict]" in source
assert "def get_config_revision(self, revision_id: str) -> str" in source
assert "def _revert_to_config_revision(self, revision_id: str) -> None" in source
def test_an_unknown_revision_is_a_value_error_that_names_it():
error = UnknownConfigRevision("r9")
assert isinstance(error, ValueError)
assert "'r9'" in str(error)
def test_a_failed_revert_is_a_runtime_error():
assert issubclass(ConfigRevertFailed, RuntimeError)
def _revision(revision_id: str, time: str) -> ConfigRevisionDict:
return {"id": revision_id, "time": time, "description": None, "user": None, "size": None}
class _Device(ConfigRevisionsMixin):
"""A device that keeps its history in memory, newest first, and saves a revert as a
new newest revision with a stamp of its own on the first line."""
def __init__(self) -> None:
self.history = [
(_revision("r3", "2026-11-13T21:00:00+00:00"), "#saved 3\nb\n"),
(_revision("r2", "2026-11-12T21:00:00+00:00"), "#saved 2\na\n"),
(_revision("r1", "2026-11-11T21:00:00+00:00"), "#saved 1\na\n"),
]
self.calls: list[str] = []
def get_config_revisions(self) -> list[ConfigRevisionDict]:
self.calls.append("list")
return [revision for revision, _ in self.history]
def get_config_revision(self, revision_id: str) -> str:
self.calls.append(f"read {revision_id}")
for revision, text in self.history:
if revision["id"] == revision_id:
return text
raise UnknownConfigRevision(revision_id)
def _revert_to_config_revision(self, revision_id: str) -> None:
self.calls.append(f"revert {revision_id}")
body = self.get_config_revision(revision_id).split("\n", 1)[1]
self.history.insert(0, (_revision("r4", "2026-11-14T08:00:00+00:00"), f"#saved 4\n{body}"))
def config_revision_content(self, text: str) -> str:
return text.split("\n", 1)[1]
class TestRevertConfigRevision:
def test_reverts_and_answers_with_the_revision_now_in_place(self):
device = _Device()
now = device.revert_config_revision("r1")
assert now["id"] == "r4"
assert device.calls == ["list", "read r1", "revert r1", "read r1", "list", "read r4"]
def test_the_revision_is_read_before_the_revert_which_may_rotate_it_out(self):
class Rotating(_Device):
def _revert_to_config_revision(self, revision_id: str) -> None:
super()._revert_to_config_revision(revision_id)
self.history.pop() # the oldest, r1, drops out
assert Rotating().revert_config_revision("r1")["id"] == "r4"
def test_a_revision_not_in_the_history_is_refused_before_anything_changes(self):
device = _Device()
with pytest.raises(UnknownConfigRevision, match="r9"):
device.revert_config_revision("r9")
assert device.calls == ["list"]
def test_a_revert_the_device_did_not_take_fails(self):
# OPNsense answers "reverted" even when it fell back to the previous file.
class Deaf(_Device):
def _revert_to_config_revision(self, revision_id: str) -> None:
self.calls.append(f"revert {revision_id}")
with pytest.raises(ConfigRevertFailed, match="r1"):
Deaf().revert_config_revision("r1")
def test_a_history_that_is_empty_afterwards_fails(self):
class Forgetful(_Device):
def _revert_to_config_revision(self, revision_id: str) -> None:
self.history.clear()
with pytest.raises(ConfigRevertFailed):
Forgetful().revert_config_revision("r1")
def test_what_a_save_stamps_anew_does_not_count(self):
# r2 and r1 differ only in their stamp: reverting to r2 leaves r1's content.
device = _Device()
assert device.revert_config_revision("r2")["id"] == "r4"
def test_a_refusal_of_the_device_reaches_the_caller(self):
class Refusing(_Device):
def _revert_to_config_revision(self, revision_id: str) -> None:
raise ConfigRevertFailed("read-only user")
with pytest.raises(ConfigRevertFailed, match="read-only"):
Refusing().revert_config_revision("r1")
class TestConfigRevisionContent:
def test_by_default_the_whole_text(self):
class Plain(ConfigRevisionsMixin):
pass
assert Plain().config_revision_content("a\nb\n") == "a\nb\n"
def test_without_a_stamp_every_save_differs(self):
class Plain(_Device):
def config_revision_content(self, text: str) -> str:
return ConfigRevisionsMixin.config_revision_content(self, text)
with pytest.raises(ConfigRevertFailed):
Plain().revert_config_revision("r1")
+150
View File
@@ -298,6 +298,156 @@ class TestApplyWithARefusedReservation:
assert issubclass(DhcpApplyError, RuntimeError)
#: What Kea answers through OPNsense's API for an address another reservation holds.
_KEA_DUPLICATE = (
"{'result': 'failed', 'validations': {'reservation.ip_address': 'Duplicate entry exists.'}}"
)
class _KeaLikeServer(_FakeDhcpServer):
"""A server that keeps each address unique at every write, as Kea does: a write
taking an address another reservation still holds is refused."""
def apply_dhcp_reservation(
self, reservation: DhcpReservationDict, *, uuid: str | None = None
) -> dict[str, Any]:
if any(r["ip"] == reservation["ip"] and r["uuid"] != uuid for r in self.live):
raise RuntimeError(_KEA_DUPLICATE)
stored = {**reservation, "uuid": uuid or f"new-{len(self.live)}"}
self.live = [r for r in self.live if r["uuid"] != uuid] + [stored] # type: ignore[list-item]
return super().apply_dhcp_reservation(reservation, uuid=uuid)
def ip_of(self, mac: str) -> str:
return next(r["ip"] for r in self.live if r["mac"] == mac)
_A, _B, _C, _N = (f"02:00:00:00:00:0{i}" for i in "abcd")
def _at(mac: str, ip: str, uuid: str = "") -> DhcpReservationDict:
return _reservation(mac=mac, ip=ip, uuid=uuid, subnet="192.0.2.0/24")
class TestApplyOrderReleasesBeforeTaking:
"""An address is released before another change takes it (#106)."""
def test_an_update_that_frees_an_address_goes_first(self):
# The issue: B takes A's address, A moves on; desired lists B first.
driver = _KeaLikeServer([_at(_A, "192.0.2.10", "ua"), _at(_B, "192.0.2.11", "ub")])
lines = list(
driver.apply_dhcp_reservationset([_at(_B, "192.0.2.10"), _at(_A, "192.0.2.20")])
)
assert driver.ip_of(_A) == "192.0.2.20"
assert driver.ip_of(_B) == "192.0.2.10"
assert lines == [
f"[update] {_A} -> 192.0.2.20 (ip)",
f"[update] {_B} -> 192.0.2.10 (ip)",
"[commit] applied 0 add(s), 2 update(s)",
]
def test_a_chain_is_written_from_its_free_end(self):
live = [
_at(_A, "192.0.2.10", "ua"),
_at(_B, "192.0.2.11", "ub"),
_at(_C, "192.0.2.12", "uc"),
]
driver = _KeaLikeServer(live)
list(
driver.apply_dhcp_reservationset(
[_at(_A, "192.0.2.11"), _at(_B, "192.0.2.12"), _at(_C, "192.0.2.13")]
)
)
assert [e["reservation"]["mac"] for e in driver.applied] == [_C, _B, _A]
assert [driver.ip_of(m) for m in (_A, _B, _C)] == ["192.0.2.11", "192.0.2.12", "192.0.2.13"]
def test_an_add_comes_after_the_update_that_frees_its_address(self):
driver = _KeaLikeServer([_at(_A, "192.0.2.10", "ua")])
lines = list(
driver.apply_dhcp_reservationset([_at(_N, "192.0.2.10"), _at(_A, "192.0.2.20")])
)
assert driver.ip_of(_N) == "192.0.2.10"
assert driver.ip_of(_A) == "192.0.2.20"
assert lines[-1] == "[commit] applied 1 add(s), 1 update(s)"
def test_addresses_match_however_they_are_spelled(self):
driver = _FakeDhcpServer([_at(_A, "2001:DB8::10", "ua"), _at(_B, "2001:DB8::11", "ub")])
list(driver.apply_dhcp_reservationset([_at(_B, "2001:db8::10"), _at(_A, "2001:db8::20")]))
assert [e["reservation"]["mac"] for e in driver.applied] == [_A, _B]
def test_an_update_that_keeps_its_address_frees_nothing(self):
# A hostname change is no release: the add for A's address stays refused.
driver = _KeaLikeServer([_at(_A, "192.0.2.10", "ua")])
with pytest.raises(DhcpApplyError, match="1 of 2"):
list(
driver.apply_dhcp_reservationset(
[_at(_N, "192.0.2.10"), {**_at(_A, "192.0.2.10"), "hostname": "renamed"}]
)
)
assert driver.ip_of(_A) == "192.0.2.10"
class TestApplyOrderWithASwap:
"""Two reservations trading addresses: no order frees one first. Each is still
tried -- a server that checks only at the commit takes them -- and a refusal says it
was a swap. Nothing is moved to a temporary address: the contract has no way to know
a free one, and a second write that failed would leave the reservation there."""
def _swap(self) -> list[DhcpReservationDict]:
return [_at(_A, "192.0.2.11"), _at(_B, "192.0.2.10"), _at(_N, "192.0.2.30")]
def _live(self) -> list[DhcpReservationDict]:
return [_at(_A, "192.0.2.10", "ua"), _at(_B, "192.0.2.11", "ub")]
def test_a_refused_swap_is_reported_as_one_and_the_rest_is_written(self):
driver = _KeaLikeServer(self._live())
lines: list[str] = []
with pytest.raises(DhcpApplyError, match="2 of 3"):
for line in driver.apply_dhcp_reservationset(self._swap()):
lines.append(line)
failed = [line for line in lines if line.startswith("[failed]")]
assert len(failed) == 2
assert all("Duplicate entry exists" in line for line in failed)
assert f"swap with {_B}" in failed[0] and f"swap with {_A}" in failed[1]
assert driver.ip_of(_A) == "192.0.2.10" and driver.ip_of(_B) == "192.0.2.11"
assert driver.ip_of(_N) == "192.0.2.30"
assert driver.commits == 1
def test_a_server_that_takes_the_swap_gets_it(self):
driver = _FakeDhcpServer(self._live())
lines = list(driver.apply_dhcp_reservationset(self._swap()))
assert len(driver.applied) == 3
assert not any("swap" in line for line in lines)
def test_a_rotation_of_three_is_a_swap_too(self):
driver = _KeaLikeServer([*self._live(), _at(_C, "192.0.2.12", "uc")])
lines: list[str] = []
with pytest.raises(DhcpApplyError, match="3 of 3"):
for line in driver.apply_dhcp_reservationset(
[_at(_A, "192.0.2.11"), _at(_B, "192.0.2.12"), _at(_C, "192.0.2.10")]
):
lines.append(line)
by_mac = {line.split()[1]: line for line in lines if line.startswith("[failed]")}
assert f"swap with {_B}, {_C}" in by_mac[_A]
assert f"swap with {_A}, {_C}" in by_mac[_B]
assert f"swap with {_A}, {_B}" in by_mac[_C]
class TestKeysThatCannotNameOneReservation:
def test_a_live_reservation_without_mac_is_never_matched(self):
driver = _FakeDhcpServer([_reservation(uuid="b", mac="")])
+90
View File
@@ -16,6 +16,7 @@ from typing import Any
import pytest
from napalm_device_types import DhcpServerMixin
from napalm_device_types.dhcp import DhcpApplyError
from napalm_device_types.models import DhcpSubnetDict
@@ -235,3 +236,92 @@ def test_calling_a_missing_device_method_fails_loudly() -> None:
def test_a_subnet_that_cannot_name_one_raises(desired):
with pytest.raises(ValueError):
_FakeDhcpServer([]).diff_dhcp_subnets(desired)
# ── apply: a refused subnet does not hold back the others (#113) ──────────────
class _RefusingDhcpServer(_FakeDhcpServer):
"""A server that refuses the subnets in *refused*, as Kea refuses one that
overlaps a subnet it already serves."""
def __init__(self, live: list[DhcpSubnetDict] | None = None, *, refused: set[str]) -> None:
super().__init__(live)
self.refused = refused
def apply_dhcp_subnet(
self, subnet: DhcpSubnetDict, *, uuid: str | None = None
) -> dict[str, Any]:
if subnet["subnet"] in self.refused:
raise RuntimeError(f"{subnet['subnet']} overlaps a subnet the server has")
return super().apply_dhcp_subnet(subnet, uuid=uuid)
def _three_subnets() -> list[DhcpSubnetDict]:
return [
_subnet(subnet="192.0.2.0/26"),
_subnet(subnet="192.0.2.64/26"),
_subnet(subnet="192.0.2.128/26"),
]
def test_a_refused_subnet_does_not_stop_the_others() -> None:
fake = _RefusingDhcpServer(refused={"192.0.2.64/26"})
with pytest.raises(DhcpApplyError):
list(fake.apply_dhcp_subnetset(_three_subnets()))
assert [e["subnet"]["subnet"] for e in fake.applied] == ["192.0.2.0/26", "192.0.2.128/26"]
def test_what_was_written_is_committed() -> None:
fake = _RefusingDhcpServer(refused={"192.0.2.64/26"})
with pytest.raises(DhcpApplyError):
list(fake.apply_dhcp_subnetset(_three_subnets()))
assert fake.commits == 1
def test_a_set_whose_every_write_is_refused_still_commits() -> None:
# A run that stopped before its commit left saved writes behind, which the device
# reports as configured: no later diff would commit them.
fake = _RefusingDhcpServer(refused={"192.0.2.64/26"})
with pytest.raises(DhcpApplyError):
list(fake.apply_dhcp_subnetset([_three_subnets()[1]]))
assert fake.commits == 1
def test_progress_names_the_refused_subnet_and_why() -> None:
fake = _RefusingDhcpServer(
[_subnet(uuid="dev-1", subnet="192.0.2.128/26")], refused={"192.0.2.64/26"}
)
lines: list[str] = []
with pytest.raises(DhcpApplyError) as raised:
for line in fake.apply_dhcp_subnetset(
[*_three_subnets()[:2], _subnet(subnet="192.0.2.128/26", description="Office")]
):
lines.append(line)
assert lines == [
"[add] 192.0.2.0/26",
"[failed] 192.0.2.64/26: 192.0.2.64/26 overlaps a subnet the server has",
"[update] 192.0.2.128/26 (description)",
"[commit] applied 1 add(s), 1 update(s)",
]
assert (raised.value.failed, raised.value.attempted) == (1, 3)
assert str(raised.value) == "1 of 3 DHCP subnet change(s) were refused"
def test_a_refused_update_is_counted_like_an_add() -> None:
live = _subnet(uuid="dev-1", subnet="192.0.2.0/24")
fake = _RefusingDhcpServer([live], refused={"192.0.2.0/24"})
with pytest.raises(DhcpApplyError, match="1 of 1"):
list(fake.apply_dhcp_subnetset([_subnet(subnet="192.0.2.0/24", description="Office")]))
assert fake.applied == []
assert fake.commits == 1
+60 -2
View File
@@ -154,6 +154,24 @@ class TestKernelRebootPending:
"5.14.0-503.11.1.el9_5.x86_64",
),
("6.9.7-200.fc40.x86_64", ["6.9.7-200.fc40.x86_64"], None),
# Oracle UEK: 112.5.1 is newer than 112.5, as rpm counts -- a number after
# 5 beats the letters ``el`` (#109, build-arm-01).
(
"6.12.0-208.112.5.1.el9uek.aarch64",
[
"6.12.0-207.111.5.6.el9uek.aarch64",
"6.12.0-208.112.5.el9uek.aarch64",
"6.12.0-208.112.5.1.el9uek.aarch64",
],
None,
),
(
"6.12.0-208.112.5.el9uek.aarch64",
["6.12.0-208.112.5.el9uek.aarch64", "6.12.0-208.112.5.1.el9uek.aarch64"],
"6.12.0-208.112.5.1.el9uek.aarch64",
),
# A release candidate (``~``) comes before its release, in rpm and in dpkg.
("6.13~rc5-1-amd64", ["6.13~rc5-1-amd64", "6.13-1-amd64"], "6.13-1-amd64"),
],
)
def test_the_newer_kernel_of_the_running_flavour(self, running, installed, newer):
@@ -598,8 +616,8 @@ class TestInterruptedRpm:
The outputs are shaped after the documented formats -- ``rpm -qa --qf``,
libdnf's ``trans`` table (``state`` 0 = UNKNOWN until ``finish()`` sets DONE
1 or ERROR 2), yum's ``transaction-all.*`` journal that
``yum-complete-transaction`` reads -- and still have to be checked against a
real Oracle Linux 9 host.
``yum-complete-transaction`` reads. Checked against Oracle Linux 9 (build-arm-01,
2026-11-12): its UEK kernels are installonly only by libdnf's built-in defaults.
"""
INSTALLONLY = ("kernel-core", "kernel-modules", "kernel-uek-core")
@@ -804,6 +822,29 @@ case "$*" in
esac
"""
#: An rpm on an Oracle Linux host: its UEK kernel ``vendor-kernel`` provides nothing
#: of dnf's upstream defaults and sits in the database three times, like glibc twice.
_RPM_VENDOR_KERNEL_STUB = """#!/bin/sh
case "$*" in
*--whatprovides*) for a; do [ "$a" = vendor-kernel ] && echo vendor-kernel; done
echo 'no package provides kernel-PAE' ;;
-qa*) printf '%s\\n' 'vendor-kernel aarch64' 'vendor-kernel aarch64' 'vendor-kernel aarch64' \\
'glibc aarch64' 'glibc aarch64' ;;
esac
"""
#: libdnf's Python binding, as far as the command asks it: the installonlypkgs a
#: distribution built into its libdnf -- here one more than upstream's.
_LIBDNF_CONF = """class _Option:
def getValueString(self):
return "kernel, installonlypkg(kernel), vendor-kernel"
class ConfigMain:
def installonlypkgs(self):
return _Option()
"""
#: /lib/apk/db/installed shaped after apk-tools' database.c: busybox's script failed.
_APK_DB = """C:Q1abc=
P:alpine-baselayout
@@ -886,6 +927,23 @@ class TestThePackageManagerCommandOnARealShell:
assert status["package_manager"] == "dnf"
assert status["interrupted_packages"] == [{"name": "glibc", "state": "duplicate"}]
def test_the_installonly_packages_libdnf_was_built_with(self, tmp_path, shell):
"""Oracle builds kernel-uek into libdnf's defaults; its UEK packages provide
nothing of upstream's, so only libdnf itself tells (#107)."""
_skip_busybox_rpm(shell)
env = _stubs(tmp_path, "rpm", "dnf")
(tmp_path / "bin" / "rpm").write_text(_RPM_VENDOR_KERNEL_STUB)
libdnf = tmp_path / "py" / "libdnf"
libdnf.mkdir(parents=True)
(libdnf / "__init__.py").write_text("")
(libdnf / "conf.py").write_text(_LIBDNF_CONF)
env["PYTHONPATH"] = str(tmp_path / "py")
out = _run(PACKAGE_MANAGERS_COMMAND, env, shell)
status = parse_host_status(out)
assert status["interrupted_packages"] == [{"name": "glibc", "state": "duplicate"}]
def test_dnfs_last_transaction_that_never_ended(self, tmp_path, shell):
_skip_busybox_rpm(shell)
history = tmp_path / "history.sqlite"
+31
View File
@@ -37,9 +37,40 @@ class TestRules:
"selector": "*.*",
"complete": True,
"ruleset": None,
"template": "RSYSLOG_SyslogProtocol23Format",
},
]
def test_a_rule_without_a_template_sends_rsyslogs_default(self):
"""None: RFC 3164, the sender's local time without a zone (NetOrk/netork#493)."""
(rule,) = _rules("*.* @graylog.example.net:514\n")
assert rule["template"] is None
def test_an_actions_template_parameter(self):
(rule,) = _rules(
'action(type="omfwd" target="10.0.0.7" Template="RSYSLOG_ForwardFormat")\n'
)
assert rule["template"] == "RSYSLOG_ForwardFormat"
def test_the_legacy_default_template_holds_for_the_rules_after_it(self):
rules = _rules(
"*.* @a\n$ActionForwardDefaultTemplate RSYSLOG_SyslogProtocol23Format\n"
"*.* @@b:514\n*.* @c;RSYSLOG_TraditionalForwardFormat\n"
)
assert [(r["target"], r["template"]) for r in rules] == [
("a", None),
("b", "RSYSLOG_SyslogProtocol23Format"),
("c", "RSYSLOG_TraditionalForwardFormat"),
]
def test_the_forward_modules_template_holds_for_its_actions(self):
text = (
'module(load="builtin:omfwd" template="RSYSLOG_SyslogProtocol23Format")\n'
'action(type="omfwd" target="10.0.0.7")\n'
)
(rule,) = _rules(text)
assert rule["template"] == "RSYSLOG_SyslogProtocol23Format"
def test_two_ats_are_tcp_and_a_missing_port_stays_unknown(self):
(rule,) = _rules("*.* @@logs.lan\n")
assert (rule["protocol"], rule["target"], rule["port"]) == ("tcp", "logs.lan", None)
+74
View File
@@ -18,9 +18,11 @@ import pytest
from napalm_device_types.package_updates import (
APT_UPGRADABLE_COMMAND,
DNF_INSTALLED_CVES_COMMAND,
DNF_SECURITY_COMMAND,
nevra_name,
parse_apt_upgradable,
parse_dnf_installed_cves,
parse_dnf_security,
)
@@ -198,3 +200,75 @@ __DNF_RC=0
)
def test_the_name_of_a_nevra(self, nevra, name):
assert nevra_name(nevra) == name
class TestDnfInstalledCves:
"""The CVEs the host's installed advisories close, per package (netOrk #2394).
Where netOrk's knowledge base holds no records of a release, the host's own
advisories are the distribution's word that a backport is in. Lines from real
hosts: Oracle Linux 9.8 (dnf 4.14, aarch64) and Amazon Linux 2023 (dnf 4.14).
"""
ORACLE = """\
CVE-2024-6501 Low/Sec. NetworkManager-1:1.48.10-2.0.1.el9_5.aarch64
CVE-2024-3661 Moderate/Sec. NetworkManager-1:1.48.10-5.0.1.el9_5.aarch64
CVE-2023-38545 Important/Sec. curl-7.76.1-23.el9_2.4.aarch64
CVE-2023-38545 Important/Sec. curl-7.76.1-26.el9_3.2.aarch64
CVE-2023-4911 Important/Sec. glibc-common-2.34-60.0.3.el9.aarch64
CVE-2023-4911 Important/Sec. glibc-common-2.34-60.0.3.el9_2.7.aarch64
__DNF_RC=0
"""
AMAZON = """\
CVE-2022-3715 Medium/Sec. bash-5.2.15-1.amzn2023.0.2.x86_64
CVE-2022-23491 Important/Sec. ca-certificates-2021.2.50-1.0.amzn2023.0.6.noarch
CVE-2023-32803 Important/Sec. ca-certificates-2023.2.60-1.0.amzn2023.0.2.noarch
__DNF_RC=0
"""
def test_cves_per_package_once_each_and_sorted(self):
assert parse_dnf_installed_cves(self.ORACLE) == {
"NetworkManager": ["CVE-2024-3661", "CVE-2024-6501"],
"curl": ["CVE-2023-38545"],
"glibc-common": ["CVE-2023-4911"],
}
def test_amazon_linux(self):
assert parse_dnf_installed_cves(self.AMAZON) == {
"bash": ["CVE-2022-3715"],
"ca-certificates": ["CVE-2022-23491", "CVE-2023-32803"],
}
def test_lines_that_name_no_cve_are_left_out(self):
"""A GPG prompt a repository without its key prints, an advisory id."""
out = (
"Importing GPG key 0x6E93CD0C:\n"
' Userid : "https://packagecloud.io/crowdsec/crowdsec"\n'
"ELSA-2023-5453 Important/Sec. glibc-2.34-60.0.3.el9_2.7.aarch64\n"
"__DNF_RC=0\n"
)
assert parse_dnf_installed_cves(out) == {}
def test_a_failed_dnf_raises_instead_of_reading_as_no_advisories(self):
with pytest.raises(ValueError, match="exit status 1"):
parse_dnf_installed_cves("__DNF_RC=1\n")
def test_output_cut_short_raises(self):
with pytest.raises(ValueError, match="no exit status"):
parse_dnf_installed_cves(self.ORACLE.replace("__DNF_RC=0\n", ""))
def test_the_command_lists_installed_advisories_with_their_cves_from_the_cache(self, tmp_path):
args = tmp_path / "args"
dnf = tmp_path / "dnf"
dnf.write_text(f"#!/bin/sh\nprintf '%s\\n' \"$@\" > {args}\n")
dnf.chmod(0o755)
env = {"PATH": f"{tmp_path}:{os.environ.get('PATH', '/usr/bin:/bin')}"}
out = subprocess.run(
["sh", "-c", DNF_INSTALLED_CVES_COMMAND], capture_output=True, text=True, env=env
).stdout
assert {"--cacheonly", "updateinfo", "list", "--installed", "--with-cve"} <= set(
args.read_text().split()
)
assert out == "__DNF_RC=0\n"
+49
View File
@@ -299,3 +299,52 @@ def test_an_account_that_cannot_log_in_is_another_services_and_left_alone(
driver.add_service_account("netork", NETORK_KEY)
assert not _root_commands(host)
assert service_sudoers_rule_path(_Driver.SUDOERS_DIR, "netork") not in host.files
OLD_KEY = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOldNetork netork@central"
def test_a_replaced_key_goes_once_the_new_one_is_in(host: _ServiceHost, driver: _Driver) -> None:
host.login = "netork"
host.accounts["netork"] = _Account(900, "/home/netork", groups=["netork"])
host.files["/home/netork/.ssh/authorized_keys"] = f"{OWN_KEY}\n{OLD_KEY}\n"
result = driver.add_service_account("netork", NETORK_KEY, replaces=[OLD_KEY])
assert result["success"], result["output"]
assert [s["step"] for s in result["steps"]] == ["account", "sudo", "keys", "old keys"]
assert host.files["/home/netork/.ssh/authorized_keys"] == f"{OWN_KEY}\n{NETORK_KEY}\n"
def test_the_new_key_is_never_one_that_goes(host: _ServiceHost, driver: _Driver) -> None:
host.login = "netork"
host.accounts["netork"] = _Account(900, "/home/netork", groups=["netork"])
host.files["/home/netork/.ssh/authorized_keys"] = f"{NETORK_KEY}\n"
result = driver.add_service_account(
"netork", NETORK_KEY, replaces=[f"{NETORK_KEY.split()[0]} {NETORK_KEY.split()[1]}"]
)
assert result["success"], result["output"]
assert [s["step"] for s in result["steps"]] == ["account", "sudo", "keys"]
assert host.files["/home/netork/.ssh/authorized_keys"] == f"{NETORK_KEY}\n"
def test_a_replaced_key_stays_while_the_new_one_is_not_in(
host: _ServiceHost, driver: _Driver
) -> None:
host.login = "netork"
host.accounts["netork"] = _Account(900, "/home/netork", groups=["netork"])
host.files["/home/netork/.ssh/authorized_keys"] = f"{OLD_KEY}\n"
host.failing.add("as install")
result = driver.add_service_account("netork", NETORK_KEY, replaces=[OLD_KEY])
assert not result["success"]
assert "old keys" not in [s["step"] for s in result["steps"]]
assert host.files["/home/netork/.ssh/authorized_keys"] == f"{OLD_KEY}\n"
def test_replaced_keys_are_a_list_not_one_string(driver: _Driver) -> None:
with pytest.raises(ValueError, match="list"):
driver.add_service_account("netork", NETORK_KEY, replaces=OLD_KEY)