refactor(config): take the revision contract from napalm-device-types
driver / standard (pull_request) Successful in 17s
driver / lint (pull_request) Successful in 15s
driver / types (pull_request) Successful in 1m0s
driver / test (3.12) (pull_request) Successful in 1m11s
driver / test (3.13) (pull_request) Successful in 1m3s
driver / lowest (pull_request) Successful in 35s
driver / build (pull_request) Successful in 29s
driver / security (pull_request) Successful in 1m1s
CI / driver (pull_request) Successful in 2m44s
driver / publish (pull_request) Skipped

napalm-device-types 4.38.0 declares ConfigRevisionsMixin: the listing,
the reading, ConfigRevisionDict and the exceptions, and the revert as a
template. The driver keeps the device communication: the three backup
API calls, the revert step, and config_revision_content(), which leaves
out the <revision> block every save writes into config.xml and the
indentation between elements.

The template reads the newest revision after the revert and checks that
it holds the revision's configuration. OPNsense answers "reverted" even
when restoreBackup() could not load the file and kept the previous one;
that now raises ConfigRevertFailed instead of passing. A refusal of the
firewall raises ConfigRevertFailed too. An id the history does not list
still never reaches a request path.

Refs NetOrk/netork#2511
This commit is contained in:
Christian Manivong committed 2026-11-13 22:33:58 +01:00
1 parent 5a20023c54
commit a65a3ed6e3
5 files changed
+174 -62

No files matched your search

+8 -2
View File
@@ -11,8 +11,12 @@ All notable changes to this driver. The format follows
lists the revisions OPNsense keeps of config.xml, newest first, each with its id, its
time in UTC, its description, its user and its size; `get_config_revision(id)`
returns one revision's config.xml; `revert_config_revision(id)` makes it the
configuration again through `/api/core/backup/revert_backup` and refuses an id that
is not in the history (`UnknownConfigRevision`). A revert saves config.xml and
configuration again through `/api/core/backup/revert_backup`, refuses an id that is
not in the history (`UnknownConfigRevision`), and raises `ConfigRevertFailed` when the
newest revision afterwards does not hold the revision's configuration -- OPNsense
answers "reverted" even when it kept the previous file. `config_revision_content()`
compares config.xml without the `<revision>` block every save writes. The contract is
napalm-device-types 4.38.0's `ConfigRevisionsMixin`. A revert saves config.xml and
nothing else: no service is reloaded and the firewall does not reboot.
- Time sync (NetOrk/netork#2363): `get_time_sync()` from napalm-device-types 4.35.0's
`TimeSyncMixin` reads the time servers (`<system><timeservers>`, in their order) and
@@ -26,6 +30,8 @@ All notable changes to this driver. The format follows
### Changed
- Requires napalm-device-types 4.35.0, which declares the time-sync contract.
- Requires napalm-device-types 4.38.0, which declares the configuration-history
contract.
- `load_replace_candidate` is `None` instead of NAPALM's stub that raised, so a caller
can tell that the driver replaces no configuration: OPNsense has no API that takes an
uploaded config.xml.
+13 -5
View File
@@ -16,7 +16,7 @@ NAPALM community driver for **OPNsense** firewalls (read-only, via REST API).
|---|---|
| Python | 3.12 |
| NAPALM | 4.0 |
| napalm-device-types | 4.35.0 |
| napalm-device-types | 4.38.0 |
| requests | 2.28 |
## Installation
@@ -208,10 +208,18 @@ first being the configuration in place now:
]
```
`get_config_revision(revision_id)` returns that revision's config.xml as text.
`revert_config_revision(revision_id)` makes it the configuration again; it refuses an
id that is not in the history now (`UnknownConfigRevision`, a `ValueError`) and raises
`RuntimeError` when the firewall refuses, e.g. for a user with `user-config-readonly`.
The contract is napalm-device-types' `ConfigRevisionsMixin`; this driver does the API
calls. `get_config_revision(revision_id)` returns that revision's config.xml as text.
`revert_config_revision(revision_id)` makes it the configuration again and returns the
revision in place afterwards. It refuses an id that is not in the history now
(`UnknownConfigRevision`, a `ValueError`) before anything changes, reads the revision,
calls `revert_backup`, and then checks that the newest revision holds the revision's
configuration: OPNsense answers `reverted` even when it could not load the file and kept
the previous one. A refusal of the firewall (for a user with `user-config-readonly`) or
a failed check raises `ConfigRevertFailed`, a `RuntimeError`. That is five requests.
`config_revision_content(text)` is config.xml without the `<revision>` block every save
writes (who, when, from where), in canonical XML, with the indentation between elements
left out: two revisions with the same configuration compare equal through it.
**What a revert does.** OPNsense copies the revision over config.xml and saves it. The
history gets a new newest entry with the revision's content, described as
+41 -42
View File
@@ -1,5 +1,11 @@
"""The firewall's own configuration history (NetOrk/netork#2511).
napalm-device-types' ``ConfigRevisionsMixin`` is the contract: it declares the listing
and the reading and runs the revert as a template -- refuse an id that is not in the
history, read the revision, revert, check that the newest revision holds it. This
module talks to the firewall: the three API calls, and ``config_revision_content``,
which leaves out the ``<revision>`` block every save writes into config.xml.
OPNsense keeps a copy of config.xml for every save, as ``/conf/backup/config-<unix
time>.xml`` -- *System → Configuration → History*. It keeps ``<system><backupcount>``
of them (100 when unset) and drops the oldest on the next save. The newest is the
@@ -26,8 +32,19 @@ from __future__ import annotations
import logging
import re
# config.xml as the firewall's own API hands it over; ElementTree resolves no external
# entities, and expat limits entity expansion since 2.4.1.
import xml.etree.ElementTree as ET # nosec B405
from datetime import UTC, datetime
from typing import TYPE_CHECKING, Any, TypedDict
from typing import TYPE_CHECKING, Any
from napalm_device_types import (
ConfigRevertFailed,
ConfigRevisionDict,
ConfigRevisionsMixin,
UnknownConfigRevision,
)
logger = logging.getLogger(__name__)
@@ -40,28 +57,13 @@ REVERT_PATH = "/api/core/backup/revert_backup/{revision_id}"
#: What ``revert_backup`` answers for a revision it has no file for.
_NOT_FOUND = "not_found"
_REVERTED = "reverted"
#: What every save writes into config.xml anew: who saved it, when, from where.
_STAMP = "revision"
#: A revision's file name in /conf/backup: its unix time, and ``_<ns>`` after a save
#: within the same microsecond. Nothing else may reach a request path.
_REVISION_ID = re.compile(r"config-([0-9]+(?:\.[0-9]+)?)(?:_[0-9]+)?\.xml")
class ConfigRevisionDict(TypedDict):
"""One revision of the firewall's configuration (``get_config_revisions``)."""
id: str # what get_config_revision and revert_config_revision take
time: str | None # when it was saved, UTC ISO 8601; None when the firewall does not say
description: str | None # what made the change, as the firewall records it; None: none
user: str | None # who saved it ("root@192.0.2.20"); None when the firewall names nobody
size: int | None # the file's size in bytes; None when not given
class UnknownConfigRevision(ValueError):
"""The firewall's configuration history has no revision of that id."""
def __init__(self, revision_id: object) -> None:
super().__init__(f"no configuration revision {revision_id!r} in the firewall's history")
def _checked_id(revision_id: object) -> str:
if not isinstance(revision_id, str) or not _REVISION_ID.fullmatch(revision_id):
raise UnknownConfigRevision(revision_id)
@@ -121,7 +123,7 @@ def config_revisions(answer: Any) -> list[ConfigRevisionDict]:
return revisions
class OPNsenseConfigRevisionsMixin:
class OPNsenseConfigRevisionsMixin(ConfigRevisionsMixin):
"""List, read and revert to the firewall's configuration revisions; see the module."""
#: No API takes an uploaded config.xml (NAPALM's base stub would only raise).
@@ -138,18 +140,6 @@ class OPNsenseConfigRevisionsMixin:
def get_config_revisions(self) -> list[ConfigRevisionDict]:
"""The firewall's configuration history, newest first; the first is in place now.
Example::
[
{
"id": "config-1794603885.1239.xml",
"time": "2026-11-13T21:04:45+00:00",
"description": "/api/firewall/filter/apply made changes",
"user": "apiuser@192.0.2.10",
"size": 58213,
},
]
:raises Exception: when the history cannot be read, e.g. by a key without the
backup privilege.
"""
@@ -167,27 +157,36 @@ class OPNsenseConfigRevisionsMixin:
raise UnknownConfigRevision(checked)
return text
def revert_config_revision(self, revision_id: str) -> None:
"""Make revision *revision_id* the firewall's configuration (``revert_backup``).
def config_revision_content(self, text: str) -> str:
"""config.xml without the root's ``<revision>`` block, canonical (XML C14N).
The revision is saved as config.xml and as the history's new newest entry;
no service is reloaded and the firewall does not reboot (see the module).
The indentation between elements does not count; a value's own blanks do.
:raises UnknownConfigRevision: for an id that is not in the history now.
:raises RuntimeError: when the firewall refuses the revert, e.g. for a key
that may not change the configuration.
:raises Exception: when the firewall cannot be reached.
:raises xml.etree.ElementTree.ParseError: for a text that is no XML.
"""
# The firewall's own config.xml: see the import.
root = ET.fromstring(text) # nosec B314
for stamp in root.findall(_STAMP):
root.remove(stamp)
for element in root.iter():
if element.text is not None and not element.text.strip():
element.text = None
if element.tail is not None and not element.tail.strip():
element.tail = None
return ET.canonicalize(ET.tostring(root, encoding="unicode"))
def _revert_to_config_revision(self, revision_id: str) -> None:
"""``revert_backup``: config.xml becomes the revision, saved as the newest one."""
checked = _checked_id(revision_id)
if checked not in {r["id"] for r in self.get_config_revisions()}:
raise UnknownConfigRevision(checked)
answer = self._post(REVERT_PATH.format(revision_id=checked))
status = answer.get("status") if isinstance(answer, dict) else None
if status == _NOT_FOUND:
raise UnknownConfigRevision(checked)
if status != _REVERTED:
reason = answer.get("errorMessage") if isinstance(answer, dict) else None
raise RuntimeError(f"The firewall did not revert to {checked}: {reason or status}")
raise ConfigRevertFailed(
f"The firewall did not revert to {checked}: {reason or status}"
)
def _get_latest_backup_id(self) -> str | None:
"""Return the ID of the most recent server-side config backup, or ``None``.
+1 -1
View File
@@ -23,7 +23,7 @@ classifiers = [
]
dependencies = [
"napalm>=5.1", # older releases import pkg_resources, which a 3.12 venv lacks
"napalm-device-types>=4.35.0",
"napalm-device-types>=4.38.0",
"requests>=2.28.0",
]
+111 -12
View File
@@ -14,14 +14,16 @@ first configuration of a new install is.
from __future__ import annotations
import copy
import json
from pathlib import Path
from unittest.mock import MagicMock, patch
from xml.etree.ElementTree import ParseError
import pytest
import requests
from napalm_device_types import ConfigRevertFailed, ConfigRevisionsMixin, UnknownConfigRevision
from napalm_opnsense.config_revisions import UnknownConfigRevision
from napalm_opnsense.opnsense import OPNsenseDriver
FIXTURES = Path(__file__).parent / "fixtures"
@@ -32,15 +34,50 @@ HISTORY_PATH = "/api/core/backup/backups/this"
NEWEST = "config-1794603885.1239.xml"
OLDEST = "config-1793516400.0012.xml"
WITH_SUFFIX = "config-1794475810.0548_123456789.xml"
#: The id of the revision a revert saves, as the fake firewall names it.
REVERTED = "config-1794700000.0001.xml"
def _stamped(config: str, user: str, time: str) -> str:
"""*config* with the <revision> block every save writes into config.xml."""
block = (
f" <revision>\n <username>{user}</username>\n <time>{time}</time>\n"
" <description>/api/core/backup/revert_backup made changes</description>\n"
" </revision>\n"
)
return config.replace("</opnsense>", block + "</opnsense>")
#: The oldest revision is the sample; every later one names the host otherwise.
FILES = {
OLDEST: _stamped(CONFIG_XML, "root@192.0.2.20", "1793516400.00"),
WITH_SUFFIX: _stamped(CONFIG_XML, "apiuser@192.0.2.10", "1794475810.05"),
"config-1794603482.8721.xml": _stamped(
CONFIG_XML.replace("<hostname>OPNsense</hostname>", "<hostname>gw</hostname>"),
"root@192.0.2.20",
"1794603482.87",
),
NEWEST: _stamped(
CONFIG_XML.replace("<hostname>OPNsense</hostname>", "<hostname>gw1</hostname>"),
"apiuser@192.0.2.10",
"1794603885.12",
),
}
class Firewall:
"""Answers as the firewall's backup API would; every request is recorded."""
"""Answers as the firewall's backup API would; every request is recorded.
def __init__(self, history=HISTORY, files=None, revert_answer=None):
self.history = history
self.files = files if files is not None else {OLDEST: CONFIG_XML}
A revert it takes saves the revision as a new newest one, restamped; one it does
not take (``takes=False``) still answers "reverted", as OPNsense does when it falls
back to the previous file.
"""
def __init__(self, history=HISTORY, files=None, revert_answer=None, takes=True):
self.history = copy.deepcopy(history)
self.files = dict(FILES if files is None else files)
self.revert_answer = revert_answer or {"status": "reverted"}
self.takes = takes
self.requests: list[tuple[str, str]] = []
def get(self, url, timeout=None):
@@ -68,9 +105,17 @@ class Firewall:
response.raise_for_status.return_value = None
if isinstance(self.revert_answer, Exception):
response.raise_for_status.side_effect = self.revert_answer
elif self.revert_answer.get("status") == "reverted" and self.takes:
self._save(path.rsplit("/", 1)[1])
response.json.return_value = self.revert_answer
return response
def _save(self, revision_id):
body = self.files[revision_id].split(" <revision>")[0] + "</opnsense>\n"
self.files[REVERTED] = _stamped(body, "apiuser@192.0.2.10", "1794700000.00")
item = {"time": "1794700000.00", "description": "", "username": "", "id": REVERTED}
self.history["items"].insert(0, item)
@property
def posts(self) -> list[str]:
return [path for method, path in self.requests if method == "POST"]
@@ -205,7 +250,7 @@ class TestGetConfigRevision:
def test_the_revisions_config_xml(self, driver):
firewall = _on(driver, Firewall())
assert driver.get_config_revision(OLDEST) == CONFIG_XML
assert driver.get_config_revision(OLDEST) == FILES[OLDEST]
assert firewall.requests == [("GET", f"/api/core/backup/download/this/{OLDEST}")]
def test_a_revision_the_firewall_has_no_file_for_raises(self, driver):
@@ -237,14 +282,18 @@ class TestGetConfigRevision:
class TestRevertConfigRevision:
def test_reverts_through_the_backup_api(self, driver):
def test_reverts_through_the_backup_api_and_checks_the_result(self, driver):
firewall = _on(driver, Firewall())
driver.revert_config_revision(OLDEST)
now = driver.revert_config_revision(OLDEST)
assert now["id"] == REVERTED
assert firewall.requests == [
("GET", HISTORY_PATH),
("GET", f"/api/core/backup/download/this/{OLDEST}"),
("POST", f"/api/core/backup/revert_backup/{OLDEST}"),
("GET", HISTORY_PATH),
("GET", f"/api/core/backup/download/this/{REVERTED}"),
]
def test_an_id_with_the_suffix_of_a_fast_save(self, driver):
@@ -254,6 +303,13 @@ class TestRevertConfigRevision:
assert firewall.posts == [f"/api/core/backup/revert_backup/{WITH_SUFFIX}"]
def test_a_revert_the_firewall_did_not_take_fails(self, driver):
# restoreBackup() falls back to the previous file and the answer is "reverted".
_on(driver, Firewall(takes=False))
with pytest.raises(ConfigRevertFailed, match=OLDEST):
driver.revert_config_revision(OLDEST)
def test_a_revision_not_in_the_history_is_refused(self, driver):
firewall = _on(driver, Firewall())
@@ -263,13 +319,13 @@ class TestRevertConfigRevision:
assert firewall.posts == []
@pytest.mark.parametrize("revision_id", ["", "../config.xml", "config-1.xml/x", None])
def test_an_id_that_names_no_revision_is_refused_before_any_request(self, driver, revision_id):
def test_an_id_that_names_no_revision_never_reaches_a_request_path(self, driver, revision_id):
firewall = _on(driver, Firewall())
with pytest.raises(UnknownConfigRevision):
driver.revert_config_revision(revision_id)
assert firewall.requests == []
assert firewall.requests == [("GET", HISTORY_PATH)]
def test_a_history_that_cannot_be_read_reverts_nothing(self, driver):
firewall = _on(driver, Firewall(history=requests.HTTPError("403 Forbidden")))
@@ -290,13 +346,13 @@ class TestRevertConfigRevision:
reason = "User apiuser denied for write access (user-config-readonly set)"
_on(driver, Firewall(revert_answer={"errorMessage": reason, "errorTitle": "Error"}))
with pytest.raises(RuntimeError, match="user-config-readonly"):
with pytest.raises(ConfigRevertFailed, match="user-config-readonly"):
driver.revert_config_revision(OLDEST)
def test_any_other_answer_is_a_failed_revert(self, driver):
_on(driver, Firewall(revert_answer={"status": "failed"}))
with pytest.raises(RuntimeError, match="failed"):
with pytest.raises(ConfigRevertFailed, match="failed"):
driver.revert_config_revision(OLDEST)
def test_an_http_error_raises(self, driver):
@@ -306,7 +362,50 @@ class TestRevertConfigRevision:
driver.revert_config_revision(OLDEST)
class TestConfigRevisionContent:
def test_the_revision_block_does_not_count(self, driver):
# Same configuration, saved by someone else at another time.
assert driver.config_revision_content(FILES[OLDEST]) == driver.config_revision_content(
FILES[WITH_SUFFIX]
)
def test_a_changed_setting_counts(self, driver):
assert driver.config_revision_content(FILES[OLDEST]) != driver.config_revision_content(
FILES[NEWEST]
)
def test_a_configuration_without_a_revision_block(self, driver):
stamped = driver.config_revision_content(FILES[OLDEST])
assert driver.config_revision_content(CONFIG_XML) == stamped
def test_a_revision_element_deeper_down_counts(self, driver):
# Only the root's <revision> is the save's stamp.
deeper = CONFIG_XML.replace("<theme>", "<theme><revision>7</revision>", 1)
assert driver.config_revision_content(deeper) != driver.config_revision_content(CONFIG_XML)
def test_how_the_xml_is_written_does_not_count(self, driver):
# Attribute order, quotes and empty elements are no difference of the configuration.
a = '<?xml version="1.0"?>\n<opnsense><x a="1" b="2"></x></opnsense>'
b = "<opnsense><x b='2' a='1'/></opnsense>"
assert driver.config_revision_content(a) == driver.config_revision_content(b)
def test_the_blanks_of_a_value_count(self, driver):
padded = CONFIG_XML.replace("<theme>opnsense</theme>", "<theme>opnsense </theme>")
assert driver.config_revision_content(padded) != driver.config_revision_content(CONFIG_XML)
def test_text_that_is_no_xml_raises(self, driver):
with pytest.raises(ParseError):
driver.config_revision_content("<html><body>Login")
class TestWhatTheDriverOffers:
def test_it_keeps_the_contract_of_device_types(self):
assert issubclass(OPNsenseDriver, ConfigRevisionsMixin)
def test_it_reads_and_reverts_revisions(self):
for name in ("get_config_revisions", "get_config_revision", "revert_config_revision"):
assert callable(getattr(OPNsenseDriver, name, None)), name