feat(scheduled-jobs): the cron jobs of config.xml over the REST API (#43) #44

Merged
christianmanivong merged 2 commits from feature/scheduled-jobs into main 2026-11-03 08:20:27 +00:00
8 changed files with 1091 additions and 5 deletions

No files matched your search

+11
View File
@@ -6,7 +6,18 @@ All notable changes to this driver. The format follows
## [Unreleased]
### Added
- Scheduled jobs (NetOrk/netork#1933, #43): `get_scheduled_jobs()` reads every cron job
of config.xml over `/api/cron/settings/get`, with who owns it -- another component's
job is `platform`, one written by `put_scheduled_job()` `application`, the rest
`admin`. `manage_scheduled_job()` enables and disables any of them;
`put_scheduled_job()` and `delete_scheduled_job()` write and remove an application's
own, whose command is a configd action. `get_scheduled_job_commands()` lists the
actions configd offers. Running a job now is refused: the API cannot run a configd
action. Every write reads the job again first and refuses one that changed.
### Changed
- Requires napalm-device-types 4.26.0, which declares the scheduled-jobs contract.
- Python 3.12 is the floor (`requires-python = ">=3.12"`): netOrk runs every driver
on 3.12, so 3.10 and 3.11 are no longer tested (NAPALM/driver-standard#6).
- The floor on napalm is 5.1: older releases import `pkg_resources`, which a Python
+29 -1
View File
@@ -16,7 +16,7 @@ NAPALM community driver for **OPNsense** firewalls (read-only, via REST API).
|---|---|
| Python | 3.12 |
| NAPALM | 4.0 |
| napalm-device-types | 4.21.1 |
| napalm-device-types | 4.26.0 |
| requests | 2.28 |
## Installation
@@ -92,10 +92,15 @@ arguments.
| `get_listening_sockets` | ✅ | `GET /api/diagnostics/interface/get_socket_statistics` + `/api/core/service/search` + `/api/wireguard/service/show` (WireGuard's kernel sockets by listen port) |
| `get_virtual_ips` | ✅ | `GET /api/interfaces/overview/export`: every address bound to a CARP VHID (CARP address and IP aliases on it), with this node's CARP state. `interfaces/vip_settings` is not read: it carries the CARP passwords |
| `ping_sweep` | ✅ ³ | same endpoints, one batch of parallel jobs at a time |
| `get_scheduled_jobs()` | ✅ ⁴ | `GET /api/cron/settings/get`: every cron job of config.xml |
| `manage_scheduled_job(key, action, digest=)` | ✅ ⁴ | enable, disable: `POST /api/cron/settings/toggle_job/<uuid>/1` (or `/0`) + `/api/cron/service/reconfigure`; run now is refused |
| `put_scheduled_job(spec)` / `delete_scheduled_job(key, digest=)` | ✅ ⁴ | an application's own job: `add_job`, `set_job/<uuid>`, `del_job/<uuid>` + `reconfigure` |
| `get_scheduled_job_commands()` | ✅ ⁴ | `GET /api/cron/settings/get_job`: the configd actions a job can run |
> ¹ Requires the `os-lldpd` plugin. Returns empty dict if the plugin is not installed.
> ² Requires the `os-frr` (FRR/Quagga) plugin. Returns empty dict if the plugin is not installed or FRR is not running.
> ³ Overrides the generic per-host loop from `napalm-device-types`.
> ⁴ See [Scheduled jobs](#scheduled-jobs).
### Ping
@@ -120,6 +125,29 @@ statistics line lands in the log slightly after the request that triggered it.
`ttl` and `vrf` are accepted for NAPALM compatibility and ignored — the API
has no equivalent.
### Scheduled jobs
OPNsense keeps its cron jobs in config.xml (System → Settings → Cron), and each
runs a **configd action**, not a shell line. `key` is `opnsense:<uuid>`, `kind`
`cron`, `source` `config.xml`, `run_as` `root`. A job another component
registered (its origin is not `cron`, such as the intrusion detection's rule
update) is `platform`; one whose description starts with
`napalm-scheduled-job: <name>` was written by `put_scheduled_job` and is the only
kind that can be edited or deleted; every other job is `admin`. Every job can be
enabled and disabled.
`put_scheduled_job` takes a configd action as `command`, with its parameters as
the words after it: the actions are what `get_scheduled_job_commands()` lists,
anything else is refused (`ScheduledJobRefused`), and so is `@reboot` and a
parameter with blanks, quotes or backslashes, which OPNsense releases before and
after 26.7 would split differently. The schedule is written as its five fields,
month and weekday names as numbers.
Not covered: the API cannot run a configd action now, so no job takes `run`;
what OPNsense writes into the crontab beyond config.xml (`periodic`, the
plugins' system jobs) cannot be read over the API; the time zone is not read
(`None`).
## Config management
OPNsense does not expose a single generic "push config" endpoint. Config is
+3 -3
View File
@@ -74,6 +74,7 @@ from napalm_opnsense.ospf_mixin import OPNsenseOspfMixin
from napalm_opnsense.ping_mixin import OPNsensePingMixin
from napalm_opnsense.port_forwards import alias_index, port_forwards, wan_interfaces
from napalm_opnsense.resolver_mixin import OPNsenseResolverMixin
from napalm_opnsense.scheduled_jobs import OPNsenseScheduledJobsMixin
from napalm_opnsense.virtual_ips import carp_addresses
from napalm_opnsense.wireguard import wireguard_tunnels
@@ -101,6 +102,7 @@ def _carp_master(entry: dict[str, Any]) -> bool:
class OPNsenseDriver(
OPNsenseScheduledJobsMixin,
OPNsensePingMixin,
ConfigSyncMixin,
OPNsenseResolverMixin,
@@ -1991,9 +1993,7 @@ class OPNsenseDriver(
``enable`` and ``disable`` are not supported on OPNsense (plugins
are enabled/disabled by installing or removing them).
"""
import re as _re
if not _re.match(r"^[a-zA-Z0-9_\-]+$", name):
if not re.match(r"^[a-zA-Z0-9_\-]+$", name):
raise ValueError(f"Invalid service name: {name!r}")
if action not in ("start", "stop", "restart"):
return {
+314
View File
@@ -0,0 +1,314 @@
"""Scheduled jobs on OPNsense: the cron jobs of its configuration (napalm-device-types).
OPNsense keeps its cron jobs in config.xml (System > Settings > Cron) and writes
them into the crontab itself; each job runs one **configd action** with optional
parameters, never a shell line. The REST API reads and writes them:
``/api/cron/settings/get`` answers with every job at once, the action as the
option its ``command`` list has selected; ``get_job`` without a uuid is a blank
job whose ``command`` options are the actions configd offers; ``add_job``,
``set_job``, ``del_job`` and ``toggle_job`` write, ``/api/cron/service/reconfigure``
regenerates the crontab. OPNsense routes ``search_jobs`` and ``searchJobs`` to the
same action; the snake-case spelling is the documented one.
**Who owns a job.** A job another component registered (its ``origin`` is not
``cron``: the intrusion detection's rule update, say) is ``platform``. A job whose
description starts with ``napalm-scheduled-job: <name>`` was written through
:meth:`OPNsenseScheduledJobsMixin.put_scheduled_job` (``application``); every other
one an admin made (``admin``). Every job can be enabled and disabled; only an
application's job can be edited or deleted.
**Not here.** The API cannot run a configd action now, so no job takes ``run``.
What OPNsense puts in the crontab beyond config.xml -- ``periodic``, the plugins'
own system jobs -- is not reachable over the API and is not read. The time zone
is not read either (None).
**Parameters, one per line.** Up to 26.1 OPNsense split the parameters the way a
shell does; 26.7 takes one argument per line. Arguments without blanks, quotes
or backslashes, one per line, mean the same to both, so those are all a job
written here takes.
"""
from __future__ import annotations
import re
from collections.abc import Iterable
from typing import TYPE_CHECKING, Any
from napalm_device_types.cron import ALIASES, is_reboot
from napalm_device_types.scheduled_jobs import (
ACTIONS,
DELETE,
EDIT,
ScheduledJobChanged,
ScheduledJobDict,
ScheduledJobRefused,
ScheduledJobResultDict,
ScheduledJobsMixin,
ScheduledJobSpecDict,
ScheduledJobsReadingDict,
check_job_name,
check_spec,
job_digest,
)
_SETTINGS = "/api/cron/settings"
_RECONFIGURE = "/api/cron/service/reconfigure"
_KEY = "opnsense:"
#: The origin of a job made in the cron settings themselves.
_OWN_ORIGIN = "cron"
#: How a job written by put_scheduled_job begins its description.
MARKER = "napalm-scheduled-job: "
_FIELDS = ("minutes", "hours", "days", "months", "weekdays")
_SWITCH = ["enable", "disable"]
_OWN = [*_SWITCH, EDIT, DELETE]
#: OPNsense's month and weekday fields take numbers (months also upper case names).
_MONTHS = ("jan", "feb", "mar", "apr", "may", "jun", "jul", "aug", "sep", "oct", "nov", "dec")
_WEEKDAYS = ("sun", "mon", "tue", "wed", "thu", "fri", "sat")
_NAME = re.compile(r"[A-Za-z]+")
#: A parameter every OPNsense release hands configd as one argument.
_PLAIN_ARGUMENT = re.compile(r"[^\s'\"\\]+")
def _selected(field: Any) -> str:
"""The option a list field has selected (none: ""); a plain value as it is."""
if not isinstance(field, dict):
return str(field or "")
chosen = (k for k, o in field.items() if isinstance(o, dict) and o.get("selected") in (1, "1"))
return next(chosen, "")
def _command(node: dict[str, Any]) -> str:
"""The configd action and its parameters, as one line."""
parameters = str(node.get("parameters") or "").splitlines()
return " ".join([_selected(node.get("command")), *(p.strip() for p in parameters if p.strip())])
def _schedule(uuid: str, node: dict[str, Any]) -> str:
fields = [node.get(field) for field in _FIELDS]
if not all(isinstance(f, str) and f.strip() for f in fields):
raise ValueError(f"cron job {uuid} came back without its schedule")
return " ".join(str(f).strip() for f in fields)
def _ownership(origin: str, description: str) -> tuple[str, str | None, str]:
"""(managed_by, the application's job name or None, the description to show)."""
if origin != _OWN_ORIGIN:
return "platform", None, description
if not description.startswith(MARKER):
return "admin", None, description
name, _, rest = description[len(MARKER) :].partition(" ")
try:
check_job_name(name)
except ScheduledJobRefused:
return "admin", None, description
return "application", name, rest.strip()
def scheduled_job(uuid: str, node: dict[str, Any]) -> ScheduledJobDict:
"""One job of the cron settings, as napalm-device-types reads it."""
origin = str(node.get("origin") or _OWN_ORIGIN)
managed_by, name, description = _ownership(origin, str(node.get("description") or ""))
job: dict[str, Any] = {
"key": f"{_KEY}{uuid}",
"kind": "cron",
"source": "config.xml",
# The crontab line calls configctl; configd runs the action as root.
"run_as": "root",
"schedule": _schedule(uuid, node),
"schedule_kind": "cron",
"command": _command(node),
"enabled": str(node.get("enabled")) == "1",
"managed_by": managed_by,
"actions": list(_OWN if name else _SWITCH),
}
if name:
job["name"] = name
if description:
job["description"] = description
job["digest"] = job_digest(job)
return job # type: ignore[return-value]
def scheduled_jobs(answer: Any) -> list[ScheduledJobDict]:
"""Every job in an answer of ``/api/cron/settings/get``.
:raises ValueError: when the answer has no list of jobs -- never "no jobs".
"""
try:
jobs = answer["job"]["jobs"]["job"]
except (KeyError, TypeError) as exc:
raise ValueError("the cron settings came back without their jobs") from exc
if jobs == []: # an empty ArrayField comes back as a JSON list
return []
if not isinstance(jobs, dict):
raise ValueError(f"the cron settings came back with jobs as {type(jobs).__name__}")
return [scheduled_job(uuid, node) for uuid, node in jobs.items()]
def _numbers(field: str, names: tuple[str, ...], low: int) -> str:
return _NAME.sub(lambda m: str(names.index(m.group().lower()) + low), field)
def cron_fields(schedule: str) -> dict[str, str]:
"""The five fields OPNsense stores of *schedule*, which :func:`check_spec` passed."""
if is_reboot(schedule):
raise ScheduledJobRefused("OPNsense's cron jobs have no @reboot")
text = schedule.strip()
fields = ALIASES.get(text.lower(), text).split()
fields[3] = _numbers(fields[3], _MONTHS, 1)
fields[4] = _numbers(fields[4], _WEEKDAYS, 0)
return dict(zip(_FIELDS, fields, strict=True))
def configd_action(command: str, actions: Iterable[str]) -> tuple[str, list[str]]:
"""(the configd action *command* names, its parameters).
The action is the longest one configd offers that *command* starts with; the
words after it are its parameters.
"""
if "\n" in command.strip():
raise ScheduledJobRefused("an OPNsense job runs one configd action, on one line")
words = command.split()
known = set(actions)
for end in range(len(words), 0, -1):
action = " ".join(words[:end])
if action in known:
parameters = words[end:]
for parameter in parameters:
if not _PLAIN_ARGUMENT.fullmatch(parameter):
raise ScheduledJobRefused(f"not a parameter OPNsense passes on: {parameter!r}")
return action, parameters
raise ScheduledJobRefused(
f"configd offers no action {command!r}; get_scheduled_job_commands() lists them"
)
def _saved(result: dict[str, Any], what: str) -> None:
if result.get("result") == "saved":
return
if result.get("validations"):
raise ScheduledJobRefused(f"OPNsense refused {what}: {result['validations']}")
raise RuntimeError(f"OPNsense did not save {what}: {result}")
class OPNsenseScheduledJobsMixin(ScheduledJobsMixin):
"""The cron jobs of config.xml; needs the driver's ``_get`` and ``_post``."""
if TYPE_CHECKING: # pragma: no cover - the driver's REST helpers
def _get(self, path: str) -> dict[str, Any]: ...
def _post(self, path: str, data: dict[str, Any] | None = None) -> dict[str, Any]: ...
def get_scheduled_job_commands(self) -> list[dict[str, str]]:
"""The configd actions a job can run: ``[{"command", "description"}]``, by command.
:raises ValueError: when the list came back empty -- configd did not answer.
"""
answer = self._get(f"{_SETTINGS}/get_job")
job = answer.get("job") if isinstance(answer, dict) else None
options = job.get("command") if isinstance(job, dict) else None
if not isinstance(options, dict) or not options:
raise ValueError("the cron settings came back without configd's actions")
return [
{"command": key, "description": str(option.get("value") or key)}
for key, option in sorted(options.items())
if key and isinstance(option, dict)
]
def get_scheduled_jobs(self) -> ScheduledJobsReadingDict:
"""Every cron job of config.xml, in one request; the time zone is not read.
:raises ValueError: when the answer came back without its jobs.
"""
return {"jobs": scheduled_jobs(self._get(f"{_SETTINGS}/get")), "time_zone": None}
def _sj_job(self, key: str, digest: str) -> ScheduledJobDict:
job = next((j for j in self.get_scheduled_jobs()["jobs"] if j["key"] == key), None)
if job is None:
raise LookupError(f"no scheduled job {key}")
if job["digest"] != digest:
raise ScheduledJobChanged(f"{key} changed on the host since it was read")
return job
def _sj_own(self, name: str, digest: str | None) -> ScheduledJobDict | None:
jobs = self.get_scheduled_jobs()["jobs"]
current = next((j for j in jobs if j.get("name") == name), None)
if digest is None:
if current is not None:
raise ScheduledJobChanged(f"a job {name} is there already")
return None
if current is None:
raise LookupError(f"no scheduled job named {name}")
if current["digest"] != digest:
raise ScheduledJobChanged(f"{current['key']} changed on the host since it was read")
return current
def _sj_reload(self) -> None:
result = self._post(_RECONFIGURE)
if result.get("status") != "ok":
raise RuntimeError(f"cron did not reload: {result}")
def manage_scheduled_job(self, key: str, action: str, *, digest: str) -> ScheduledJobResultDict:
"""Enable or disable the job *key*; no job runs now (the API cannot)."""
if action not in ACTIONS:
raise ScheduledJobRefused(f"unknown action {action!r}; allowed: {list(ACTIONS)}")
job = self._sj_job(key, digest)
if action not in job["actions"]:
raise ScheduledJobRefused(
f"{key} cannot {action}: OPNsense's API does not run a configd action now"
)
flag = "1" if action == "enable" else "0"
uuid = key.removeprefix(_KEY)
result = self._post(f"{_SETTINGS}/toggle_job/{uuid}/{flag}")
if result.get("result") not in ("Enabled", "Disabled"):
raise RuntimeError(f"OPNsense did not {action} {key}: {result}")
self._sj_reload()
return {"success": True, "output": f"{key} {action}d"}
def put_scheduled_job(
self, spec: ScheduledJobSpecDict, *, digest: str | None = None
) -> ScheduledJobResultDict:
"""Create or change the application's job ``spec["name"]``.
``command`` is a configd action (:meth:`get_scheduled_job_commands`), with
its parameters after it.
"""
check_spec(spec)
if spec["run_as"] != "root":
raise ScheduledJobRefused("an OPNsense job runs its configd action as root")
fields = cron_fields(spec["schedule"])
commands = (c["command"] for c in self.get_scheduled_job_commands())
action, parameters = configd_action(spec["command"], commands)
current = self._sj_own(spec["name"], digest)
job = {
"enabled": "1" if spec["enabled"] else "0",
**fields,
"command": action,
"parameters": "\n".join(parameters),
"description": f"{MARKER}{spec['name']} {spec.get('description') or ''}".rstrip(),
}
if current is None:
result = self._post(f"{_SETTINGS}/add_job", {"job": job})
uuid = str(result.get("uuid") or "")
else:
uuid = current["key"].removeprefix(_KEY)
result = self._post(f"{_SETTINGS}/set_job/{uuid}", {"job": job})
_saved(result, f"the job {spec['name']}")
self._sj_reload()
return {"success": True, "output": f"{_KEY}{uuid} {'changed' if current else 'created'}"}
def delete_scheduled_job(self, key: str, *, digest: str) -> ScheduledJobResultDict:
"""Delete the application's job *key*; any other job is only switched."""
job = self._sj_job(key, digest)
if job["managed_by"] != "application":
raise ScheduledJobRefused(
f"{key} was not written by an application; it is only switched"
)
result = self._post(f"{_SETTINGS}/del_job/{key.removeprefix(_KEY)}")
if result.get("result") != "deleted":
raise RuntimeError(f"OPNsense did not delete {key}: {result}")
self._sj_reload()
return {"success": True, "output": f"{key} deleted"}
+1 -1
View File
@@ -23,7 +23,7 @@ classifiers = [
]
dependencies = [
"napalm>=5.1", # older releases import pkg_resources, which a 3.12 venv lacks
"napalm-device-types>=4.21.1",
"napalm-device-types>=4.27.0",
"requests>=2.28.0",
]
+198
View File
@@ -0,0 +1,198 @@
{
"job": {
"jobs": {
"job": {
"0b6e3c52-6a1e-4c7a-9a0e-2f0a1d6c3b11": {
"origin": "cron",
"enabled": "1",
"minutes": "0",
"hours": "3",
"days": "*",
"months": "*",
"weekdays": "1-5",
"who": "root",
"command": {
"filter refresh_aliases": {
"value": "Update and reload firewall aliases",
"selected": 0
},
"firmware auto-update": {
"value": "Automatic firmware update",
"selected": 0
},
"ids update": {
"value": "Update and reload intrusion detection rules",
"selected": 0
},
"system ha_reconfigure_backup": {
"value": "HA update and reconfigure backup",
"selected": 0
},
"system remote backup": {
"value": "Remote backup",
"selected": 1
},
"unbound dnsbl": {
"value": "Update Unbound DNSBLs",
"selected": 0
}
},
"parameters": "",
"description": "Nightly configuration backup"
},
"1f2d8e04-93b7-4f4e-8c55-7d1e0c9a2b22": {
"origin": "IDS",
"enabled": "0",
"minutes": "30",
"hours": "1",
"days": "*",
"months": "*",
"weekdays": "*",
"who": "root",
"command": {
"filter refresh_aliases": {
"value": "Update and reload firewall aliases",
"selected": 0
},
"firmware auto-update": {
"value": "Automatic firmware update",
"selected": 0
},
"ids update": {
"value": "Update and reload intrusion detection rules",
"selected": 1
},
"system ha_reconfigure_backup": {
"value": "HA update and reconfigure backup",
"selected": 0
},
"system remote backup": {
"value": "Remote backup",
"selected": 0
},
"unbound dnsbl": {
"value": "Update Unbound DNSBLs",
"selected": 0
}
},
"parameters": "",
"description": "ids rule updates"
},
"2a7c9b16-4d3e-4b8f-a1c2-5e6f7a8b9c33": {
"origin": "cron",
"enabled": "1",
"minutes": "*/15",
"hours": "*",
"days": "*",
"months": "*",
"weekdays": "*",
"who": "root",
"command": {
"filter refresh_aliases": {
"value": "Update and reload firewall aliases",
"selected": 0
},
"firmware auto-update": {
"value": "Automatic firmware update",
"selected": 0
},
"ids update": {
"value": "Update and reload intrusion detection rules",
"selected": 0
},
"system ha_reconfigure_backup": {
"value": "HA update and reconfigure backup",
"selected": 0
},
"system remote backup": {
"value": "Remote backup",
"selected": 0
},
"unbound dnsbl": {
"value": "Update Unbound DNSBLs",
"selected": 1
}
},
"parameters": "",
"description": "napalm-scheduled-job: dnsbl-refresh Refresh the DNS blocklists"
},
"3c8d0e27-5f4a-4c9b-b2d3-6f7a8b9c0d44": {
"origin": "cron",
"enabled": "1",
"minutes": "0",
"hours": "4",
"days": "1",
"months": "*",
"weekdays": "*",
"who": "root",
"command": {
"filter refresh_aliases": {
"value": "Update and reload firewall aliases",
"selected": 1
},
"firmware auto-update": {
"value": "Automatic firmware update",
"selected": 0
},
"ids update": {
"value": "Update and reload intrusion detection rules",
"selected": 0
},
"system ha_reconfigure_backup": {
"value": "HA update and reconfigure backup",
"selected": 0
},
"system remote backup": {
"value": "Remote backup",
"selected": 0
},
"unbound dnsbl": {
"value": "Update Unbound DNSBLs",
"selected": 0
}
},
"parameters": "--quiet\n--all\n",
"description": "Monthly alias refresh"
},
"4d9e1f38-6a5b-4dac-83e4-7a8b9c0d1e55": {
"origin": "cron",
"enabled": "0",
"minutes": "0",
"hours": "5",
"days": "*",
"months": "*",
"weekdays": "0",
"who": "root",
"command": {
"filter refresh_aliases": {
"value": "Update and reload firewall aliases",
"selected": 0
},
"firmware auto-update": {
"value": "Automatic firmware update",
"selected": 1
},
"ids update": {
"value": "Update and reload intrusion detection rules",
"selected": 0
},
"system ha_reconfigure_backup": {
"value": "HA update and reconfigure backup",
"selected": 0
},
"system remote backup": {
"value": "Remote backup",
"selected": 0
},
"unbound dnsbl": {
"value": "Update Unbound DNSBLs",
"selected": 0
}
},
"parameters": "",
"description": "napalm-scheduled-job: Not_A_Name written by hand"
}
}
}
}
}
@@ -0,0 +1,40 @@
{
"job": {
"origin": "cron",
"enabled": "1",
"minutes": "0",
"hours": "0",
"days": "*",
"months": "*",
"weekdays": "*",
"who": "root",
"command": {
"filter refresh_aliases": {
"value": "Update and reload firewall aliases",
"selected": 0
},
"firmware auto-update": {
"value": "Automatic firmware update",
"selected": 0
},
"ids update": {
"value": "Update and reload intrusion detection rules",
"selected": 0
},
"system ha_reconfigure_backup": {
"value": "HA update and reconfigure backup",
"selected": 0
},
"system remote backup": {
"value": "Remote backup",
"selected": 0
},
"unbound dnsbl": {
"value": "Update Unbound DNSBLs",
"selected": 0
}
},
"parameters": "",
"description": ""
}
}
+495
View File
@@ -0,0 +1,495 @@
"""Scheduled jobs on OPNsense: the cron jobs of config.xml, over the REST API (#43).
OPNsense keeps its cron jobs in its configuration (``System > Settings > Cron``)
and runs each as a configd action. ``/api/cron/settings/get`` answers with every
job, the action as the option its ``command`` list has selected;
``get_job`` without a uuid is a blank job, and its ``command`` options are the
actions configd offers. The fixtures are synthetic, in the shape of an OPNsense
26.7 answer: an admin's job, one the intrusion detection registered (origin
``IDS``), one an application wrote, one with parameters, and one whose marker
names no valid job.
"""
from __future__ import annotations
import copy
import json
from pathlib import Path
from unittest.mock import MagicMock, patch
import pytest
from napalm_device_types import ScheduledJobsMixin
from napalm_device_types.scheduled_jobs import (
ScheduledJobChanged,
ScheduledJobRefused,
job_digest,
)
from napalm_opnsense.opnsense import OPNsenseDriver
FIXTURES = Path(__file__).parent / "fixtures"
SETTINGS = json.loads((FIXTURES / "cron_settings_get.json").read_text())
BLANK_JOB = json.loads((FIXTURES / "cron_settings_get_job.json").read_text())
ADMIN = "opnsense:0b6e3c52-6a1e-4c7a-9a0e-2f0a1d6c3b11"
PLATFORM = "opnsense:1f2d8e04-93b7-4f4e-8c55-7d1e0c9a2b22"
OWN = "opnsense:2a7c9b16-4d3e-4b8f-a1c2-5e6f7a8b9c33"
OWN_UUID = OWN.removeprefix("opnsense:")
WITH_PARAMETERS = "opnsense:3c8d0e27-5f4a-4c9b-b2d3-6f7a8b9c0d44"
BAD_MARKER = "opnsense:4d9e1f38-6a5b-4dac-83e4-7a8b9c0d1e55"
SAVED = {"result": "saved"}
RELOADED = {"status": "ok"}
class Firewall:
"""Answers the cron API as a firewall would, and records every POST."""
def __init__(self, settings=SETTINGS, blank=BLANK_JOB, answers=None):
self.settings = copy.deepcopy(settings)
self.blank = blank
self.answers = answers or {}
self.posts: list[tuple[str, dict | None]] = []
def get(self, path: str):
if path == "/api/cron/settings/get":
return self.settings
if path == "/api/cron/settings/get_job":
return self.blank
raise AssertionError(f"unexpected GET {path}")
def post(self, path: str, data=None):
self.posts.append((path, data))
for prefix, answer in self.answers.items():
if path.startswith(prefix):
return answer
if path == "/api/cron/service/reconfigure":
return RELOADED
if path.startswith("/api/cron/settings/toggle_job/"):
return {"result": "Enabled" if path.endswith("/1") else "Disabled", "changed": True}
if path.startswith("/api/cron/settings/del_job/"):
return {"result": "deleted"}
if path == "/api/cron/settings/add_job":
return {"result": "saved", "uuid": "5eaf2049-7b6c-4ebd-94f5-8b9c0d1e2f66"}
if path.startswith("/api/cron/settings/set_job/"):
return SAVED
raise AssertionError(f"unexpected POST {path}")
@property
def paths(self) -> list[str]:
return [path for path, _ in self.posts]
def _driver(firewall: Firewall) -> OPNsenseDriver:
with patch("napalm_opnsense.opnsense.requests.Session"):
drv = OPNsenseDriver(hostname="fw", username="k", password="s")
drv.session = MagicMock()
drv._get = firewall.get # type: ignore[method-assign]
drv._post = firewall.post # type: ignore[method-assign]
return drv
def _jobs(firewall: Firewall | None = None) -> dict[str, dict]:
reading = _driver(firewall or Firewall()).get_scheduled_jobs()
return {job["key"]: job for job in reading["jobs"]}
def _digest(key: str) -> str:
return _jobs()[key]["digest"]
def _spec(**over) -> dict:
spec = {
"name": "alias-refresh",
"schedule": "*/30 * * * *",
"run_as": "root",
"command": "filter refresh_aliases",
"enabled": True,
"description": "Keep the aliases fresh",
}
spec.update(over)
return spec
def test_the_driver_offers_every_verb():
assert issubclass(OPNsenseDriver, ScheduledJobsMixin)
for verb in (
"get_scheduled_jobs",
"manage_scheduled_job",
"put_scheduled_job",
"delete_scheduled_job",
"get_scheduled_job_commands",
):
assert callable(getattr(OPNsenseDriver, verb))
class TestReading:
def test_every_job_of_the_configuration(self):
reading = _driver(Firewall()).get_scheduled_jobs()
assert reading["time_zone"] is None
assert [job["key"] for job in reading["jobs"]] == [
ADMIN,
PLATFORM,
OWN,
WITH_PARAMETERS,
BAD_MARKER,
]
def test_an_admins_job(self):
job = _jobs()[ADMIN]
assert job == {
"key": ADMIN,
"kind": "cron",
"source": "config.xml",
"run_as": "root",
"schedule": "0 3 * * 1-5",
"schedule_kind": "cron",
"command": "system remote backup",
"enabled": True,
"managed_by": "admin",
"actions": ["enable", "disable"],
"description": "Nightly configuration backup",
"digest": job["digest"],
}
assert job["digest"] == job_digest(job)
def test_a_job_another_service_registered_is_the_platforms(self):
job = _jobs()[PLATFORM]
assert (job["managed_by"], job["enabled"], job["command"]) == (
"platform",
False,
"ids update",
)
assert job["actions"] == ["enable", "disable"]
def test_a_job_an_application_wrote_carries_its_name(self):
job = _jobs()[OWN]
assert job["managed_by"] == "application"
assert job["name"] == "dnsbl-refresh"
assert job["description"] == "Refresh the DNS blocklists"
assert job["actions"] == ["enable", "disable", "edit", "delete"]
def test_a_marker_without_a_valid_name_is_an_admins_job(self):
job = _jobs()[BAD_MARKER]
assert job["managed_by"] == "admin"
assert "name" not in job
assert job["description"] == "napalm-scheduled-job: Not_A_Name written by hand"
def test_a_marker_on_another_services_job_is_still_the_platforms(self):
firewall = Firewall()
node = firewall.settings["job"]["jobs"]["job"][PLATFORM.removeprefix("opnsense:")]
node["description"] = "napalm-scheduled-job: ids-update taken over"
job = _jobs(firewall)[PLATFORM]
assert job["managed_by"] == "platform"
assert "name" not in job
def test_the_parameters_follow_the_action(self):
assert _jobs()[WITH_PARAMETERS]["command"] == "filter refresh_aliases --quiet --all"
def test_a_job_with_no_description_has_none(self):
firewall = Firewall()
firewall.settings["job"]["jobs"]["job"][ADMIN.removeprefix("opnsense:")]["description"] = ""
assert "description" not in _jobs(firewall)[ADMIN]
def test_an_action_configd_no_longer_offers_reads_as_no_command(self):
firewall = Firewall()
node = firewall.settings["job"]["jobs"]["job"][ADMIN.removeprefix("opnsense:")]
node["command"] = {k: {**v, "selected": 0} for k, v in node["command"].items()}
assert _jobs(firewall)[ADMIN]["command"] == ""
def test_a_command_given_as_a_plain_value_is_taken_as_it_is(self):
firewall = Firewall()
node = firewall.settings["job"]["jobs"]["job"][ADMIN.removeprefix("opnsense:")]
node["command"] = "system remote backup"
assert _jobs(firewall)[ADMIN]["command"] == "system remote backup"
def test_enabling_a_job_changes_its_digest(self):
firewall = Firewall()
firewall.settings["job"]["jobs"]["job"][ADMIN.removeprefix("opnsense:")]["enabled"] = "0"
assert _jobs(firewall)[ADMIN]["digest"] != _digest(ADMIN)
def test_no_jobs_is_an_empty_list(self):
firewall = Firewall(settings={"job": {"jobs": {"job": []}}})
assert _driver(firewall).get_scheduled_jobs()["jobs"] == []
@pytest.mark.parametrize(
"answer",
[{}, [], {"job": {}}, {"job": {"jobs": {}}}, {"job": {"jobs": {"job": "x"}}}],
)
def test_an_answer_without_the_jobs_raises(self, answer):
with pytest.raises(ValueError, match="jobs"):
_driver(Firewall(settings=answer)).get_scheduled_jobs()
def test_a_job_without_its_schedule_raises(self):
firewall = Firewall()
del firewall.settings["job"]["jobs"]["job"][ADMIN.removeprefix("opnsense:")]["hours"]
with pytest.raises(ValueError, match="schedule"):
_driver(firewall).get_scheduled_jobs()
class TestCommands:
def test_the_actions_configd_offers(self):
commands = _driver(Firewall()).get_scheduled_job_commands()
assert commands[0] == {
"command": "filter refresh_aliases",
"description": "Update and reload firewall aliases",
}
assert [c["command"] for c in commands] == [
"filter refresh_aliases",
"firmware auto-update",
"ids update",
"system ha_reconfigure_backup",
"system remote backup",
"unbound dnsbl",
]
def test_the_blank_placeholder_is_no_action(self):
blank = copy.deepcopy(BLANK_JOB)
blank["job"]["command"][""] = {"value": "None", "selected": 1}
commands = _driver(Firewall(blank=blank)).get_scheduled_job_commands()
assert "" not in [c["command"] for c in commands]
@pytest.mark.parametrize(
"answer", [{}, [], {"job": []}, {"job": {"command": {}}}, {"job": {"command": "x"}}]
)
def test_no_actions_raises(self, answer):
# configd that did not answer leaves the list empty: that is no reading.
with pytest.raises(ValueError, match="configd"):
_driver(Firewall(blank=answer)).get_scheduled_job_commands()
class TestManage:
@pytest.mark.parametrize(
"action, flag, word", [("enable", "1", "enabled"), ("disable", "0", "disabled")]
)
def test_switching_a_job_toggles_it_and_reloads_cron(self, action, flag, word):
firewall = Firewall()
result = _driver(firewall).manage_scheduled_job(PLATFORM, action, digest=_digest(PLATFORM))
assert result == {"success": True, "output": f"{PLATFORM} {word}"}
assert firewall.paths == [
f"/api/cron/settings/toggle_job/{PLATFORM.removeprefix('opnsense:')}/{flag}",
"/api/cron/service/reconfigure",
]
def test_running_a_job_now_is_refused(self):
firewall = Firewall()
with pytest.raises(ScheduledJobRefused, match="run"):
_driver(firewall).manage_scheduled_job(ADMIN, "run", digest=_digest(ADMIN))
assert firewall.posts == []
def test_an_unknown_action_is_refused(self):
with pytest.raises(ScheduledJobRefused, match="unknown action"):
_driver(Firewall()).manage_scheduled_job(ADMIN, "edit", digest=_digest(ADMIN))
def test_an_unknown_job_raises_lookup_error(self):
with pytest.raises(LookupError):
_driver(Firewall()).manage_scheduled_job("opnsense:nope", "enable", digest="x")
def test_a_job_changed_since_it_was_read_is_not_touched(self):
firewall = Firewall()
with pytest.raises(ScheduledJobChanged):
_driver(firewall).manage_scheduled_job(ADMIN, "disable", digest="0" * 16)
assert firewall.posts == []
def test_a_refused_toggle_raises_and_cron_is_not_reloaded(self):
firewall = Firewall(answers={"/api/cron/settings/toggle_job/": {"result": "failed"}})
with pytest.raises(RuntimeError, match="failed"):
_driver(firewall).manage_scheduled_job(ADMIN, "disable", digest=_digest(ADMIN))
assert "/api/cron/service/reconfigure" not in firewall.paths
def test_a_cron_that_does_not_reload_raises(self):
firewall = Firewall(answers={"/api/cron/service/reconfigure": {"status": "failed"}})
with pytest.raises(RuntimeError, match="reload"):
_driver(firewall).manage_scheduled_job(ADMIN, "disable", digest=_digest(ADMIN))
class TestPut:
def test_a_new_job_is_added_with_its_marker_and_cron_reloads(self):
firewall = Firewall()
result = _driver(firewall).put_scheduled_job(_spec())
assert result == {
"success": True,
"output": "opnsense:5eaf2049-7b6c-4ebd-94f5-8b9c0d1e2f66 created",
}
assert firewall.posts == [
(
"/api/cron/settings/add_job",
{
"job": {
"enabled": "1",
"minutes": "*/30",
"hours": "*",
"days": "*",
"months": "*",
"weekdays": "*",
"command": "filter refresh_aliases",
"parameters": "",
"description": "napalm-scheduled-job: alias-refresh Keep the aliases fresh",
}
},
),
("/api/cron/service/reconfigure", None),
]
def test_an_application_job_is_changed_in_place(self):
firewall = Firewall()
spec = _spec(name="dnsbl-refresh", command="unbound dnsbl", enabled=False)
del spec["description"]
result = _driver(firewall).put_scheduled_job(spec, digest=_digest(OWN))
assert result == {"success": True, "output": f"{OWN} changed"}
path, data = firewall.posts[0]
assert path == f"/api/cron/settings/set_job/{OWN_UUID}"
assert data["job"]["enabled"] == "0"
assert data["job"]["description"] == "napalm-scheduled-job: dnsbl-refresh"
assert firewall.paths[-1] == "/api/cron/service/reconfigure"
def test_the_words_after_the_action_are_its_parameters_one_per_line(self):
firewall = Firewall()
_driver(firewall).put_scheduled_job(
_spec(command=" filter refresh_aliases --quiet --all ")
)
job = firewall.posts[0][1]["job"]
assert (job["command"], job["parameters"]) == ("filter refresh_aliases", "--quiet\n--all")
def test_the_longest_action_the_command_starts_with_wins(self):
blank = copy.deepcopy(BLANK_JOB)
blank["job"]["command"]["filter"] = {"value": "Reload the filter", "selected": 0}
firewall = Firewall(blank=blank)
_driver(firewall).put_scheduled_job(_spec(command="filter refresh_aliases"))
job = firewall.posts[0][1]["job"]
assert (job["command"], job["parameters"]) == ("filter refresh_aliases", "")
@pytest.mark.parametrize(
"schedule, fields",
[
("@daily", ("0", "0", "*", "*", "*")),
("@hourly", ("0", "*", "*", "*", "*")),
("0 6 * jan-Mar MON-fri", ("0", "6", "*", "1-3", "1-5")),
("5 4 1,15 * sun,7", ("5", "4", "1,15", "*", "0,7")),
],
)
def test_the_schedule_is_written_as_its_five_fields(self, schedule, fields):
firewall = Firewall()
_driver(firewall).put_scheduled_job(_spec(schedule=schedule))
job = firewall.posts[0][1]["job"]
assert tuple(job[f] for f in ("minutes", "hours", "days", "months", "weekdays")) == fields
@pytest.mark.parametrize(
"over, match",
[
({"schedule": "@reboot"}, "@reboot"),
({"run_as": "nobody"}, "root"),
({"command": "rm -rf /"}, "configd offers no action"),
({"command": "filter refresh_aliases\nids update"}, "one line"),
({"command": "filter refresh_aliases 'a b'"}, "parameter"),
({"command": "filter refresh_aliases a\\b"}, "parameter"),
({"name": "Not A Name"}, "job name"),
({"schedule": "61 * * * *"}, "range"),
],
)
def test_what_opnsense_cannot_take_is_refused_before_anything_is_written(self, over, match):
firewall = Firewall()
with pytest.raises(ScheduledJobRefused, match=match):
_driver(firewall).put_scheduled_job(_spec(**over))
assert firewall.posts == []
def test_a_new_job_whose_name_is_taken_is_not_written(self):
firewall = Firewall()
with pytest.raises(ScheduledJobChanged, match="dnsbl-refresh"):
_driver(firewall).put_scheduled_job(_spec(name="dnsbl-refresh"))
assert firewall.posts == []
def test_a_change_to_a_job_that_is_gone_raises_lookup_error(self):
with pytest.raises(LookupError, match="alias-refresh"):
_driver(Firewall()).put_scheduled_job(_spec(), digest="0" * 16)
def test_a_change_to_a_job_changed_since_it_was_read_is_not_written(self):
firewall = Firewall()
with pytest.raises(ScheduledJobChanged):
_driver(firewall).put_scheduled_job(_spec(name="dnsbl-refresh"), digest="0" * 16)
assert firewall.posts == []
def test_what_opnsense_refuses_to_save_is_refused(self):
refusal = {"result": "failed", "validations": {"job.description": "Text too long"}}
firewall = Firewall(answers={"/api/cron/settings/add_job": refusal})
with pytest.raises(ScheduledJobRefused, match="Text too long"):
_driver(firewall).put_scheduled_job(_spec())
assert "/api/cron/service/reconfigure" not in firewall.paths
def test_a_failed_save_without_a_reason_raises(self):
firewall = Firewall(answers={"/api/cron/settings/set_job/": {"result": "failed"}})
with pytest.raises(RuntimeError, match="failed"):
_driver(firewall).put_scheduled_job(_spec(name="dnsbl-refresh"), digest=_digest(OWN))
assert "/api/cron/service/reconfigure" not in firewall.paths
class TestDelete:
def test_an_application_job_is_deleted_and_cron_reloads(self):
firewall = Firewall()
result = _driver(firewall).delete_scheduled_job(OWN, digest=_digest(OWN))
assert result == {"success": True, "output": f"{OWN} deleted"}
assert firewall.paths == [
f"/api/cron/settings/del_job/{OWN_UUID}",
"/api/cron/service/reconfigure",
]
@pytest.mark.parametrize("key", [ADMIN, PLATFORM, BAD_MARKER])
def test_a_job_no_application_wrote_is_only_switched(self, key):
firewall = Firewall()
with pytest.raises(ScheduledJobRefused, match="only switched"):
_driver(firewall).delete_scheduled_job(key, digest=_digest(key))
assert firewall.posts == []
def test_a_job_changed_since_it_was_read_is_not_deleted(self):
firewall = Firewall()
with pytest.raises(ScheduledJobChanged):
_driver(firewall).delete_scheduled_job(OWN, digest="0" * 16)
assert firewall.posts == []
def test_a_delete_opnsense_did_not_do_raises(self):
firewall = Firewall(answers={"/api/cron/settings/del_job/": {"result": "not found"}})
with pytest.raises(RuntimeError, match="not found"):
_driver(firewall).delete_scheduled_job(OWN, digest=_digest(OWN))
assert "/api/cron/service/reconfigure" not in firewall.paths