feat(time-sync): keep DHCP's NTP servers and read them apart #37

Merged
christianmanivong merged 1 commits from feat/time-sync-dhcp-ntp-2380 into main 2026-11-11 22:01:33 +00:00
5 changed files with 234 additions and 35 deletions

No files matched your search

+13 -5
View File
@@ -7,13 +7,21 @@ recorded here; their history is the git log.
## [Unreleased]
### Changed
- Needs napalm-device-types 4.36.0 or later (`TimeSyncDict.dhcp_ntp_servers`).
- A time sync write no longer sets `use_dhcp=0`, as the first time sync commit (#36) did;
a router that commit wrote keeps `use_dhcp=0` until it is set back.
### Added
- Time sync (napalm-device-types `TimeSyncMixin`, NetOrk/netork#2364): `get_time_sync()`
reads the router's NTP servers and its zone from one `uci -q show system`;
`configure_time_sync()` writes both. The servers replace `system.ntp.server`, enable
the client and turn off the servers of a DHCP lease (`use_dhcp=0`), so the router
takes time from exactly these; `sysntpd` restarts. The zone is written as its name
(`zonename`, in LuCI's spelling) and its POSIX rule (`timezone`, from
reads the router's NTP servers and its zone from `uci -q show system`;
`configure_time_sync()` writes both. The servers replace `system.ntp.server` and enable
the client; `sysntpd` restarts. `use_dhcp` stays as the router has it: the servers of
a DHCP lease (option 42) count (NetOrk/netork#2380). `dhcp_ntp_servers` names them --
what sysntpd adds while `use_dhcp` is on, each interface's `data.ntpserver` in `ubus
call network.interface dump` (only the `dhcp_interface` list's where it names any),
read in the same round trip; None where netifd gives no answer. The zone is written as
its name (`zonename`, in LuCI's spelling) and its POSIX rule (`timezone`, from
napalm-device-types' `posix_tz`), then `/etc/init.d/system reload` and a restart of
logd apply it -- the rule, not the name, is what libc reads (NetOrk/netork#1849). A
value the router already has is not written again and nothing restarts. The zone
+9 -3
View File
@@ -122,9 +122,15 @@ Which interfaces is the caller's choice; at least one is required.
writes both values into UCI's `system` config and applies them; a value the router
already has is not written again, so nothing restarts.
* The servers replace the `system.ntp.server` list; the client is enabled, the servers
of a DHCP lease are turned off (`use_dhcp=0`) so the router has exactly these, and
`sysntpd` restarts.
* The servers replace the `system.ntp.server` list; the client is enabled and
`sysntpd` restarts. `use_dhcp` stays as the router has it: the NTP servers of a DHCP
lease (option 42) count (NetOrk/netork#2380).
`get_time_sync()` reads the configured servers and the zone from `uci -q show system`,
and in the same round trip the servers a lease brought, as `dhcp_ntp_servers`: what
sysntpd adds to ntpd's list while `use_dhcp` is on -- every interface's `data.ntpserver`
in `ubus call network.interface dump`, or only the `dhcp_interface` list's. `[]` while
`use_dhcp` or the client is off; None where netifd gave no answer.
* The zone goes into `zonename` (as LuCI keeps it, with spaces for underscores) and its
POSIX rule into `timezone`, the value libc reads -- an IANA name there leaves the
router on UTC (NetOrk/netork#1849). `/etc/init.d/system reload` applies it, and logd
+70 -14
View File
@@ -16,9 +16,13 @@ OpenWrt keeps both values in ``/etc/config/system`` (NetOrk/netork#2364):
* **The servers** are the ``system.ntp.server`` list, which ``/etc/init.d/sysntpd``
hands to BusyBox ntpd while ``system.ntp.enabled`` is on. A disabled client has no
servers: the list names nothing the clock takes time from. sysntpd also adds the
servers of a DHCP lease unless ``use_dhcp`` is ``0``; a write turns that off, so the
router ends up with exactly the servers given.
servers: the list names nothing the clock takes time from.
* **What DHCP brings** (option 42) counts too (NetOrk/netork#2380): while ``use_dhcp``
is on (the default) sysntpd adds the NTP servers of every interface's lease -- or of
the ``dhcp_interface`` list's only -- that netifd keeps as ``data.ntpserver`` in
``ubus call network.interface dump`` (``/lib/netifd/dhcp.script`` stores udhcpc's
``ntpsrv`` there). They are read from the same place, in the same order, as
``dhcp_ntp_servers``; a write leaves ``use_dhcp`` as the router has it.
* **The zone** is kept twice: ``zonename`` (the IANA name, as LuCI writes it -- with
spaces where the IANA name has underscores) and ``timezone``, the POSIX rule
``/etc/init.d/system`` writes to ``/tmp/TZ`` for libc. The rule is what the clock
@@ -27,7 +31,8 @@ OpenWrt keeps both values in ``/etc/config/system`` (NetOrk/netork#2364):
without a matching name is read as the one zone that has it, or as no zone where
several do (``CET-1CEST,...`` is Berlin, Paris, Madrid, ...).
Reading is one ``uci -q show system`` as the login user. Every write compares
Reading is one round trip as the login user: ``uci -q show system``, then netifd's
dump (:data:`READ_WITH_DHCP`). A write reads ``uci`` alone. Every write compares
first and sends nothing when the router already has the values -- an application
aligns on every poll -- else sets the options, commits ``system`` and applies them:
``/etc/init.d/sysntpd restart`` for the servers, ``/etc/init.d/system reload`` and a
@@ -40,6 +45,7 @@ root (``run_command(privileged=True)``); OpenWrt has no sudo.
from __future__ import annotations
import json
import re
import shlex
from collections import defaultdict
@@ -56,6 +62,14 @@ if TYPE_CHECKING:
from napalm_device_types import CommandResult
READ_COMMAND = "uci -q show system"
#: The line between uci's answer and netifd's in :data:`READ_WITH_DHCP`.
DHCP_MARK = "@@napalm-openwrt: network.interface dump"
#: ``get_time_sync``'s reading: uci's ``system`` config, then netifd's interfaces, where
#: a lease keeps its NTP servers. Its exit code is uci's: ubus may refuse, uci may not.
READ_WITH_DHCP = (
f"{READ_COMMAND} && {{ echo {shlex.quote(DHCP_MARK)}; "
"ubus call network.interface dump 2>/dev/null || true; }"
)
#: The rule ``/etc/init.d/system`` uses where the section names none.
DEFAULT_RULE = "UTC"
#: A zone of the database that stands for "not known", not for a place.
@@ -89,7 +103,14 @@ class SystemUci:
@property
def dhcp_servers(self) -> bool:
return (self._one(self.ntp, "use_dhcp") or "1").lower() not in _FALSE
"""Whether sysntpd adds a lease's servers: the client runs and ``use_dhcp`` is on."""
use_dhcp = (self._one(self.ntp, "use_dhcp") or "1").lower() not in _FALSE
return self.has_ntp and self.ntp_enabled and use_dhcp
@property
def dhcp_interfaces(self) -> list[str]:
"""The interfaces whose leases count; [] for every interface, as sysntpd does."""
return [i for value in self.ntp.get("dhcp_interface", []) for i in value.split()]
@property
def servers(self) -> list[str]:
@@ -139,6 +160,31 @@ def parse_system_uci(output: str) -> SystemUci:
return parsed
def _lease_servers(entry: Any, interfaces: list[str]) -> list[str]:
"""One interface's ``data.ntpserver``, if it is one of *interfaces* (or they are [])."""
if not isinstance(entry, dict) or (interfaces and entry.get("interface") not in interfaces):
return []
data = entry.get("data")
servers = data.get("ntpserver") if isinstance(data, dict) else None
return servers.split() if isinstance(servers, str) else []
def lease_ntp_servers(dump: str, interfaces: list[str]) -> list[str] | None:
"""The NTP servers in netifd's ``network.interface dump``, as sysntpd collects them:
each interface's ``data.ntpserver`` (apart by spaces) in netifd's order, of
*interfaces* only where it names any, each once. None for no readable answer."""
try:
parsed = json.loads(dump)
except ValueError:
return None
if not isinstance(parsed, dict) or not isinstance(parsed.get("interface"), list):
return None
found: list[str] = []
for entry in parsed["interface"]:
found.extend(s for s in _lease_servers(entry, interfaces) if s not in found)
return found
def _time_of(rule: str) -> tuple[Any, ...] | None:
"""The offsets and transitions of *rule*, without the names it gives them."""
parts = parse_posix_tz(rule)
@@ -218,24 +264,34 @@ class OpenWrtTimeSyncMixin(TimeSyncMixin):
# -- reading ---------------------------------------------------------------------------------
def _ts_read(self) -> SystemUci:
result = self.run_command(READ_COMMAND, timeout=30)
def _ts_run_read(self, command: str) -> str:
result = self.run_command(command, timeout=30)
if result.exit_code != 0:
said = result.stderr.strip() or result.stdout.strip() or f"exit {result.exit_code}"
raise RuntimeError(f"{READ_COMMAND}: {said}")
return parse_system_uci(result.stdout)
return result.stdout
def _ts_read(self) -> SystemUci:
return parse_system_uci(self._ts_run_read(READ_COMMAND))
def get_time_sync(self) -> TimeSyncDict:
"""The router's time servers and zone, from one ``uci -q show system``.
"""The router's time servers and zone, and what its DHCP leases bring, in one
round trip (:data:`READ_WITH_DHCP`).
``dhcp_ntp_servers`` is None where sysntpd would add a lease's servers but
netifd gave no answer.
:raises RuntimeError: when uci could not be read.
"""
reading = self._ts_read()
uci, _, dump = self._ts_run_read(READ_WITH_DHCP).partition(f"{DHCP_MARK}\n")
reading = parse_system_uci(uci)
dhcp = lease_ntp_servers(dump, reading.dhcp_interfaces) if reading.dhcp_servers else []
return {
"ntp_servers": reading.servers,
"timezone": reading.zone,
"synchronized": None,
"daemon": None,
"dhcp_ntp_servers": dhcp,
}
def get_ntp_servers(self) -> dict[str, dict[str, Any]]:
@@ -257,9 +313,7 @@ class OpenWrtTimeSyncMixin(TimeSyncMixin):
def _write_ntp_servers(self, servers: list[str]) -> FixStepOutcomeDict:
now = self._ts_read()
if [s.lower() for s in now.servers] == [
s.lower() for s in servers
] and not now.dhcp_servers:
if [s.lower() for s in now.servers] == [s.lower() for s in servers]:
return {"ok": True, "output": "the router already has these servers"}
commands = []
if not now.has_ntp:
@@ -269,7 +323,6 @@ class OpenWrtTimeSyncMixin(TimeSyncMixin):
commands += [_uci_command("uci", "add_list", f"system.ntp.server={s}") for s in servers]
commands += [
"uci set system.ntp.enabled=1",
"uci set system.ntp.use_dhcp=0",
"uci commit system",
"/etc/init.d/sysntpd enable",
"/etc/init.d/sysntpd restart",
@@ -294,8 +347,11 @@ class OpenWrtTimeSyncMixin(TimeSyncMixin):
__all__ = [
"DHCP_MARK",
"READ_WITH_DHCP",
"OpenWrtTimeSyncMixin",
"SystemUci",
"lease_ntp_servers",
"parse_system_uci",
"same_time",
"zone_for_rule",
+1 -1
View File
@@ -21,7 +21,7 @@ classifiers = [
]
dependencies = [
"napalm>=5.1", # older releases import pkg_resources, which a 3.12 venv lacks
"napalm_device_types>=4.35.0", # TimeSyncMixin, posix_tz
"napalm_device_types>=4.36.0", # TimeSyncMixin with dhcp_ntp_servers, posix_tz
"netmiko>=4.0.0",
"paramiko>=5.0.0", # CVE-2026-44405
"netaddr>=0.7.19", # IPAddress; below it the package has none (CI lowest job)
+141 -12
View File
@@ -8,11 +8,15 @@ OpenWrt keeps both values in UCI's ``system`` config: the servers in the
The device is a fake ``uci`` behind ``run_command``: it renders ``uci -q show system``
the way uci prints it, and applies the ``uci set``/``delete``/``add_list`` lines a
write sends, so the template's ``verify`` step reads back what was written.
write sends, so the template's ``verify`` step reads back what was written. Behind it
is netifd's ``ubus call network.interface dump``, where a DHCP lease's NTP servers
(option 42) are ``data.ntpserver`` -- sysntpd hands them to ntpd too while
``use_dhcp`` is on, and they count (NetOrk/netork#2380).
"""
from __future__ import annotations
import json
import shlex
from typing import Any
from unittest.mock import MagicMock, patch
@@ -23,7 +27,7 @@ from napalm_device_types.posix_tz import UnknownTimezone, posix_tz
from napalm_device_types.time_sync import TIME_SYNC_VALUES
from napalm_openwrt.openwrt import OpenWrtDriver
from napalm_openwrt.time_sync_mixin import parse_system_uci, zone_for_rule
from napalm_openwrt.time_sync_mixin import DHCP_MARK, parse_system_uci, zone_for_rule
# ``uci show system`` of OpenWrt 23.05 out of the box: zonename 'UTC', the rule GMT0,
# the four pool servers as a list.
@@ -81,6 +85,23 @@ system.ntp.enabled='1'
"""
def _interface(name: str, ntpserver: str | None = None, **more: Any) -> dict[str, Any]:
"""One interface of ``ubus call network.interface dump``, as netifd prints it."""
data = {} if ntpserver is None else {"ntpserver": ntpserver}
return {"interface": name, "up": True, "proto": "dhcp", "data": data, **more}
#: A router whose WAN lease brings two NTP servers (udhcpc's ``ntpsrv``, apart by
#: spaces, as /lib/netifd/dhcp.script stores it) and a LAN of its own.
LEASE = {
"interface": [
_interface("lan", proto="static"),
_interface("wan", "192.168.1.1 192.168.1.2", leasetime=43200),
_interface("wan6", proto="dhcpv6"),
]
}
def _render(sections: list[tuple[str, str, dict[str, list[str] | str]]]) -> str:
"""*sections* as ``uci show system`` prints them."""
lines = []
@@ -115,13 +136,16 @@ def _parse(output: str) -> list[tuple[str, str, dict[str, list[str] | str]]]:
class _Uci:
"""A router's ``system`` config behind ``run_command``; keeps every command it ran."""
def __init__(self, show: str = STOCK, fail: str = "") -> None:
def __init__(
self, show: str = STOCK, fail: str = "", dump: dict[str, Any] | str | None = None
) -> None:
self.sections = _parse(show)
self.fail = fail
self.dump = {"interface": []} if dump is None else dump
self.ran: list[tuple[str, bool]] = []
def writes(self) -> list[str]:
return [c for c, _ in self.ran if c != "uci -q show system"]
return [c for c, _ in self.ran if not c.startswith("uci -q show system")]
def __call__(
self,
@@ -134,6 +158,10 @@ class _Uci:
self.ran.append((command, privileged))
if command == "uci -q show system":
return CommandResult(_render(self.sections), "", 0)
if command.startswith("uci -q show system &&"):
assert "ubus call network.interface dump" in command
dump = self.dump if isinstance(self.dump, str) else json.dumps(self.dump, indent=1)
return CommandResult(f"{_render(self.sections)}{DHCP_MARK}\n{dump}\n", "", 0)
for part in command.split(" && "):
if self.fail and self.fail in part:
return CommandResult("", f"{self.fail}: failed", 1)
@@ -173,8 +201,8 @@ def _driver(uci: _Uci, username: str = "root") -> OpenWrtDriver:
return drv
def _read(show: str) -> dict[str, Any]:
return dict(_driver(_Uci(show)).get_time_sync())
def _read(show: str, dump: dict[str, Any] | str | None = None) -> dict[str, Any]:
return dict(_driver(_Uci(show, dump=dump)).get_time_sync())
# -- the driver takes part ----------------------------------------------------------------------
@@ -203,14 +231,94 @@ def test_a_stock_router_reads_its_pool_servers_and_utc():
"timezone": "UTC",
"synchronized": None,
"daemon": None,
"dhcp_ntp_servers": [],
}
def test_the_read_is_one_uci_show_as_the_login_user():
def test_the_read_is_one_round_trip_as_the_login_user():
uci = _Uci()
_driver(uci).get_time_sync()
assert uci.ran == [("uci -q show system", False)]
((command, privileged),) = uci.ran
assert command.startswith("uci -q show system &&")
assert privileged is False
# -- reading what DHCP brought ------------------------------------------------------------------
def test_a_lease_s_ntp_servers_are_read_apart_from_the_configured_ones():
reading = _read(STOCK, LEASE)
assert reading["dhcp_ntp_servers"] == ["192.168.1.1", "192.168.1.2"]
assert reading["ntp_servers"] == [
"0.openwrt.pool.ntp.org",
"1.openwrt.pool.ntp.org",
"2.openwrt.pool.ntp.org",
"3.openwrt.pool.ntp.org",
]
def test_every_interface_s_lease_counts_in_netifd_s_order_each_once():
dump = {
"interface": [
_interface("wan", "192.168.1.1"),
_interface("iot", "10.40.100.1 192.168.1.1"),
]
}
assert _read(STOCK, dump)["dhcp_ntp_servers"] == ["192.168.1.1", "10.40.100.1"]
def test_an_interface_without_a_lease_or_with_an_odd_one_brings_nothing():
dump = {
"interface": [
"wan",
{"interface": "wan", "data": "192.168.1.1"},
{"interface": "wan", "data": {"ntpserver": ["192.168.1.1"]}},
{"interface": "wan"},
_interface("iot", "10.40.100.1"),
]
}
assert _read(STOCK, dump)["dhcp_ntp_servers"] == ["10.40.100.1"]
def test_dhcp_interface_names_the_interfaces_whose_leases_count():
show = STOCK + "system.ntp.dhcp_interface='iot'\n"
dump = {
"interface": [
_interface("wan", "192.168.1.1"),
_interface("iot", "10.40.100.1"),
]
}
assert _read(show, dump)["dhcp_ntp_servers"] == ["10.40.100.1"]
def test_use_dhcp_off_takes_nothing_from_a_lease():
show = STOCK + "system.ntp.use_dhcp='0'\n"
assert _read(show, LEASE)["dhcp_ntp_servers"] == []
def test_a_disabled_client_takes_nothing_from_a_lease():
show = STOCK.replace("system.ntp.enabled='1'", "system.ntp.enabled='0'")
assert _read(show, LEASE)["dhcp_ntp_servers"] == []
@pytest.mark.parametrize(
"dump", ["", "Command failed: Permission denied", "{", "[]", '{"interface": {}}']
)
def test_without_netifd_s_answer_dhcp_is_unknown(dump):
assert _read(STOCK, dump)["dhcp_ntp_servers"] is None
def test_without_netifd_s_answer_a_router_that_takes_nothing_from_dhcp_still_says_so():
show = STOCK + "system.ntp.use_dhcp='0'\n"
assert _read(show, "")["dhcp_ntp_servers"] == []
def test_a_zone_set_through_luci_reads_by_its_name():
@@ -346,12 +454,33 @@ def test_the_servers_replace_the_list_and_ntpd_restarts():
assert all(privileged for c, privileged in uci.ran if c in uci.writes())
def test_writing_servers_turns_off_the_ones_dhcp_brings():
# sysntpd adds the servers of a DHCP lease to the list unless use_dhcp is 0.
uci = _Uci(STOCK)
_driver(uci).configure_time_sync(ntp_servers=["10.0.0.1"])
def test_writing_servers_leaves_the_ones_dhcp_brings():
# DHCP option 42 counts (NetOrk/netork#2380): sysntpd keeps adding a lease's servers.
uci = _Uci(STOCK, dump=LEASE)
result = _driver(uci).configure_time_sync(ntp_servers=["10.0.0.1"])
assert result["success"] is True, result["output"]
assert "use_dhcp" not in uci._section("ntp")
assert "use_dhcp" not in uci.writes()[0]
def test_writing_servers_keeps_use_dhcp_as_the_router_has_it():
uci = _Uci(STOCK + "system.ntp.use_dhcp='0'\n")
result = _driver(uci).configure_time_sync(ntp_servers=["10.0.0.1"])
assert result["success"] is True, result["output"]
assert uci._section("ntp")["use_dhcp"] == "0"
assert "use_dhcp" not in uci.writes()[0]
def test_the_same_servers_with_dhcp_on_write_nothing():
uci = _Uci(LUCI_BERLIN, dump=LEASE)
result = _driver(uci).configure_time_sync(
ntp_servers=["0.openwrt.pool.ntp.org", "1.openwrt.pool.ntp.org"]
)
assert result["success"] is True, result["output"]
assert uci.writes() == []
def test_writing_servers_enables_a_disabled_client():