feat(scheduled-jobs): scheduled jobs on QTS's crontab (#17) #18

Merged
christianmanivong merged 2 commits from feature/scheduled-jobs into main 2026-11-03 08:20:28 +00:00
7 changed files with 700 additions and 4 deletions

No files matched your search

+1 -1
View File
@@ -11,7 +11,7 @@ jobs:
uses: NAPALM/driver-standard/.gitea/workflows/driver.yml@main
with:
standard-ref: main
# Until napalm-linux 0.15.0 is on PyPI.
# Until napalm-device-types 4.26.0 and napalm-linux 0.16.0 are on PyPI.
pre-install: "git+https://git.netork.io/NAPALM/napalm-device-types.git git+https://git.netork.io/NAPALM/napalm-linux.git"
secrets:
PYPI_TOKEN: ${{ secrets.PYPI_TOKEN }}
+25
View File
@@ -7,6 +7,31 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Added
- Scheduled jobs on QTS (#17, NetOrk/netork#1933). napalm-linux now brings
napalm-device-types' POSIX scheduled jobs with Debian's paths, which are wrong on QTS;
`QtsScheduledJobsMixin` puts them on QTS's:
- `get_scheduled_jobs()` reads `/etc/config/crontab` -- the crontab QTS keeps across a
boot, admin's (uid 0), without a user field -- and the other users' crontabs in
crond's spool, in one round trip. crond's copy of admin's is not read twice. No
`/etc/crontab`, cron.d, run-parts, anacron or systemd: BusyBox crond reads none of
them. The time zone is QTS's own (`getcfg System 'Time Zone'`) when it is an IANA
name. A box without `/etc/config/crontab` raises rather than reporting no jobs.
- The firmware's own lines (`/sbin/hwclock -s`, `/sbin/qfstrim`,
`/etc/init.d/backup_conf.sh`, ...) are `platform`, by a conservative list of the
commands stock QTS writes; a line that also runs something else stays the admin's.
They only **run**: QTS's scripts and UI keep these lines, and whether they leave one
switched off alone needs a real box (#13).
- `manage_scheduled_job()` switches the admin's lines and runs any line now. A write
goes into `/etc/config/crontab` (`cp`, so the file keeps its mode, owner and links),
then `crontab /etc/config/crontab` and `/etc/init.d/crond.sh restart` -- without
them QTS forgets it on the next boot. A run is bounded by `sleep` and `kill`, not
`timeout`, which QTS's BusyBox cannot be relied on to have.
- `put_scheduled_job()` and `delete_scheduled_job()` keep an application's job as
marked lines in `/etc/config/crontab`, never in a cron.d. Only as admin (`root` is
taken for it): every other crontab lives in RAM and is gone after a boot.
- Requires napalm-device-types 4.26.0 and napalm-linux 0.16.0.
### Changed
- Personal accounts are switched off (`get_local_accounts`, `add_local_account`,
`lock_local_account` = `None`). napalm-linux now adds and locks accounts with
+3
View File
@@ -35,6 +35,9 @@ output is how a driver ends up passing its own tests and failing on a real NAS.
| `container_engines`, `open_container_engine` | Container Station's docker, `system dial-stdio` | ✅ |
| `get_services`, `get_users`, `get_processes` | inherited from `LinuxDriver` | ✅ |
| `get_health_metrics` | inherited (UCD-MIB over SNMP) | ✅ |
| `get_scheduled_jobs` | `/etc/config/crontab` (admin's), crond's spool | ✅ synthetic fixtures |
| `manage_scheduled_job` | the crontab, then `crontab` + `crond.sh restart`; the firmware's lines only run | ✅ synthetic fixtures |
| `put_scheduled_job`, `delete_scheduled_job` | marked lines in `/etc/config/crontab`, as admin | ✅ synthetic fixtures |
| `ping`, `ping_sweep` | inherited from `LinuxDriver` | ✅ |
| Snapshots, quotas, replication, QPKG install/remove | — | out of scope for v1 |
+7 -1
View File
@@ -32,6 +32,8 @@ from napalm_device_types import (
)
from napalm_linux.linux import LinuxDriver
from napalm_qnap_qts.scheduled_jobs import QtsScheduledJobsMixin
#: QNAP Systems' IANA enterprise number. Used by discovery to recognise a NAS
#: from its SNMP sysObjectID before anyone has supplied credentials.
QNAP_ENTERPRISE_OID = "1.3.6.1.4.1.24681"
@@ -47,9 +49,13 @@ _DOCKER_GLOBS = (
_VERSION_RE = re.compile(r"(\d+)\.")
class QnapQtsDriver(StorageDriver, HypervisorDriver, LinuxDriver):
class QnapQtsDriver(StorageDriver, HypervisorDriver, QtsScheduledJobsMixin, LinuxDriver):
"""NAPALM driver for QNAP NAS systems running QTS.
Scheduled jobs are napalm-device-types' POSIX ones on QTS's paths
(:class:`~napalm_qnap_qts.scheduled_jobs.QtsScheduledJobsMixin`), ahead of the
Debian paths ``LinuxDriver`` would give them.
Declares all three roles it fills. The role bases carry no implementations,
so listing them alongside ``LinuxDriver`` costs nothing and shadows nothing;
what the driver can actually do is whatever it implements below.
+212
View File
@@ -0,0 +1,212 @@
"""Scheduled jobs on QTS: admin's crontab in ``/etc/config``, BusyBox crond, no systemd.
napalm-device-types' :class:`~napalm_device_types.PosixScheduledJobsMixin` reaches this
driver through ``LinuxDriver`` with Debian's paths. On QTS they are wrong:
* **The crontab that survives a boot is** ``/etc/config/crontab``: admin's (uid 0), no
user field, on the configuration partition. ``/etc`` is in RAM; ``/etc/crontab`` is at
most a link to it. crond runs a copy in its spool, which QTS rebuilds from that file
on every boot -- a change made only through ``crontab`` is gone after one.
* **BusyBox crond** reads its spool and nothing else: no ``/etc/crontab``, no cron.d, no
run-parts or anacron, and QTS has no systemd. Other users' crontabs live in the spool
only, in RAM, and are read and switched as they are.
* **The firmware writes its own lines** into the same file. They are ``platform``,
recognised by :data:`QTS_PLATFORM_COMMANDS`, and only run: QTS's scripts and UI keep
those lines, and whether they leave one switched off alone needs a real box (#13).
A write puts the new text into ``/etc/config/crontab`` with ``cp`` (the file keeps its
inode, mode and owner, and a link to it stays one), then loads it with
``crontab /etc/config/crontab`` and restarts crond, as QNAP says to.
"""
from __future__ import annotations
import fnmatch
import re
import shlex
from typing import ClassVar
from napalm_device_types import (
PosixScheduledJobsMixin,
ScheduledJobRefused,
ScheduledJobResultDict,
ScheduledJobSpecDict,
job_digest,
)
from napalm_device_types._crontab import run_script
from napalm_device_types._scheduled_reading import Reading, parse_reading
from napalm_device_types.scheduled_jobs import RUN_TIMEOUT
from napalm_device_types.scheduled_jobs_posix import _result
#: The commands stock QTS 4 and 5 write into ``/etc/config/crontab``, as ``fnmatch``
#: patterns on the whole command; arguments may follow. Conservative on purpose: a line
#: that is not listed is the admin's. Taken from stock crontabs as published; to be
#: checked against a harvested one (#13).
QTS_PLATFORM_COMMANDS = (
"/sbin/hwclock -s",
"/sbin/vs_refresh",
"/sbin/clean_reset_pwd",
"/sbin/clean_upload_file",
"/sbin/notice_log_tool *",
"/sbin/qfstrim",
"/sbin/qpkg_cli --check_license *",
"/usr/bin/power_clean *",
"/etc/init.d/nss2_dusg.sh",
"/etc/init.d/storage_usage.sh",
"/etc/init.d/backup_conf.sh",
"/etc/init.d/ImRd.sh bgThGen",
"/bin/rm -rf /mnt/HDA_ROOT/twonkymedia/twonky-*",
)
#: A second command on the line: then it is not the firmware's line alone.
_SEPARATOR = re.compile(r";|\||&&|`|\$\(|(?<![>&])&(?!>)")
#: ``sh -c`` script: runs ``"$@"`` (after the limit, ``$1``) for at most ``$1`` seconds and
#: exits as it did. Sleep and kill only -- QTS's BusyBox has no ``timeout`` to rely on.
BOUNDED_RUN = (
't=$1; shift; "$@" & p=$!; '
'( i=0; while [ "$i" -lt "$t" ] && kill -0 "$p" 2>/dev/null; do sleep 1; i=$((i+1)); done; '
'kill "$p" 2>/dev/null ) </dev/null >/dev/null 2>&1 & '
'wait "$p"'
)
#: QTS keeps its time zone in uLinux.conf; only an IANA name is taken from there.
_TIME_ZONE_SH = (
"getcfg System 'Time Zone' 2>/dev/null"
" | grep '^[A-Za-z_][A-Za-z_]*/[A-Za-z0-9_+/-][A-Za-z0-9_+/-]*$'; "
"readlink /etc/localtime 2>/dev/null"
)
def is_platform_command(command: str) -> bool:
"""Whether *command* is one QTS's firmware writes into its crontab, and nothing more."""
command = command.strip()
if _SEPARATOR.search(command):
return False
return any(
fnmatch.fnmatchcase(command, p) or fnmatch.fnmatchcase(command, f"{p} *")
for p in QTS_PLATFORM_COMMANDS
)
class QtsScheduledJobsMixin(PosixScheduledJobsMixin):
"""The POSIX scheduled jobs, on QTS's paths and BusyBox userland."""
#: BusyBox crond reads none of these, and QTS has no run-parts.
CRON_SYSTEM_FILES: ClassVar[tuple[str, ...]] = ()
CRON_D_DIRS: ClassVar[tuple[str, ...]] = ()
RUN_PARTS_DIRS: ClassVar[tuple[tuple[str, str], ...]] = ()
#: crond's spool, in RAM: other users' crontabs, and the copy of admin's.
CRON_SPOOL_DIRS: ClassVar[tuple[str, ...]] = ("/var/spool/cron/crontabs",)
#: An application's job is marked lines in admin's crontab, never a cron.d file.
OWN_CRON_D_DIR: ClassVar[str | None] = None
#: No package manager owns a crontab line on QTS.
OWNERS_SH: ClassVar[str] = "cat >/dev/null"
TIME_ZONE_SH: ClassVar[str] = _TIME_ZONE_SH
#: The crontab QTS keeps across a boot, and whose it is: uid 0 is ``admin`` on QTS.
QTS_CRONTAB: ClassVar[str] = "/etc/config/crontab"
ROOT_USER: ClassVar[str] = "admin"
CROND_RESTART: ClassVar[tuple[str, ...]] = ("/etc/init.d/crond.sh", "restart")
#: Where the new text is staged, as a word ``sh`` expands; BusyBox's ``mktemp``
#: wants a template.
STAGE_DIR: ClassVar[str] = "${TMPDIR:-/tmp}"
# -- reading ---------------------------------------------------------------------------------
def _qts_report(self) -> str:
"""The report ``parse_reading`` takes: admin's crontab, the spool, the time zone.
A missing ``/etc/config/crontab`` ends it before its end marker, so the reading
raises. Every file ends with a newline of its own, so a last line without one
does not swallow the next header.
"""
crontab = shlex.quote(self.QTS_CRONTAB)
spool = " ".join(f"{shlex.quote(d)}/*" for d in self.CRON_SPOOL_DIRS)
skip = shlex.quote(self.ROOT_USER)
return (
"printf '%s%s\\n' SJ_ BEGIN; "
f"printf '[user %s]\\n' {crontab}; sed 's/^/|/' {crontab} || exit 1; echo; "
f'for f in {spool}; do [ -f "$f" ] || continue; '
f'case "${{f##*/}}" in {skip}|cron.update) continue ;; esac; '
"printf '[user %s]\\n' \"$f\"; sed 's/^/|/' \"$f\" || exit 1; echo; done; "
f"echo '[tz]'; {{ {self.TIME_ZONE_SH}; }}; echo '[owners]'; "
"printf '%s%s\\n' SJ_ END"
)
def _sj_read(self) -> Reading:
result = self.run_command(self._qts_report(), privileged=True, timeout=120)
reading = parse_reading(result.stdout)
for job in reading.jobs:
if job["source"] != self.QTS_CRONTAB:
continue
job["run_as"] = self.ROOT_USER
job["digest"] = job_digest(job)
if job["managed_by"] != "application" and is_platform_command(str(job["command"])):
job["managed_by"] = "platform"
job["actions"] = ["run"]
return reading
# -- admin's crontab -------------------------------------------------------------------------
def _sj_spool_user(self, path: str) -> str | None:
if path == self.QTS_CRONTAB:
return self.ROOT_USER
return super()._sj_spool_user(path)
def _sj_crontab(self, user: str) -> str:
if user != self.ROOT_USER:
return super()._sj_crontab(user)
return self._sj_checked(["cat", "--", self.QTS_CRONTAB]).stdout
def _sj_stage(self, content: str) -> str:
template = f'"{self.STAGE_DIR}/sj.XXXXXX"'
stage = f't=$(mktemp {template}) && chmod 600 "$t" && cat > "$t" && printf %s "$t"'
result = self.run_command(stage, timeout=30, stdin=content.encode())
path = result.stdout.strip().splitlines()[-1:] or [""]
if result.exit_code != 0 or not path[0].startswith("/"):
raise RuntimeError(f"cannot stage a file: {result.stderr.strip()}")
return path[0]
def _sj_write_crontab(self, user: str, text: str) -> None:
"""admin's into ``/etc/config/crontab`` first, then into crond; others as they are."""
if user != self.ROOT_USER:
super()._sj_write_crontab(user, text)
return
staged = self._sj_stage(text)
try:
self._sj_checked(["cp", "--", staged, self.QTS_CRONTAB])
finally:
self.run_command(shlex.join(["rm", "-f", "--", staged]), timeout=30)
self._sj_checked(["crontab", self.QTS_CRONTAB])
self._sj_checked(self.CROND_RESTART)
# -- running ---------------------------------------------------------------------------------
def _sj_as(self, user: str, script: str) -> list[str]:
if user == self.ROOT_USER:
return ["sh", "-c", script]
return super()._sj_as(user, script)
def _sj_run(self, job: dict[str, object], reading: Reading) -> ScheduledJobResultDict:
"""The line's command as crond would run it, bounded by :data:`BOUNDED_RUN`."""
line = self._sj_line(job, self._sj_text(str(job["source"])))
argv = self._sj_as(line.run_as, run_script(line.command, line.env))
bounded = ["sh", "-c", BOUNDED_RUN, "sh", str(RUN_TIMEOUT), *argv]
return _result(self._sj_root(bounded, timeout=RUN_TIMEOUT + 30), "ran")
# -- an application's own jobs ---------------------------------------------------------------
def put_scheduled_job(
self, spec: ScheduledJobSpecDict, *, digest: str | None = None
) -> ScheduledJobResultDict:
"""An application's job as marked lines in admin's crontab, the only one QTS keeps.
``root`` is taken for admin, uid 0 on QTS.
"""
if spec.get("run_as") not in (self.ROOT_USER, "root"):
raise ScheduledJobRefused(
f"QTS keeps only {self.ROOT_USER}'s crontab across a boot; "
f"a job cannot run as {spec.get('run_as')!r}"
)
return super().put_scheduled_job({**spec, "run_as": self.ROOT_USER}, digest=digest)
+2 -2
View File
@@ -40,8 +40,8 @@ classifiers = [
# processes, Docker) from LinuxDriver rather than reimplementing it.
dependencies = [
"napalm>=4.0",
"napalm-device-types>=4.22.0",
"napalm-linux>=0.15.0",
"napalm-device-types>=4.27.0",
"napalm-linux>=0.16.0",
"netmiko>=4.0.0",
"paramiko>=5.0.0", # CVE-2026-44405
]
+450
View File
@@ -0,0 +1,450 @@
"""Scheduled jobs on QTS: admin's persistent crontab, BusyBox crond, no systemd (#17).
QTS inherits napalm-device-types' ``PosixScheduledJobsMixin`` through ``LinuxDriver``,
whose Debian paths are wrong here: the crontab that survives a boot is
``/etc/config/crontab`` (admin's, no user field), crond runs a copy of it, and the
userland is BusyBox. The fixtures are synthetic -- no QTS box was reachable (#13).
Every command is run by a real shell over a directory that stands in for the NAS:
``sh`` (dash here) and, where it is installed, BusyBox's ``ash`` with nothing but
BusyBox applets on ``PATH``. ``crontab``, ``getcfg`` and ``crond.sh`` are scripts that
keep a log.
"""
from __future__ import annotations
import os
import shutil
import subprocess
import time
from pathlib import Path
import pytest
from napalm_device_types import CommandResult, job_digest
from napalm_device_types.scheduled_jobs import ScheduledJobChanged, ScheduledJobRefused
from napalm_linux.linux import LinuxDriver
from napalm_qnap_qts import QnapQtsDriver
from napalm_qnap_qts.scheduled_jobs import BOUNDED_RUN, is_platform_command
CRONTAB_TOOL = """#!/bin/sh
echo "crontab $*" >> "$LOG"
if [ "$1" = -u ]; then
if [ "$3" = -l ]; then
[ -f "$SPOOL/$2" ] || { echo "no crontab for $2" >&2; exit 1; }
cat "$SPOOL/$2"
else
cp "$3" "$SPOOL/$2"
fi
else
cp "$1" "$SPOOL/admin"
fi
"""
GETCFG = """#!/bin/sh
[ "$1" = System ] && [ "$2" = "Time Zone" ] && cat "$FIX/tz"
"""
LOG_ONLY = """#!/bin/sh
echo "$(basename "$0") $*" >> "$LOG"
"""
#: What BusyBox provides on QTS that these commands need -- and nothing else.
_APPLETS = (
"sh cat cp rm chmod mktemp printf sed grep readlink sleep echo test [ basename kill"
).split()
BUSYBOX = shutil.which("busybox") or "busybox"
SHELLS = ["sh"] + (["busybox"] if shutil.which("busybox") else [])
def _tool(directory: Path, name: str, body: str) -> None:
path = directory / name
path.write_text(body)
path.chmod(0o755)
class Nas(QnapQtsDriver):
"""A QTS box in a directory; every command is run by a real shell, as admin."""
def __init__(self, root: Path, shell: str) -> None: # noqa: D107 - no NAPALM setup here
self.root = root
self.shell = shell
self.QTS_CRONTAB = str(root / "etc/config/crontab") # type: ignore[misc]
self.CRON_SPOOL_DIRS = (str(root / "spool"),) # type: ignore[misc]
self.CROND_RESTART = (str(root / "bin/crond.sh"), "restart") # type: ignore[misc]
self.commands: list[str] = []
def run_command(self, command, *, privileged=False, timeout=60, stdin=None): # type: ignore[no-untyped-def]
self.commands.append(command)
path = f"{self.root / 'bin'}:{os.environ['PATH']}"
argv = ["sh", "-c", command]
if self.shell == "busybox":
path = f"{self.root / 'bin'}:{self.root / 'bb'}"
# This BusyBox runs its own applets ahead of PATH: crontab is the fake by name.
fake = f'crontab() {{ "{self.root}/bin/crontab" "$@"; }}; '
argv = [BUSYBOX, "sh", "-c", fake + command]
env = {
"PATH": path,
"LOG": str(self.root / "log"),
"SPOOL": str(self.root / "spool"),
"FIX": str(self.root / "fix"),
"TMPDIR": str(self.root / "tmp"),
}
done = subprocess.run(argv, input=stdin, capture_output=True, env=env, timeout=30)
return CommandResult(done.stdout.decode(), done.stderr.decode(), done.returncode)
def log(self) -> list[str]:
log = self.root / "log"
return log.read_text().splitlines() if log.exists() else []
@pytest.fixture(params=SHELLS)
def root(request: pytest.FixtureRequest, tmp_path: Path) -> Path:
for d in ("etc/config", "spool", "bin", "bb", "fix", "tmp"):
(tmp_path / d).mkdir(parents=True)
crontab = tmp_path / "etc/config/crontab"
crontab.write_text(
"# m h dom m dow cmd\n"
"0 4 * * * /sbin/hwclock -s\n"
"2 0 * * * /sbin/qfstrim\n"
"4 3 * * 3 /etc/init.d/backup_conf.sh\n"
"0 2 * * * /sbin/qpkg_cli --check_license 0 > /dev/null 2>/dev/null\n"
"0 3 * * * /bin/rm -rf /mnt/HDA_ROOT/twonkymedia/twonky-*\n"
f"30 1 * * * echo ran > {tmp_path}/ran\n"
"15 2 * * * /sbin/qfstrim; /share/CACHEDEV1_DATA/tidy.sh\n"
)
crontab.chmod(0o600)
# crond's copy of it, and the file BusyBox's crontab leaves for crond.
shutil.copy(crontab, tmp_path / "spool/admin")
(tmp_path / "spool/cron.update").write_text("admin\n")
(tmp_path / "spool/guest").write_text("*/5 * * * * /share/homes/guest/poll.sh\n")
(tmp_path / "fix/tz").write_text("Europe/Berlin\n")
_tool(tmp_path / "bin", "crontab", CRONTAB_TOOL)
_tool(tmp_path / "bin", "getcfg", GETCFG)
_tool(tmp_path / "bin", "crond.sh", LOG_ONLY)
for applet in _APPLETS:
(tmp_path / "bb" / applet).symlink_to(BUSYBOX)
tmp_path.joinpath("shell").write_text(request.param)
return tmp_path
def _nas(root: Path) -> Nas:
return Nas(root, root.joinpath("shell").read_text())
def _jobs(nas: Nas) -> dict[str, dict]:
return {j["key"]: j for j in nas.get_scheduled_jobs()["jobs"]}
def _by_command(nas: Nas, command: str) -> dict:
return next(j for j in _jobs(nas).values() if j["command"] == command)
def _crontab(root: Path) -> str:
return (root / "etc/config/crontab").read_text()
class TestVerbs:
"""The inheritance trap: what LinuxDriver offers is not what QTS offers."""
def test_exactly_these_scheduled_job_verbs(self):
"""A scheduled-jobs verb napalm-linux or napalm-device-types adds later fails
here until someone has decided whether QTS can take it."""
offered = {
name
for name in dir(QnapQtsDriver)
if "scheduled" in name
and not name.startswith("_")
and callable(getattr(QnapQtsDriver, name))
}
assert offered == {
"get_scheduled_jobs",
"manage_scheduled_job",
"put_scheduled_job",
"delete_scheduled_job",
}
def test_every_verb_is_the_qts_one(self):
"""The four verbs reach QTS's reading and writing, not Linux's paths."""
assert QnapQtsDriver._sj_read is not LinuxDriver._sj_read
assert QnapQtsDriver._sj_write_crontab is not LinuxDriver._sj_write_crontab
assert QnapQtsDriver._sj_run is not LinuxDriver._sj_run
assert QnapQtsDriver.put_scheduled_job is not LinuxDriver.put_scheduled_job
def test_no_linux_paths(self):
"""BusyBox crond reads its spool only: no /etc/crontab, cron.d, run-parts,
anacron or systemd -- and an application's job never goes to a cron.d."""
assert QnapQtsDriver.CRON_SYSTEM_FILES == ()
assert QnapQtsDriver.CRON_D_DIRS == ()
assert QnapQtsDriver.RUN_PARTS_DIRS == ()
assert QnapQtsDriver.OWN_CRON_D_DIR is None
assert QnapQtsDriver.QTS_CRONTAB == "/etc/config/crontab"
assert QnapQtsDriver.ROOT_USER == "admin"
def test_the_actions_each_job_takes(self, root: Path):
nas = _nas(root)
nas.put_scheduled_job({**SPEC})
jobs = _jobs(nas)
assert _by_command(nas, "/sbin/qfstrim")["actions"] == ["run"]
assert _by_command(nas, f"echo ran > {root}/ran")["actions"] == [
"enable",
"disable",
"run",
]
assert jobs["job:tidy"]["actions"] == ["enable", "disable", "run", "edit", "delete"]
class TestReading:
def test_the_persistent_crontab_is_admins(self, root: Path):
nas = _nas(root)
job = _by_command(nas, f"echo ran > {root}/ran")
assert job["source"] == str(root / "etc/config/crontab")
assert (job["kind"], job["run_as"], job["schedule"]) == ("cron", "admin", "30 1 * * *")
assert (job["managed_by"], job["enabled"]) == ("admin", True)
assert job["digest"] == job_digest(job)
def test_one_round_trip_without_linux_sources(self, root: Path):
nas = _nas(root)
nas.get_scheduled_jobs()
(command,) = nas.commands
for linux in ("systemctl", "/etc/cron.d", "anacrontab", "dpkg"):
assert linux not in command
def test_the_firmwares_lines_are_the_platforms(self, root: Path):
nas = _nas(root)
platform = {j["command"] for j in _jobs(nas).values() if j["managed_by"] == "platform"}
assert platform == {
"/sbin/hwclock -s",
"/sbin/qfstrim",
"/etc/init.d/backup_conf.sh",
"/sbin/qpkg_cli --check_license 0 > /dev/null 2>/dev/null",
"/bin/rm -rf /mnt/HDA_ROOT/twonkymedia/twonky-*",
}
def test_crond_s_copy_is_not_read_twice(self, root: Path):
sources = {j["source"] for j in _jobs(_nas(root)).values()}
assert sources == {str(root / "etc/config/crontab"), str(root / "spool/guest")}
def test_another_users_crontab_from_the_spool(self, root: Path):
guest = _by_command(_nas(root), "/share/homes/guest/poll.sh")
assert (guest["run_as"], guest["managed_by"]) == ("guest", "admin")
def test_the_time_zone_is_qts_s_own(self, root: Path):
assert _nas(root).get_scheduled_jobs()["time_zone"] == "Europe/Berlin"
def test_a_time_zone_that_is_no_iana_name_is_not_taken(self, root: Path):
(root / "fix/tz").write_text("(GMT+01:00) Amsterdam, Berlin\n")
nas = _nas(root)
nas.TIME_ZONE_SH = nas.TIME_ZONE_SH.replace( # type: ignore[misc]
"/etc/localtime", str(root / "localtime")
)
assert nas.get_scheduled_jobs()["time_zone"] is None
(root / "localtime").symlink_to("/usr/share/zoneinfo/Asia/Tokyo")
assert nas.get_scheduled_jobs()["time_zone"] == "Asia/Tokyo"
def test_a_last_line_without_newline_keeps_the_next_file(self, root: Path):
(root / "etc/config/crontab").write_text("0 4 * * * /sbin/hwclock -s")
assert {j["command"] for j in _jobs(_nas(root)).values()} == {
"/sbin/hwclock -s",
"/share/homes/guest/poll.sh",
}
def test_without_its_crontab_the_reading_fails(self, root: Path):
"""Never a shorter list: no /etc/config/crontab is a box not as QTS leaves it."""
(root / "etc/config/crontab").unlink()
with pytest.raises(ValueError, match="cut short"):
_nas(root).get_scheduled_jobs()
class TestPlatformCommands:
@pytest.mark.parametrize(
"command",
[
"/sbin/hwclock -s",
"/sbin/vs_refresh",
"/sbin/qfstrim",
"/etc/init.d/nss2_dusg.sh",
"/usr/bin/power_clean -c 2>/dev/null",
"/sbin/qpkg_cli --check_license 0 > /dev/null 2>&1",
],
)
def test_the_firmwares(self, command: str):
assert is_platform_command(command)
@pytest.mark.parametrize(
"command",
[
"/share/CACHEDEV1_DATA/backup.sh",
"/sbin/qfstrim; /share/x.sh",
"/sbin/qfstrim && /share/x.sh",
"/sbin/qfstrim | logger",
"/sbin/qfstrim & /share/x.sh",
"/sbin/qfstrim $(cat /share/x)",
"/sbin/hwclock -w",
"/etc/init.d/smb.sh restart",
],
)
def test_not_the_firmwares(self, command: str):
"""Conservative: an admin's line that starts like the firmware's stays admin's."""
assert not is_platform_command(command)
class TestSwitching:
def test_a_line_switched_off_survives_a_boot(self, root: Path):
"""Written to /etc/config/crontab, loaded into crond, crond restarted -- in
that order, so QTS keeps it."""
nas = _nas(root)
job = _by_command(nas, f"echo ran > {root}/ran")
result = nas.manage_scheduled_job(job["key"], "disable", digest=job["digest"])
assert result["success"]
assert f"#napalm-disabled: 30 1 * * * echo ran > {root}/ran\n" in _crontab(root)
assert nas.log() == [
f"crontab {root}/etc/config/crontab",
"crond.sh restart",
]
assert (root / "spool/admin").read_text() == _crontab(root)
assert os.stat(root / "etc/config/crontab").st_mode & 0o777 == 0o600
assert not list((root / "tmp").iterdir()) # staged in $TMPDIR, and removed
off = _jobs(nas)[job["key"]]
assert not off["enabled"]
nas.manage_scheduled_job(job["key"], "enable", digest=off["digest"])
assert f"\n30 1 * * * echo ran > {root}/ran\n" in _crontab(root)
def test_every_other_line_is_left_as_it_was(self, root: Path):
nas = _nas(root)
before = _crontab(root)
job = _by_command(nas, f"echo ran > {root}/ran")
nas.manage_scheduled_job(job["key"], "disable", digest=job["digest"])
changed = [ln for ln in _crontab(root).splitlines() if ln not in before.splitlines()]
assert changed == [f"#napalm-disabled: 30 1 * * * echo ran > {root}/ran"]
def test_the_firmwares_line_is_not_switched(self, root: Path):
"""QTS's own scripts and UI keep these lines; whether they leave one switched
off alone needs a real box."""
nas = _nas(root)
job = _by_command(nas, "/sbin/qfstrim")
with pytest.raises(ScheduledJobRefused):
nas.manage_scheduled_job(job["key"], "disable", digest=job["digest"])
def test_a_change_it_did_not_see(self, root: Path):
nas = _nas(root)
job = _by_command(nas, f"echo ran > {root}/ran")
with pytest.raises(ScheduledJobChanged):
nas.manage_scheduled_job(job["key"], "disable", digest="0" * 16)
assert nas.log() == []
def test_another_users_line_goes_through_crontab(self, root: Path):
nas = _nas(root)
guest = _by_command(nas, "/share/homes/guest/poll.sh")
nas.manage_scheduled_job(guest["key"], "disable", digest=guest["digest"])
assert (root / "spool/guest").read_text().startswith("#napalm-disabled: */5")
assert nas.log()[-1].startswith("crontab -u guest /")
assert "napalm-disabled" not in _crontab(root)
def test_nothing_is_written_when_staging_fails(self, root: Path):
nas = _nas(root)
nas.STAGE_DIR = str(root / "no-such-dir") # type: ignore[misc]
job = _by_command(nas, f"echo ran > {root}/ran")
before = _crontab(root)
with pytest.raises(RuntimeError, match="cannot stage"):
nas.manage_scheduled_job(job["key"], "disable", digest=job["digest"])
assert _crontab(root) == before
assert nas.log() == []
def test_a_failed_load_says_so(self, root: Path):
_tool(root / "bin", "crontab", "#!/bin/sh\necho 'crontab: bad' >&2\nexit 1\n")
nas = _nas(root)
job = _by_command(nas, f"echo ran > {root}/ran")
with pytest.raises(RuntimeError, match="bad"):
nas.manage_scheduled_job(job["key"], "disable", digest=job["digest"])
class TestRunning:
def test_a_line_runs_now_as_admin(self, root: Path):
nas = _nas(root)
job = _by_command(nas, f"echo ran > {root}/ran")
result = nas.manage_scheduled_job(job["key"], "run", digest=job["digest"])
assert result == {"success": True, "output": "ran"}
assert (root / "ran").read_text() == "ran\n"
assert "timeout" not in nas.commands[-1]
def test_the_firmwares_line_runs(self, root: Path):
(root / "etc/config/crontab").write_text("2 0 * * * /sbin/qfstrim\n")
nas = _nas(root)
job = _by_command(nas, "/sbin/qfstrim")
result = nas.manage_scheduled_job(job["key"], "run", digest=job["digest"])
assert not result["success"] # /sbin/qfstrim is not on this machine
assert "qfstrim" in result["output"]
def test_another_user_runs_through_su(self, root: Path):
nas = _nas(root)
assert nas._sj_as("admin", "true") == ["sh", "-c", "true"]
assert nas._sj_as("guest", "true") == ["su", "-s", "/bin/sh", "guest", "-c", "true"]
def test_a_run_is_bounded_without_timeout(self, root: Path):
"""QTS's BusyBox has no ``timeout`` to rely on: sleep and kill bound the run."""
shell = [BUSYBOX, "sh"] if _nas(root).shell == "busybox" else ["sh"]
started = time.monotonic()
done = subprocess.run(
[*shell, "-c", BOUNDED_RUN, "sh", "1", "sleep", "20"], capture_output=True, timeout=15
)
assert time.monotonic() - started < 5
assert done.returncode != 0
quick = subprocess.run(
[*shell, "-c", BOUNDED_RUN, "sh", "5", "sh", "-c", "echo hi; exit 3"],
capture_output=True,
timeout=15,
)
assert (quick.returncode, quick.stdout) == (3, b"hi\n")
SPEC = {
"name": "tidy",
"schedule": "*/30 1-2 * * *",
"run_as": "root",
"command": "find /share/Public/tmp -mtime +7 -delete",
"enabled": True,
"description": "tidy the share",
}
class TestApplicationJobs:
def test_marked_lines_in_admins_crontab(self, root: Path):
nas = _nas(root)
assert nas.put_scheduled_job({**SPEC})["output"] == "job:tidy created"
assert _crontab(root).endswith(
"# napalm-scheduled-job: tidy tidy the share\n"
"*/30 1-2 * * * find /share/Public/tmp -mtime +7 -delete\n"
)
assert nas.log()[-2:] == [f"crontab {root}/etc/config/crontab", "crond.sh restart"]
job = _jobs(nas)["job:tidy"]
assert (job["run_as"], job["managed_by"], job["name"]) == ("admin", "application", "tidy")
assert not (root / "etc/cron.d").exists()
def test_changed_then_deleted(self, root: Path):
nas = _nas(root)
nas.put_scheduled_job({**SPEC, "run_as": "admin"})
job = _jobs(nas)["job:tidy"]
nas.put_scheduled_job({**SPEC, "enabled": False}, digest=job["digest"])
assert "#napalm-disabled: */30 1-2" in _crontab(root)
job = _jobs(nas)["job:tidy"]
assert nas.delete_scheduled_job("job:tidy", digest=job["digest"])["success"]
assert "napalm-scheduled-job: tidy" not in _crontab(root)
assert "*/30 1-2" not in _crontab(root)
assert "/sbin/qfstrim\n" in _crontab(root)
assert nas.log()[-2:] == [f"crontab {root}/etc/config/crontab", "crond.sh restart"]
def test_only_admins_crontab_survives_a_boot(self, root: Path):
with pytest.raises(ScheduledJobRefused, match="admin"):
_nas(root).put_scheduled_job({**SPEC, "run_as": "guest"})
def test_one_line_of_command(self, root: Path):
with pytest.raises(ScheduledJobRefused, match="one line"):
_nas(root).put_scheduled_job({**SPEC, "command": "a\nb"})
def test_a_job_the_application_did_not_write_is_not_deleted(self, root: Path):
nas = _nas(root)
job = _by_command(nas, f"echo ran > {root}/ran")
with pytest.raises(ScheduledJobRefused):
nas.delete_scheduled_job(job["key"], digest=job["digest"])