feat: initial QNAP QTS driver scaffold

QTS is a Linux distribution, so the driver inherits the OS surface from LinuxDriver and adds QNAP's storage, QPKG and virtualisation layers. What is here: the class with its discovery fingerprints, per-session detection of the QTS major version and of the QPKG-local docker and virsh binaries, and explicit resolution of the MRO collisions that StorageDriver creates over LinuxDriver. What is not: the storage, QPKG and VM parsers. Those need real command output from QTS 4 and QTS 5 hardware to be written against, which is what tools/harvest.sh collects and tools/sanitize.py anonymises. Two tests are marked xfail(strict) as the specification for that work.
2026-08-21 09:53:13 +07:00
commit 26d723f417
11 changed files with 893 additions and 0 deletions
@@ -0,0 +1,13 @@
 __pycache__/
 *.py[cod]
 *.egg-info/
 build/
 dist/
 .venv/
 .pytest_cache/
 .mypy_cache/
 .ruff_cache/
 .coverage
 htmlcov/
 # Raw, unsanitised harvest output — never commit device data.
 tools/harvest-out/
@@ -0,0 +1,25 @@
 # Changelog
 All notable changes to this project are documented here.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 ## [Unreleased]
 ### Added
 - Initial driver scaffold: `QnapQtsDriver(StorageDriver, LinuxDriver)` with
  discovery fingerprints (SNMP enterprise OID 1.3.6.1.4.1.24681, HTTP, port
  specs) and the `qnap_qts` entry point.
 - Explicit `DEVICE_CLASS = "storage"` so netOrk does not classify a QNAP as a
  plain Linux host and hide its Storage tab.
 - QTS major-version detection, Container Station docker path discovery and
  Virtualization Station virsh discovery, all resolved once per session in
  `open()`.
 - MRO collision resolution for `get_services`, which `StorageDriver` would
  otherwise shadow.
 - `tools/harvest.sh` and `tools/sanitize.py` for capturing and anonymising
  command output from real hardware.
 ### Pending
 - Storage, QPKG and VM parsers — blocked on the fixture harvest.
@@ -0,0 +1,132 @@
 # napalm-qnap-qts
 NAPALM driver for QNAP NAS systems running QTS, over SSH.
 QTS is a Linux distribution, so this driver inherits the whole OS surface from
 [`napalm-linux`](https://git.netork.io/christianmanivong/napalm-linux) —
 packages, services, users, processes, Docker — and adds QNAP's storage, QPKG
 and virtualisation layers on top.
 Tested against QTS 4.x and QTS 5.x.
 ## Status
 The class scaffold, discovery fingerprints and version/tool detection are in
 place. The storage, QPKG and VM parsers are **not written yet**: they are
 blocked on capturing real command output from hardware (see
 [Harvesting fixtures](#harvesting-fixtures)). Writing parsers against guessed
 output is how a driver ends up passing its own tests and failing on a real NAS.
 | Method | Source | Status |
 |---|---|---|
 | `get_facts` | `getcfg`, `getsysinfo`, `/proc/uptime` | pending harvest |
 | `get_interfaces`, `get_interfaces_ip` | `ip` / `ifconfig` | pending harvest |
 | `get_disks` | `qcli_storage -d`, `get_hd_smartinfo` | pending harvest |
 | `get_disk_pools` | `qcli_storage -p`, `/proc/mdstat` | pending harvest |
 | `get_volumes` | `qcli_storage -v`, `df` | pending harvest |
 | `get_shares` | `/etc/config/smb.conf`, `/etc/exports` | pending harvest |
 | `get_storage_services` | `getcfg`, `ss -lntup` | pending harvest |
 | `get_disk_smart` | `get_hd_smartinfo` | pending harvest |
 | `get_packages` | QPKG (`qpkg.conf` / `qpkg_cli`) | pending harvest |
 | `get_vms` | `virsh` (Virtualization Station) | pending harvest |
 | `start_vm`, `stop_vm`, `reboot_vm` | `virsh` | pending harvest |
 | `set_service_enabled` | `setcfg`, `/etc/init.d` | pending harvest |
 | `get_device_warnings` | derived from the above | pending harvest |
 | `get_docker_info` | inherited, via `_docker_bin()` | ✅ |
 | `get_services`, `get_users`, `get_processes` | inherited from `LinuxDriver` | ✅ |
 | `get_health_metrics` | inherited (UCD-MIB over SNMP) | ✅ |
 | `ping`, `ping_sweep` | inherited from `LinuxDriver` | ✅ |
 | Snapshots, quotas, replication, QPKG install/remove | — | out of scope for v1 |
 ## Requirements
 SSH must be enabled on the NAS: **Control Panel → Telnet/SSH → Allow SSH
 connection**. The driver connects on port 22 by default; the QTS web UI on
 443/8080 is not used.
 ## Install
 ```bash
 pip install -e vendor/napalm-device-types/ -e vendor/napalm-linux/ -e vendor/napalm-qnap-qts/
 ```
 ## Usage
 ```python
 from napalm_qnap_qts import QnapQtsDriver
 driver = QnapQtsDriver("nas.example.lan", "admin", "secret")
 driver.open()
 print(driver.get_facts())
 driver.close()
 ```
 Recognised `optional_args`: everything `napalm-linux` accepts (`port`,
 `sudo_password`, `secret`, …). Unknown keys are ignored.
 ## Design notes
 ### Inheritance order
 ```python
 class QnapQtsDriver(StorageDriver, LinuxDriver):
 ```
 `StorageDriver` precedes `LinuxDriver` in the MRO, so its
 `NotImplementedError` stubs shadow LinuxDriver's working implementations
 wherever the names collide — `get_services`, `get_packages`,
 `install_package`. Each collision is resolved with an explicit forwarding
 method; `TestMroForwarding` guards that they stay resolved.
 NAS services are exposed as `get_storage_services()`, not `get_services()`:
 netOrk's poller reads the former for the storage snapshot and the latter for
 the OS service list. Same convention as `napalm-openmediavault`.
 ### Device class
 The driver sets `DEVICE_CLASS = "storage"` explicitly. Without it netOrk's
 `issubclass` chain reaches `OSDriver` before `StorageDriver` and would file a
 QNAP under "linux", hiding its Storage tab. VMs and containers stay visible
 through capability introspection (`supports_vms`), not through this key — a
 QNAP running Virtualization Station is a NAS *and* a hypervisor, and
 `device_class` only holds one of those.
 ### Docker
 Container Station does not put `docker` on `PATH`; it lives under
 `/share/<pool>/.qpkg/container-station/`. The Docker *logic* stays in
 `LinuxDriver` and only the path is overridden here, via the `_docker_bin()`
 hook.
 ## Harvesting fixtures
 ```bash
 ./tools/harvest.sh admin@nas4.example.lan qts4
 ./tools/harvest.sh admin@nas5.example.lan qts5
 ./tools/sanitize.py tools/harvest-out/qts5 --extra-host mynas
 ```
 `harvest.sh` runs the command set this driver parses over a single SSH session
 and writes one file per command. `sanitize.py` replaces serial numbers, MACs,
 IP addresses and hostnames with stable placeholders — cross-references between
 files survive, so the fixtures still describe one coherent device.
 **Read the sanitised output before committing it.** The sanitiser catches
 patterns, not judgement, and fixtures live in git forever. Raw harvest output
 is gitignored and must never be committed.
 Where QTS 4 and QTS 5 differ, keep both fixtures and parametrise the test over
 the pair, so the version divergence is part of the test matrix rather than a
 later surprise.
 ## Development
 ```bash
 pip install -e ".[dev]"
 python -m pytest -q
 ruff check .
 ```
 ## License
 Apache-2.0
@@ -0,0 +1,5 @@
 """napalm-qnap-qts – NAPALM driver for QNAP NAS systems running QTS."""
 from napalm_qnap_qts.qnap_qts import QnapQtsDriver
 __all__ = ["QnapQtsDriver"]
@@ -0,0 +1,156 @@
 # Licensed under the Apache License, Version 2.0
 """NAPALM driver for QNAP NAS systems running QTS.
 QTS is a Linux distribution, so this driver inherits the whole OS surface from
 :class:`napalm_linux.linux.LinuxDriver` — packages, services, users, processes,
 Docker — and adds QNAP's own storage, QPKG and virtualisation layers on top:
 * Physical disk inventory and SMART via ``qcli_storage`` and ``get_hd_smartinfo``
 * Storage pools and volumes via ``qcli_storage`` and ``/proc/mdstat``
 * SMB/NFS/AFP/FTP shares from ``/etc/config/smb.conf`` and ``/etc/exports``
 * QPKG packages instead of a distribution package manager
 * Virtualization Station guests via ``virsh``
 * Container Station via LinuxDriver's Docker support, redirected to the
  QPKG-local ``docker`` binary
 Connects via SSH. SSH has to be enabled on the NAS first
 (Control Panel → Telnet/SSH). Tested against QTS 4.x and QTS 5.x.
 """
 from __future__ import annotations
 import re
 from typing import Any
 from napalm_device_types import FingerprintRule, PortSpec, StorageDriver
 from napalm_linux.linux import LinuxDriver
 #: QNAP Systems' IANA enterprise number. Used by discovery to recognise a NAS
 #: from its SNMP sysObjectID before anyone has supplied credentials.
 QNAP_ENTERPRISE_OID = "1.3.6.1.4.1.24681"
 #: Where Container Station puts the docker binary. It is not on PATH, and the
 #: volume name varies with which pool the app was installed on, so this is a
 #: glob evaluated on the device rather than a fixed path.
 _DOCKER_GLOBS = (
    "/share/*/.qpkg/container-station/bin/docker",
    "/share/*/.qpkg/container-station/usr/bin/docker",
 )
 _VERSION_RE = re.compile(r"(\d+)\.")
 class QnapQtsDriver(StorageDriver, LinuxDriver):
    """NAPALM driver for QNAP NAS systems running QTS.
    **Inheritance order matters.** ``StorageDriver`` precedes ``LinuxDriver`` in
    the MRO, so its ``NotImplementedError`` stubs shadow LinuxDriver's working
    implementations wherever the names collide — ``get_services``,
    ``get_packages`` and ``install_package``. Each collision is resolved
    explicitly below rather than left to the MRO; see ``TestMroForwarding``.
    NAS services are exposed as ``get_storage_services()``, not
    ``get_services()``, because netOrk's poller reads the former for the storage
    snapshot and the latter for the OS service list. Same convention as
    napalm-openmediavault.
    """
    TYPE_LABEL = "Storage"
    # Declared outright: the driver inherits LinuxDriver for the OS surface, so
    # netOrk's issubclass chain would reach OSDriver first and classify a QNAP
    # as "linux", hiding its Storage tab. VMs and containers stay visible
    # through capability introspection, not through this key.
    DEVICE_CLASS = "storage"
    VENDOR = "QNAP"
    DRIVER_NAME = "qnap_qts"
    driver_name = "qnap_qts"
    NETMIKO_DEVICE_TYPE = "linux"
    SNMP_OBJECT_ID_PREFIX = QNAP_ENTERPRISE_OID
    SNMP_FINGERPRINT = [
        FingerprintRule("qnap", weight=9.0),
        FingerprintRule("nas", weight=1.0),
    ]
    # QTS answers with a stock OpenSSH banner, so SSH alone cannot identify a
    # QNAP — it only confirms the transport this driver needs.
    SSH_FINGERPRINT = [
        FingerprintRule("openssh", weight=1.0),
    ]
    # Mandatory: without it every unidentified NAS web UI would score as a QNAP.
    HTTP_FINGERPRINT = [
        FingerprintRule("qnap", weight=9.0, mandatory=True),
        FingerprintRule("qts", weight=4.0),
    ]
    PORT_SPECS = [
        PortSpec("https", 443),
        PortSpec("http", 8080),
    ]
    # ── Lifecycle ─────────────────────────────────────────────────────────────
    def open(self) -> None:
        """Connect, then resolve the facts that decide which code paths run.
        QTS 4 and QTS 5 differ in the output of several tools, and the QPKG apps
        that provide docker and virsh live on whichever storage pool they were
        installed on. Resolving all of that once per session keeps every getter
        below free of discovery round trips.
        """
        super().open()
        self._qts_major = self._detect_qts_major()
        self._docker_path = self._discover_docker_path()
        self._virsh_path = self._discover_virsh_path()
    def _detect_qts_major(self) -> int | None:
        """Return the QTS major version, or None when it cannot be read.
        Deliberately not fatal: a NAS that answers nothing useful here is still
        worth polling, and the newer code path is the better default.
        """
        try:
            raw = self._send("getcfg System Version").strip()
        except Exception:
            return None
        match = _VERSION_RE.match(raw)
        return int(match.group(1)) if match else None
    def _discover_docker_path(self) -> str:
        """Locate Container Station's docker binary, falling back to PATH."""
        try:
            found = self._send(f"ls {' '.join(_DOCKER_GLOBS)} 2>/dev/null | head -1").strip()
        except Exception:
            return "docker"
        return found.splitlines()[0].strip() if found else "docker"
    def _discover_virsh_path(self) -> str | None:
        """Locate Virtualization Station's virsh, or None when it is not installed.
        None is a normal outcome — plenty of QNAP models never run VMs — and it
        is what makes ``get_vms`` return an empty list instead of raising.
        """
        try:
            found = self._send(
                "ls /share/*/.qpkg/QKVM/usr/bin/virsh /share/*/.qpkg/*/bin/virsh "
                "2>/dev/null | head -1"
            ).strip()
        except Exception:
            return None
        return found.splitlines()[0].strip() or None if found else None
    def _docker_bin(self) -> str:
        """Override LinuxDriver's hook: docker is not on PATH under QTS."""
        return getattr(self, "_docker_path", None) or "docker"
    # ── MRO collision resolution ──────────────────────────────────────────────
    #
    # StorageDriver comes first in the MRO and its stubs would otherwise win.
    def get_services(self) -> Any:
        """OS services, not NAS services — this is what netOrk's poller reads.
        Forwarded explicitly past ``StorageDriver.get_services``, which shadows
        it and returns a different shape (dict of NAS services vs. list of OS
        services).
        """
        return LinuxDriver.get_services(self)
@@ -0,0 +1,81 @@
 [build-system]
 requires = ["setuptools>=68", "wheel"]
 build-backend = "setuptools.build_meta"
 [project]
 name = "napalm-qnap-qts"
 version = "0.1.0"
 description = "NAPALM driver for QNAP NAS systems running QTS via SSH"
 readme = "README.md"
 requires-python = ">=3.9"
 license = { text = "Apache-2.0" }
 authors = [
    { name = "Christian Manivong", email = "christian@manivong.de" },
 ]
 keywords = [
    "napalm",
    "network",
    "automation",
    "qnap",
    "qts",
    "nas",
    "storage",
    "ssh",
    "driver",
 ]
 classifiers = [
    "Development Status :: 3 - Alpha",
    "Intended Audience :: Developers",
    "Intended Audience :: System Administrators",
    "License :: OSI Approved :: Apache Software License",
    "Operating System :: OS Independent",
    "Programming Language :: Python :: 3",
    "Programming Language :: Python :: 3.9",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
    "Topic :: System :: Networking",
    "Topic :: System :: Systems Administration",
    "Typing :: Typed",
 ]
 # napalm-linux is a hard dependency, not an optional one: QTS is a Linux system
 # and this driver inherits the whole OS surface (packages, services, users,
 # processes, Docker) from LinuxDriver rather than reimplementing it.
 dependencies = [
    "napalm>=4.0",
    "napalm-device-types>=0.5.0",
    "napalm-linux>=0.1.0",
    "netmiko>=4.0.0",
    "paramiko>=5.0.0",  # CVE-2026-44405
 ]
 [project.optional-dependencies]
 dev = [
    "pytest",
    "pytest-cov",
    "black",
    "ruff",
    "mypy",
 ]
 [project.entry-points."napalm.drivers"]
 qnap_qts = "napalm_qnap_qts:QnapQtsDriver"
 [project.urls]
 Repository = "https://git.netork.io/christianmanivong/napalm-qnap-qts"
 [tool.setuptools.packages.find]
 where = ["."]
 include = ["napalm_qnap_qts*"]
 [tool.pytest.ini_options]
 testpaths = ["tests"]
 [tool.ruff]
 line-length = 100
 target-version = "py39"
 # Same rule set the netOrk repo gates on, so this driver is held to the
 # standard of the project that consumes it rather than to ruff's defaults.
 [tool.ruff.lint]
 select = ["E", "F", "I", "UP"]
@@ -0,0 +1,165 @@
 """Unit tests for the QNAP QTS driver.
 Command-output fixtures are captured from real hardware via tools/harvest.sh —
 one QTS 4 box and one QTS 5 box — and pasted in as module constants. Parsers are
 written against those, never against guessed output.
 """
 from __future__ import annotations
 from unittest.mock import MagicMock, patch
 import pytest
 from napalm_qnap_qts import QnapQtsDriver
 #: Parsers need real command output to be written against. These tests are the
 #: specification for the work that tools/harvest.sh unblocks. strict=True means
 #: the suite goes red the moment one starts passing, which is the reminder to
 #: delete the marker rather than leave it lying around.
 _PENDING_HARVEST = "blocked on fixture harvest from real QTS 4 / QTS 5 hardware"
@pytest.fixture()
 def driver():
    """A driver with its transport mocked out, bypassing __init__.
    Same shape as napalm-linux's fixture: nothing in __init__ needs patching,
    so constructing the object by hand is cheaper and clearer than mocking
    ConnectHandler.
    """
    d = QnapQtsDriver.__new__(QnapQtsDriver)
    d.hostname = "testnas"
    d.username = "admin"
    d.password = "pass"  # noqa: S105
    d.timeout = 60
    d.port = 22
    d._secret = "pass"  # noqa: S105
    d._forced_pkg_manager = None
    d._pkg_manager = None
    d._sudo_password = None
    d.netmiko_optional_args = {}
    d._device = MagicMock()
    d._qts_major = 5
    d._docker_path = "docker"
    d._virsh_path = None
    return d
 class TestDriverIdentity:
    """Read without a connection by netOrk's discovery and /drivers endpoint."""
    def test_driver_name(self):
        assert QnapQtsDriver.DRIVER_NAME == "qnap_qts"
    def test_lowercase_driver_name_alias_matches(self):
        """netOrk's register_driver path reads the lowercase attribute."""
        assert QnapQtsDriver.driver_name == QnapQtsDriver.DRIVER_NAME
    def test_vendor(self):
        assert QnapQtsDriver.VENDOR == "QNAP"
    def test_type_label_is_storage(self):
        assert QnapQtsDriver.TYPE_LABEL == "Storage"
    def test_declares_device_class_explicitly(self):
        """Inheriting LinuxDriver would otherwise get it classified as "linux",
        and netOrk would hide the Storage tab."""
        assert QnapQtsDriver.DEVICE_CLASS == "storage"
    def test_declares_at_least_one_fingerprint_source(self):
        """Discovery silently skips a driver that declares no fingerprint data."""
        assert (
            QnapQtsDriver.HTTP_FINGERPRINT
            or QnapQtsDriver.SNMP_FINGERPRINT
            or QnapQtsDriver.SSH_FINGERPRINT
        )
    def test_snmp_object_id_is_the_qnap_enterprise_oid(self):
        assert QnapQtsDriver.SNMP_OBJECT_ID_PREFIX == "1.3.6.1.4.1.24681"
    def test_http_fingerprint_is_mandatory_to_avoid_matching_any_nas(self):
        patterns = {r.pattern: r for r in QnapQtsDriver.HTTP_FINGERPRINT}
        assert patterns["qnap"].mandatory is True
 class TestMroForwarding:
    """StorageDriver precedes LinuxDriver in the MRO, so its NotImplementedError
    stubs shadow LinuxDriver's working implementations. Every collision has to be
    resolved deliberately — this is the class of bug that makes a driver look
    fine until it runs against hardware.
    """
    def test_get_services_returns_the_linux_os_service_list(self, driver):
        """netOrk's poller expects a list here (OS services). StorageDriver's
        stub would return a dict of NAS services, if it returned anything."""
        from napalm_linux.linux import LinuxDriver
        with patch.object(LinuxDriver, "get_services", return_value=[{"name": "sshd"}]) as m:
            result = driver.get_services()
        assert m.called
        assert result == [{"name": "sshd"}]
    @pytest.mark.xfail(strict=True, reason=_PENDING_HARVEST)
    def test_nas_services_live_under_a_separate_name(self):
        """get_storage_services is what netOrk's _collect.py actually reads for
        the storage snapshot — get_services is the OS list."""
        assert hasattr(QnapQtsDriver, "get_storage_services")
    @pytest.mark.xfail(strict=True, reason=_PENDING_HARVEST)
    def test_get_packages_is_not_the_storage_stub(self):
        from napalm_device_types import StorageDriver
        assert QnapQtsDriver.get_packages is not StorageDriver.get_packages
    @pytest.mark.parametrize(
        ("method", "args"),
        [
            ("install_package", ("qpkg-name",)),
            ("remove_package", ("qpkg-name",)),
            ("snapshot_create", ("DataVol1", "snap1")),
        ],
    )
    def test_out_of_scope_writers_still_raise(self, driver, method, args):
        """v1 is read-only plus safe actions. These must fail loudly rather than
        appear supported — netOrk's poller catches NotImplementedError."""
        with pytest.raises(NotImplementedError):
            getattr(driver, method)(*args)
 class TestQtsVersionDetection:
    def test_parses_major_version(self, driver):
        with patch.object(driver, "_send", return_value="5.1.5"):
            assert driver._detect_qts_major() == 5
    def test_parses_qts_four(self, driver):
        with patch.object(driver, "_send", return_value="4.5.4"):
            assert driver._detect_qts_major() == 4
    def test_tolerates_a_build_suffix(self, driver):
        with patch.object(driver, "_send", return_value="5.2.0.2782 (2026/03/14)"):
            assert driver._detect_qts_major() == 5
    def test_unreadable_version_does_not_raise(self, driver):
        """An unknown version must degrade to the newer code path, not abort the
        poll — a NAS answering nothing useful here is still worth polling."""
        with patch.object(driver, "_send", return_value=""):
            assert driver._detect_qts_major() is None
 class TestDockerBinDiscovery:
    """Container Station does not put docker on PATH."""
    def test_uses_the_container_station_path_when_present(self, driver):
        found = "/share/CACHEDEV1_DATA/.qpkg/container-station/bin/docker"
        with patch.object(driver, "_send", return_value=found):
            assert driver._discover_docker_path() == found
    def test_falls_back_to_plain_docker_when_nothing_is_found(self, driver):
        with patch.object(driver, "_send", return_value=""):
            assert driver._discover_docker_path() == "docker"
    def test_docker_bin_hook_returns_the_discovered_path(self, driver):
        driver._docker_path = "/opt/docker"
        assert driver._docker_bin() == "/opt/docker"
@@ -0,0 +1,84 @@
 """The harvest sanitiser (tools/sanitize.py).
 Fixtures are committed forever, so a miss here puts a real NAS's serial numbers
 and addresses into git history. Worth testing even though it is a dev tool.
 """
 from __future__ import annotations
 import importlib.util
 import pathlib
 import pytest
 _SPEC = importlib.util.spec_from_file_location(
    "qnap_sanitize", pathlib.Path(__file__).parent.parent / "tools" / "sanitize.py"
 )
 assert _SPEC and _SPEC.loader
 sanitize = importlib.util.module_from_spec(_SPEC)
 _SPEC.loader.exec_module(sanitize)
@pytest.fixture()
 def scrubber():
    return sanitize.Scrubber()
 class TestSerials:
    def test_labelled_serial_is_replaced(self, scrubber):
        out = scrubber.scrub("Serial Number: WD-WCC4N7RTKZ9P")
        assert "WD-WCC4N7RTKZ9P" not in out
        assert "SERIAL001XXXX" in out
    def test_lsblk_serial_field_is_replaced(self, scrubber):
        out = scrubber.scrub('NAME="sda" MODEL="WD40EFRX" SERIAL="WD-WCC4N7RTKZ9P"')
        assert "WD-WCC4N7RTKZ9P" not in out
        assert 'MODEL="WD40EFRX"' in out, "model names are not identifying and must survive"
    def test_same_serial_maps_to_the_same_placeholder(self, scrubber):
        """A disk serial appears in several harvested files; the cross-reference
        has to survive or the fixtures stop describing one coherent device."""
        first = scrubber.scrub("serial: ABC123456")
        second = scrubber.scrub('SERIAL="ABC123456"')
        assert "SERIAL001XXXX" in first
        assert "SERIAL001XXXX" in second
    def test_distinct_serials_get_distinct_placeholders(self, scrubber):
        out = scrubber.scrub("sn: AAA111222\nsn: BBB333444")
        assert "SERIAL001XXXX" in out
        assert "SERIAL002XXXX" in out
 class TestNetworkIdentifiers:
    def test_mac_is_replaced(self, scrubber):
        out = scrubber.scrub("link/ether 24:5e:be:11:22:33 brd ff:ff:ff:ff:ff:ff")
        assert "24:5e:be:11:22:33" not in out
        assert "00:11:22:33:44:" in out
    def test_ipv4_is_replaced(self, scrubber):
        out = scrubber.scrub("inet 10.7.224.12/24")
        assert "10.7.224.12" not in out
        assert "192.0.2.1" in out
    def test_loopback_and_wildcard_survive(self, scrubber):
        """Replacing these makes fixtures unreadable and reveals nothing."""
        out = scrubber.scrub("0.0.0.0:445 127.0.0.1:22")
        assert "0.0.0.0" in out
        assert "127.0.0.1" in out
    def test_same_ip_maps_consistently(self, scrubber):
        out = scrubber.scrub("gw 10.0.0.1\nvia 10.0.0.1")
        assert out.count("192.0.2.1") == 2
 class TestHostnames:
    def test_extra_host_is_replaced_case_insensitively(self):
        s = sanitize.Scrubber(["MyNAS"])
        out = s.scrub("Server Name = mynas\nhost MYNAS ok")
        assert "mynas" not in out.lower().replace("testnas", "")
        assert out.count("testnas") == 2
    def test_empty_host_entry_is_ignored(self):
        """An unset --extra-host must not turn into a regex that matches everything."""
        s = sanitize.Scrubber(["", None])
        assert s.scrub("untouched text") == "untouched text"
@@ -0,0 +1,105 @@
 #!/usr/bin/env bash
 # Collect the command outputs this driver parses, from a real QNAP NAS.
 #
 # Run once against a QTS 4 box and once against a QTS 5 box; the resulting
 # files become the test fixtures. Writing parsers against guessed output is how
 # a driver ends up passing its own tests and failing on hardware, so this runs
 # before any parser code.
 #
 #   ./tools/harvest.sh admin@nas1.example.lan qts4
 #   ./tools/harvest.sh admin@nas2.example.lan qts5
 #
 # Output lands in tools/harvest-out/<label>/ (gitignored). Sanitise with
 # ./tools/sanitize.py before pasting anything into a test file.
 set -uo pipefail
 TARGET="${1:?usage: harvest.sh <user@host> <label> [ssh-port]}"
 LABEL="${2:?usage: harvest.sh <user@host> <label> [ssh-port]}"
 PORT="${3:-22}"
 OUT="$(dirname "$0")/harvest-out/$LABEL"
 mkdir -p "$OUT"
 # One SSH session for the whole run: QTS is slow to authenticate, and opening a
 # connection per command turns a 30-second harvest into several minutes.
 CTL="$(mktemp -u /tmp/qnap-harvest-XXXXXX)"
 cleanup() { ssh -O exit -o ControlPath="$CTL" "$TARGET" 2>/dev/null || true; }
 trap cleanup EXIT
 ssh -M -o ControlPath="$CTL" -o ControlPersist=120 -p "$PORT" -N "$TARGET" &
 sleep 3
 run() {
    local name="$1"; shift
    local cmd="$*"
    printf '  %-28s %s\n' "$name" "$cmd"
    {
        echo "### command: $cmd"
        ssh -o ControlPath="$CTL" -p "$PORT" "$TARGET" "$cmd" 2>&1
        echo "### exit: $?"
    } > "$OUT/$name.txt"
 }
 echo "Harvesting $TARGET -> $OUT"
 # ── Identity / facts ──────────────────────────────────────────────────────────
 run version            'getcfg System Version'
 run build              'getcfg System "Build Number"'
 run model              'getcfg System Model'
 run servername         'getcfg System "Server Name"'
 run getsysinfo_model   'getsysinfo model'
 run getsysinfo_serial  'getsysinfo serial'
 run uptime             'cat /proc/uptime'
 run uname              'uname -a'
 run ulinux_system      'sed -n "/^\[System\]/,/^\[/p" /etc/config/uLinux.conf'
 # ── Interfaces ────────────────────────────────────────────────────────────────
 run ip_link            'ip -o link show'
 run ip_addr            'ip -o addr show'
 run ifconfig           'ifconfig -a'
 run ulinux_network     'sed -n "/^\[Network\]/,/^\[/p" /etc/config/uLinux.conf'
 # ── Storage ───────────────────────────────────────────────────────────────────
 run qcli_storage       'qcli_storage'
 run qcli_storage_d     'qcli_storage -d'
 run qcli_storage_v     'qcli_storage -v'
 run qcli_storage_p     'qcli_storage -p'
 run qcli_storage_t     'qcli_storage -t'
 run mdstat             'cat /proc/mdstat'
 run volume_conf        'cat /etc/volume.conf'
 run enclosure_conf     'cat /etc/enclosure_0.conf'
 run df                 'df -B1 -P'
 run hdnum              'getsysinfo hdnum'
 run hdtmp_all          'for i in $(seq 1 8); do echo "== slot $i"; getsysinfo hdtmp $i; done'
 run hdstatus_all       'for i in $(seq 1 8); do echo "== slot $i"; getsysinfo hdstatus $i; done'
 run smartinfo_1        'get_hd_smartinfo -d 1'
 run smartinfo_2        'get_hd_smartinfo -d 2'
 run lsblk              'lsblk -b -P -o NAME,MODEL,SERIAL,SIZE,ROTA,TYPE'
 # ── Shares & services ─────────────────────────────────────────────────────────
 run smb_conf           'cat /etc/config/smb.conf'
 run exports            'cat /etc/exports'
 run ss_listen          'ss -lntup'
 run svc_samba          'getcfg Samba Enable'
 run svc_nfs            'getcfg NFS Enable'
 run svc_ftp            'getcfg FTP Enable'
 run svc_afp            'getcfg AFP Enable'
 run svc_iscsi          'getcfg iSCSI Enable'
 run init_d             'ls /etc/init.d/'
 # ── Packages (QPKG) ───────────────────────────────────────────────────────────
 run qpkg_conf          'cat /etc/config/qpkg.conf'
 run qpkg_cli           'qpkg_cli -L'
 run qpkg_dir           'ls -d /share/*/.qpkg/*'
 # ── Virtualisation & containers ───────────────────────────────────────────────
 run virsh_which        'ls /share/*/.qpkg/QKVM/usr/bin/virsh /share/*/.qpkg/*/bin/virsh 2>&1'
 run virsh_list         'virsh list --all'
 run docker_which       'ls /share/*/.qpkg/container-station/bin/docker /share/*/.qpkg/container-station/usr/bin/docker 2>&1'
 run docker_ps          'docker ps -a --no-trunc --format "{{json .}}"'
 # ── Firmware / updates ────────────────────────────────────────────────────────
 run fw_update_cfg      'sed -n "/^\[FIRMWARE UPDATE\]/,/^\[/p" /etc/config/uLinux.conf'
 echo
 echo "Done. $(ls "$OUT" | wc -l) files in $OUT"
 echo "Next: ./tools/sanitize.py $OUT"
@@ -0,0 +1,127 @@
 #!/usr/bin/env python3
 """Scrub identifying data out of harvested QNAP output before it becomes a fixture.
 Fixtures live in a git repo forever. Serial numbers, MACs, public IPs and
 hostnames from a real NAS have no business in one, and a reviewer cannot be
 expected to spot every one of them by eye.
 Replacements are stable within a run — the same serial always becomes the same
 placeholder — so cross-references between files (a disk serial appearing in both
 qcli_storage and get_hd_smartinfo) survive and the fixtures stay coherent.
    ./tools/sanitize.py tools/harvest-out/qts5
    ./tools/sanitize.py tools/harvest-out/qts5 --extra-host mynas --in-place
 """
 from __future__ import annotations
 import argparse
 import pathlib
 import re
 import sys
 # QNAP serials are alphanumeric runs; matching them generically would eat model
 # names, so they are found via their labelled context instead.
 SERIAL_CONTEXT = re.compile(
    r"(?i)\b(serial(?:\s*number)?|serial_no|sn)\b(\s*[:=]\s*|\s+)([A-Z0-9][A-Z0-9\-]{5,})"
 )
 DISK_SERIAL_FIELD = re.compile(r'(?i)\bSERIAL="([^"]*)"')
 # One alternation rather than three passes. A MAC is also a valid match for the
 # IPv6 pattern, so separate passes let the second one rewrite what the first
 # just produced — and the placeholder MAC came back out as an IPv6 address.
 # Alternation order is the precedence: most specific first.
 ADDRESS = re.compile(
    r"(?P<mac>\b(?:[0-9A-Fa-f]{2}:){5}[0-9A-Fa-f]{2}\b)"
    r"|(?P<ipv6>\b(?:[0-9A-Fa-f]{0,4}:){3,7}[0-9A-Fa-f]{0,4}\b)"
    r"|(?P<ipv4>\b(?:\d{1,3}\.){3}\d{1,3}\b)"
 )
 # Addresses that carry no information about the owner and make fixtures harder
 # to read if replaced.
 KEEP_IPS = {"0.0.0.0", "127.0.0.1", "255.255.255.255"}
 class Scrubber:
    def __init__(self, extra_hosts: list[str] | None = None) -> None:
        self._maps: dict[str, dict[str, str]] = {}
        self._extra_hosts = [h for h in (extra_hosts or []) if h]
    def _placeholder(self, kind: str, value: str, template: str) -> str:
        bucket = self._maps.setdefault(kind, {})
        if value not in bucket:
            bucket[value] = template.format(n=len(bucket) + 1)
        return bucket[value]
    def _serial(self, value: str) -> str:
        return self._placeholder("serial", value, "SERIAL{n:03d}XXXX")
    def _mac(self, value: str) -> str:
        n = self._placeholder("mac", value.lower(), "{n}")
        return f"00:11:22:33:44:{int(n):02x}"
    def _ip(self, value: str) -> str:
        return self._placeholder("ip", value, "192.0.2.{n}")
    def _address(self, match: re.Match) -> str:
        if match.group("mac"):
            return self._mac(match.group("mac"))
        if match.group("ipv6"):
            return "2001:db8::1"
        value = match.group("ipv4")
        return value if value in KEEP_IPS else self._ip(value)
    def scrub(self, text: str) -> str:
        text = SERIAL_CONTEXT.sub(
            lambda m: f"{m.group(1)}{m.group(2)}{self._serial(m.group(3))}", text
        )
        text = DISK_SERIAL_FIELD.sub(lambda m: f'SERIAL="{self._serial(m.group(1))}"', text)
        text = ADDRESS.sub(self._address, text)
        for host in self._extra_hosts:
            text = re.sub(rf"(?i)\b{re.escape(host)}\b", "testnas", text)
        return text
 def main(argv: list[str] | None = None) -> int:
    ap = argparse.ArgumentParser(description=__doc__)
    ap.add_argument("directory", type=pathlib.Path)
    ap.add_argument(
        "--extra-host",
        action="append",
        default=[],
        help="hostname or share name to replace with 'testnas' (repeatable)",
    )
    ap.add_argument(
        "--in-place",
        action="store_true",
        help="overwrite the harvested files instead of writing a .sanitised sibling dir",
    )
    args = ap.parse_args(argv)
    if not args.directory.is_dir():
        print(f"not a directory: {args.directory}", file=sys.stderr)
        return 2
    scrubber = Scrubber(args.extra_host)
    dest = (
        args.directory
        if args.in_place
        else args.directory.with_name(args.directory.name + ".sanitised")
    )
    dest.mkdir(parents=True, exist_ok=True)
    count = 0
    for path in sorted(args.directory.glob("*.txt")):
        cleaned = scrubber.scrub(path.read_text(errors="replace"))
        (dest / path.name).write_text(cleaned)
        count += 1
    replaced = {k: len(v) for k, v in scrubber._maps.items()}
    print(f"sanitised {count} file(s) -> {dest}")
    print(f"replaced: {replaced or 'nothing'}")
    print("Read the output before committing — this catches patterns, not judgement.")
    return 0
 if __name__ == "__main__":
    raise SystemExit(main())