docs: a structure that keeps device-types vendor- and product-agnostic

Roles and capabilities declare, templates run declared steps, reconcilers
stay free of I/O. POSIX host readers, the SSH exec transport and the
privilege layer move to a new package, napalm-unix. SNMP polling, ping
sweeps, discovery scoring, guest profiles, display labels and warnings
belong to the application; drivers declare SNMP OIDs as data and report
facts. Fixes become templates whose values come from the caller.

Says in which order this happens and where the open proposals #8-#14 and
#22 go instead.

docs/STRUCTURE.md

@@ -0,0 +1,198 @@
+# Structure
+
+Status: agreed plan. Nothing here is implemented yet; the migration at the end says in
+which order it will be.
+
+## Principle
+
+**napalm-device-types is vendor-, OS-, protocol- and product-agnostic.** It holds the
+contracts every driver implements and the generic logic that works unchanged for every
+driver of a role. It knows no shell, no paramiko, no package manager, no container CLI,
+no SNMP OID, no cloud-init and no application that uses the drivers.
+
+That rules out three kinds of code that are here today:
+
+- **How a kind of host is read.** Shell commands and their parsers for POSIX hosts move to
+  a new package, **napalm-unix**, which napalm-linux and napalm-bsd depend on.
+- **What an application does with a device.** Polling a device over SNMP, sweeping a range
+  with pings, scoring discovery evidence, choosing what a provisioned guest installs, and
+  deciding what counts as a problem belong to the application, not to the drivers.
+- **Product naming.** Files, markers and messages a driver leaves on a device carry the
+  driver's name or none, never the name of the application that called it.
+
+```
+napalm-device-types        roles, capabilities, templates, models, reconcilers  <- every driver
+        ^
+napalm-unix                POSIX host readers, SSH exec transport, privilege    <- napalm-linux, napalm-bsd
+        ^
+napalm-linux, napalm-bsd   connection, platform getters, platform steps
+
+application                SNMP polling, ping sweeps, discovery scoring, guest
+                           profiles, warnings, and the values every fix is given
+```
+
+## What stays in napalm-device-types
+
+| Kind | Rule | Modules |
+|---|---|---|
+| **Role** | Declares only, under `TYPE_CHECKING`. No runtime method. | `access_point`, `switch`, `firewall`, `hypervisor`, `os`, `storage`, `phone`, `media`, `residential_gateway`, `management_controller` |
+| **Capability** | Declares only. | `packages`, `services`, `updates`, `nat_vpn`, `mac_acl`, `config_lifecycle`, `interface_filter`, `host_reboot` |
+| **Template** | A concrete method that runs a fixed sequence of abstract steps. A driver implements the steps, or overrides the whole method when its device does it in one go. All values come from the caller. | the fix routines below |
+| **Contract types** | Interfaces and data shapes other packages implement or fill. | `CommandChannelMixin`, `CommandResult`, `ByteStream`; the declaration of `ContainerEngineMixin`; the reader mixins' public method declarations; the SNMP profile type; `models` |
+| **Reconciler** | Concrete and vendor-neutral, without I/O of its own. | `firewall_rules` and `dhcp` diff/apply, `_matching`, `lag` |
+| **Introspection** | Concrete, without I/O. | `roles`; `base` (`FingerprintRule`, `PortSpec`, the fingerprint attributes) |
+
+### Fix routines are templates
+
+A change the application wants made on a device, for example "run an SNMP agent that
+answers us", is one template per purpose. The template fixes the order of the steps; the
+driver knows how each step is done on its device; the caller supplies every value.
+
+```python
+class SnmpAgentMixin:
+    def configure_snmp_agent(self, settings: SnmpAgentSettingsDict) -> FixResultDict:
+        steps = [self._install_snmp_agent, self._write_snmp_agent_config,
+                 self._enable_snmp_agent, self._verify_snmp_agent]
+        ...                                   # runs them in order, stops at the first failure
+
+    # declared only; napalm-unix implements them for net-snmp
+    def _install_snmp_agent(self) -> FixStepDict: ...
+    def _write_snmp_agent_config(self, settings: SnmpAgentSettingsDict) -> FixStepDict: ...
+    def _enable_snmp_agent(self) -> FixStepDict: ...
+    def _verify_snmp_agent(self, settings: SnmpAgentSettingsDict) -> FixStepDict: ...
+```
+
+napalm-unix implements the four steps for net-snmp, for Linux and the BSDs alike. A REST
+driver such as OPNsense overrides `configure_snmp_agent` as a whole, because its device
+takes the whole configuration in one call. The community, the allowed sources, contact
+and location are the caller's.
+
+The same shape replaces every action in today's `run_device_action`:
+
+| Today | Template |
+|---|---|
+| `fix_snmp` | `configure_snmp_agent(settings)` |
+| `fix_apt_proxy` | `configure_package_proxy(url)` |
+| `fix_docker_permissions` | `grant_container_engine_access(user)` |
+| `install_lldpd`, `install_auc` | `ensure_package(name)` |
+
+`hasattr(driver, "configure_snmp_agent")` stays a truthful answer: the mixin is mixed in
+by the driver, or by the platform package, not by a role base.
+
+### SNMP: drivers declare data, the application polls
+
+The drivers do not collect over SNMP. A driver whose device answers the standard UCD-MIB
+and IF-MIB declares nothing. A driver whose device publishes the same values under
+proprietary OIDs declares them as data, in a class attribute of a type defined here:
+
+```python
+class ProcurveDriver(SwitchDriver):
+    SNMP_HEALTH_PROFILE: ClassVar[SnmpHealthProfile] = {
+        "uptime": "1.3.6.1.4.1.11.2.14.11.1.2.6.1.0", ...
+    }
+```
+
+Polling, the standard OIDs, the counter arithmetic and the interfaces worth skipping are
+the application's.
+
+### Warnings are facts
+
+A driver reports facts: the proxy a host is configured with, a DSL line's SNR, the
+firmware version it runs and the one it is offered. It does not compare them with a
+setting of the application or with a threshold, and it does not name a warning.
+
+## What moves to napalm-unix
+
+| Module | Knows |
+|---|---|
+| `host_status` | apt, dnf, dpkg, systemd timers, `df` |
+| `kernel` | `/proc`, `/lib/modules`, gzip, base64 |
+| `listening` | `ss`, `netstat`, `/proc/<pid>/cgroup` |
+| `systemd` | `systemctl` |
+| `virtual_ips` | vip-manager, keepalived, awk |
+| `system_identity` | SMBIOS via sysfs and `dmidecode` |
+| `package_updates` | apt and dnf output |
+| `ssh_server` | OpenSSH (`sshd -T`, `sshd -t`, `sshd_config`); which algorithms to drop stays the caller's |
+| `terminal` | ANSI escape codes in shell output |
+| `channel`: `run_on_transport`, `open_stream_on_transport`, `ParamikoExecStream` | paramiko exec channels |
+| `container_engine`: `ContainerEngineConnection`, engine probing | the Docker CLI, `system dial-stdio` |
+
+It also takes what napalm-linux and napalm-bsd each carry today:
+
+- **Privilege:** one `sudo -S` / `sudo -n` / root layer, including the rule that a
+  privileged command never gets stdin while a sudo password is set, and one way to write a
+  file as root (stage with `mktemp`, then `install`). Two copies of this have already
+  drifted apart.
+- **Parsers:** `/etc/passwd` with `/etc/group`, crontab lines, `ps` with `lstart`.
+- **Fix steps:** net-snmp, the package proxy, container engine group membership.
+
+The FreeBSD part of `host_status` (`freebsd-version -k` against `-r`) moves to napalm-bsd.
+
+In napalm-unix every reader follows one pattern:
+
+- **Shape:** `FOO_COMMAND`, `parse_foo(output)`, and `FooMixin.get_foo()`.
+- **Reaching the host:** the mixin uses `run_command(command, privileged=...)` from
+  `CommandChannelMixin`. The six per-reader `_run_*` hooks, with four different signatures,
+  go away.
+- **Framing:** one frame helper writes and parses the `BEGIN`/`END` markers for every reader.
+- **Errors:** one exception for "cannot read" and one for "not on this host". A read that
+  fails or is cut short raises; it never reads as "nothing there".
+- **Names on the host:** temporary files, markers and drop-ins carry `napalm` or the
+  driver's name. A drop-in written under an older name is still recognised and replaced.
+
+Layout: `napalm_unix/{posix,linux,freebsd,openssh,transport}/`.
+
+## What moves to the application
+
+| Today | Why it is not a driver's |
+|---|---|
+| `health_metrics`, `_ucd_metrics` | Polling over SNMP is something the application does to a device; the driver is not even connected while it runs. |
+| `ping_sweep` | Looping over addresses, capping the targets and reporting progress is a scan. A device with a faster batch keeps it as a capability of its own. |
+| `rules_max_weight` | Scoring is how one application ranks drivers. The rules themselves stay with the driver. |
+| `provisioning` (`QEMU_GUEST_AGENTS`, cloud-init network-config, decompressors) | Which agent a guest gets and how its network is configured is a guest profile. The hypervisor driver is handed the packages and commands. |
+| `TYPE_LABEL`, `REBOOT_SETTLE_SECONDS` | A display name and a polling delay. |
+| `run_device_action` | Replaced by the templates above. |
+
+`OSDriver.ROLE` stops defaulting to `"linux"`. Each OS driver states its own role.
+
+## Rules, enforced by tests in this package
+
+1. Roles and capabilities own no runtime method; a template runs only declared steps.
+2. No module imports `paramiko`, `socket`, `subprocess` or `asyncio`, and none defines a
+   `*_COMMAND`.
+3. No module, string or default names an application.
+4. Role methods return TypedDicts, not `Any`.
+
+Each rule is added with the step that makes it pass.
+
+## Migration
+
+This package cannot keep re-exports pointing at napalm-unix: the contract package would
+then depend on its implementation. So a moved name is first deprecated here and then
+removed, and nothing breaks in between.
+
+| Step | Change |
+|---|---|
+| 1 | The application moves to 4.9 and the drivers that need it; napalm-bsd stops passing stdin to a privileged command while a sudo password is set. |
+| 2 | This document; the application's architecture docs and this README follow it. |
+| 3 | `SnmpHealthProfile`. The application polls on its own; the four switch and phone drivers declare their OIDs as data. `HealthMetricsMixin` is deprecated. |
+| 4 | The fix templates. napalm-linux and napalm-bsd implement the steps; the application calls the templates with its values. `run_device_action` is deprecated. The other drivers follow, one issue each. |
+| 5 | Ping sweep, scoring, provisioning, `TYPE_LABEL` and `REBOOT_SETTLE_SECONDS` move to the application and are deprecated here. Drivers report facts instead of warnings. |
+| 6 | napalm-unix, with the moved code and its tests, the unified hooks and the frame helper. napalm-linux and napalm-bsd import from it; the moved names are deprecated here. |
+| 7 | 5.0 removes everything deprecated. It also splits `models.py` by area, types the remaining `Dict[str, Any]` role returns, declares what drivers already offer undeclared (`get_networks`, `search_packages`, `first_boot_pending`, the base-system update entry), and makes `install_package` declare the result drivers return. |
+| 8 | The README shrinks to the concepts; the reader details move with the readers. |
+
+## Proposals this replaces
+
+Several open issues proposed adding shared driver code here. Under this structure:
+
+| Issue | Goes to |
+|---|---|
+| #8 netmiko session | not here: a CLI-driver helper package, or left in the drivers |
+| #9 config mode for CLI switches | not here: a CLI-driver helper package, or left in the drivers |
+| #10 LLDP rows | the projection of rows into NAPALM's two shapes stays here; the `lldpctl` parser goes to napalm-unix |
+| #11 CLI text helpers | not here; the natural port sort, which `lag` already has, may become public |
+| #12 ping | the reply builder stays here; the BusyBox and iputils parser goes to napalm-unix |
+| #13 Linux userland | napalm-unix; the snmpd part becomes the SNMP template's steps |
+| #14 `run_device_action` | replaced by the fix templates |
+| #22 hooks onto `run_command` | step 6 |