Compare commits
2
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 christianmanivong
|
c7ea15dbb4 | ||
|
christianmanivong
|
01fe680456 |
@@ -0,0 +1,241 @@
|
||||
# Structure
|
||||
|
||||
Status: agreed plan. Nothing here is implemented yet; the migration at the end says in
|
||||
which order it will be.
|
||||
|
||||
## Principle
|
||||
|
||||
**napalm-device-types is vendor-, OS-, protocol- and product-agnostic.** It holds the
|
||||
contracts every driver implements and the generic logic that works unchanged for every
|
||||
driver of a role. It knows no shell, no paramiko, no package manager, no container CLI,
|
||||
no SNMP OID, no cloud-init and no application that uses the drivers.
|
||||
|
||||
That rules out three kinds of code that are here today:
|
||||
|
||||
- **How a kind of host is read.** Shell commands and their parsers for POSIX hosts move to
|
||||
a new package, **napalm-unix**, which napalm-linux and napalm-bsd depend on.
|
||||
- **What an application does with a device.** Polling a device over SNMP, sweeping a range
|
||||
with pings, scoring discovery evidence, choosing what a provisioned guest installs, and
|
||||
deciding what counts as a problem belong to the application, not to the drivers.
|
||||
- **Product naming.** Files, markers and messages a driver leaves on a device carry the
|
||||
driver's name or none, never the name of the application that called it.
|
||||
|
||||
```
|
||||
napalm-device-types roles, capabilities, templates, models, reconcilers <- every driver
|
||||
^
|
||||
napalm-unix POSIX host readers, SSH exec transport, privilege <- napalm-linux, napalm-bsd
|
||||
^
|
||||
napalm-linux, napalm-bsd connection, platform getters, platform steps
|
||||
|
||||
application SNMP polling, ping sweeps, discovery scoring, guest
|
||||
profiles, warnings, and the values every fix is given
|
||||
```
|
||||
|
||||
## What stays in napalm-device-types
|
||||
|
||||
| Kind | Rule | Modules |
|
||||
|---|---|---|
|
||||
| **Role** | Declares only, under `TYPE_CHECKING`. No runtime method. | `access_point`, `switch`, `firewall`, `hypervisor`, `os`, `storage`, `phone`, `media`, `residential_gateway`, `management_controller` |
|
||||
| **Capability** | Declares only. | `packages`, `services`, `updates`, `nat_vpn`, `mac_acl`, `config_lifecycle`, `interface_filter`, `host_reboot` |
|
||||
| **Template** | A concrete method that runs a fixed sequence of abstract steps. A driver implements the steps, or overrides the whole method when its device does it in one go. All values come from the caller. | the fix routines below |
|
||||
| **Contract types** | Interfaces and data shapes other packages implement or fill. | `CommandChannelMixin`, `CommandResult`, `ByteStream`; the declaration of `ContainerEngineMixin`; the reader mixins' public method declarations; the SNMP profile type; `models` |
|
||||
| **Reconciler** | Concrete and vendor-neutral, without I/O of its own. | `firewall_rules` and `dhcp` diff/apply, `_matching`, `lag` |
|
||||
| **Introspection** | Concrete, without I/O. | `roles`; `base` (`FingerprintRule`, `PortSpec`, the fingerprint attributes) |
|
||||
| **Driver facts** | Class attributes that describe the device or how it is reached, not what an application does with it. | `USES_SSH`, `FULL_UPGRADE`, `platform`, `SNMP_HEALTH_PROFILE` |
|
||||
|
||||
### Fix routines are templates
|
||||
|
||||
A change the application wants made on a device, for example "run an SNMP agent that
|
||||
answers us", is one template per purpose. The template fixes the order of the steps; the
|
||||
driver knows how each step is done on its device; the caller supplies every value.
|
||||
|
||||
```python
|
||||
class SnmpAgentMixin:
|
||||
def configure_snmp_agent(self, settings: SnmpAgentSettingsDict) -> FixResultDict:
|
||||
steps = [self._install_snmp_agent, self._write_snmp_agent_config,
|
||||
self._enable_snmp_agent, self._verify_snmp_agent]
|
||||
... # runs them in order, stops at the first failure
|
||||
|
||||
# declared only; napalm-unix implements them for net-snmp
|
||||
def _install_snmp_agent(self) -> FixStepDict: ...
|
||||
def _write_snmp_agent_config(self, settings: SnmpAgentSettingsDict) -> FixStepDict: ...
|
||||
def _enable_snmp_agent(self) -> FixStepDict: ...
|
||||
def _verify_snmp_agent(self, settings: SnmpAgentSettingsDict) -> FixStepDict: ...
|
||||
```
|
||||
|
||||
napalm-unix implements the four steps for net-snmp, for Linux and the BSDs alike. A REST
|
||||
driver such as OPNsense overrides `configure_snmp_agent` as a whole, because its device
|
||||
takes the whole configuration in one call. The community, the allowed sources, contact
|
||||
and location are the caller's.
|
||||
|
||||
The same shape replaces every action in today's `run_device_action`:
|
||||
|
||||
| Today | Template |
|
||||
|---|---|
|
||||
| `fix_snmp` | `configure_snmp_agent(settings)` |
|
||||
| `fix_apt_proxy` | `configure_package_proxy(url)` |
|
||||
| `fix_docker_permissions` | `grant_container_engine_access(user)` |
|
||||
| `install_lldpd`, `install_auc` | `ensure_package(name)` |
|
||||
|
||||
The mixin is mixed in by the driver, or by the platform package, not by a role base. A
|
||||
driver that inherits a template it cannot run on its device, as a NAS driver built on the
|
||||
Linux driver inherits the package proxy, opts out by setting the method to `None`. So a
|
||||
caller asks `callable(getattr(driver, "configure_snmp_agent", None))`, never `hasattr`.
|
||||
|
||||
The caller verifies the result from its own side where it can; the SNMP agent is asked
|
||||
by the application's SNMP client, not by a probe the driver hand-builds.
|
||||
|
||||
### SNMP: drivers declare data, the application polls
|
||||
|
||||
The drivers do not collect over SNMP. A driver whose device answers the standard UCD-MIB
|
||||
and IF-MIB declares nothing. A driver whose device publishes the same values under
|
||||
proprietary OIDs declares them as data, in a class attribute of a type defined here:
|
||||
|
||||
```python
|
||||
class ProcurveDriver(SwitchDriver):
|
||||
SNMP_HEALTH_PROFILE: ClassVar[SnmpHealthProfile] = {
|
||||
"uptime": {"oids": ["1.3.6.1.4.1.11.2.14.11.1.2.4.1.5.1"], "format": "ticks"},
|
||||
"cpu": {"oids": ["1.3.6.1.4.1.11.2.14.11.5.1.9.6.1.0",
|
||||
"1.3.6.1.4.1.11.2.3.7.8.6.1.0"]},
|
||||
"memory": {"total": "...", "free": "...", "unit": "bytes"},
|
||||
}
|
||||
```
|
||||
|
||||
One OID per value is not enough for the drivers there are. The profile type has to say:
|
||||
|
||||
- **Fallbacks:** an ordered list of OIDs; the first that answers wins (ProCurve's CPU).
|
||||
- **Tables:** an OID to walk and how to fold its rows: first, average or maximum (TP-Link
|
||||
reports CPU and memory per stack unit).
|
||||
- **Text values:** a pattern that takes the number out of a string (Netgear reports CPU
|
||||
as `"5 Secs ( 3.2%)"`).
|
||||
- **Memory:** total with used or free, or a percentage only, and the unit.
|
||||
- **Uptime:** which OID, standard ones included (`sysUpTime` where a device has no
|
||||
`hrSystemUptime`), and its format.
|
||||
- **Interfaces:** a pattern of names to skip, and whether the TX-error counter counts
|
||||
drops. Today the second is `True` on every firewall, because OPNsense counts drops
|
||||
there; it becomes OPNsense's profile entry.
|
||||
|
||||
Polling, the standard OIDs, the counter arithmetic (64-bit counters first, 32-bit as the
|
||||
fallback) and the defaults are the application's.
|
||||
|
||||
### Warnings are facts
|
||||
|
||||
A driver reports facts: the proxy a host is configured with, a DSL line's SNR, the
|
||||
firmware version it runs and the one it is offered. It does not compare them with a
|
||||
setting of the application or with a threshold, and it does not name a warning.
|
||||
|
||||
The same goes for lookups a device does not make itself: what firmware a vendor
|
||||
currently publishes is the application's catalogue, not a getter.
|
||||
|
||||
### Getters read, and say when they cannot
|
||||
|
||||
- A getter changes nothing on the device. It does not start a daemon, rewrite its
|
||||
configuration or install a package on the way to an answer.
|
||||
- A read that fails raises. An empty list means "there is none", never "could not ask";
|
||||
a sync that deletes and re-adds on top of an empty answer would otherwise duplicate
|
||||
everything.
|
||||
- A write reports failure from the device's answer or exit status, never `success: True`
|
||||
by default.
|
||||
|
||||
### The application uses public methods only
|
||||
|
||||
The application never calls a driver's private methods (`_get`, `_post`,
|
||||
`_send_command`, `_action_*`). What it needs from a device becomes a declared method of
|
||||
the driver.
|
||||
|
||||
## What moves to napalm-unix
|
||||
|
||||
| Module | Knows |
|
||||
|---|---|
|
||||
| `host_status` | apt, dnf, dpkg, systemd timers, `df` |
|
||||
| `kernel` | `/proc`, `/lib/modules`, gzip, base64 |
|
||||
| `listening` | `ss`, `netstat`, `/proc/<pid>/cgroup` |
|
||||
| `systemd` | `systemctl` |
|
||||
| `virtual_ips` | vip-manager, keepalived, awk |
|
||||
| `system_identity` | SMBIOS via sysfs and `dmidecode` |
|
||||
| `package_updates` | apt and dnf output |
|
||||
| `ssh_server` | OpenSSH (`sshd -T`, `sshd -t`, `sshd_config`); which algorithms to drop stays the caller's |
|
||||
| `terminal` | ANSI escape codes in shell output |
|
||||
| `channel`: `run_on_transport`, `open_stream_on_transport`, `ParamikoExecStream` | paramiko exec channels |
|
||||
| `container_engine`: `ContainerEngineConnection`, engine probing | the Docker CLI, `system dial-stdio` |
|
||||
|
||||
It also takes what napalm-linux and napalm-bsd each carry today:
|
||||
|
||||
- **Privilege:** one `sudo -S` / `sudo -n` / root layer, including the rule that a
|
||||
privileged command never gets stdin while a sudo password is set, and one way to write a
|
||||
file as root (stage with `mktemp`, then `install`). Two copies of this have already
|
||||
drifted apart.
|
||||
- **Parsers:** `/etc/passwd` with `/etc/group`, crontab lines, `ps` with `lstart`.
|
||||
- **Fix steps:** net-snmp, the package proxy, container engine group membership.
|
||||
|
||||
The FreeBSD part of `host_status` (`freebsd-version -k` against `-r`) moves to napalm-bsd.
|
||||
|
||||
In napalm-unix every reader follows one pattern:
|
||||
|
||||
- **Shape:** `FOO_COMMAND`, `parse_foo(output)`, and `FooMixin.get_foo()`.
|
||||
- **Reaching the host:** the mixin uses `run_command(command, privileged=...)` from
|
||||
`CommandChannelMixin`. The six per-reader `_run_*` hooks, with four different signatures,
|
||||
go away.
|
||||
- **Framing:** one frame helper writes and parses the `BEGIN`/`END` markers for every reader.
|
||||
- **Errors:** one exception for "cannot read" and one for "not on this host". A read that
|
||||
fails or is cut short raises; it never reads as "nothing there".
|
||||
- **Names on the host:** temporary files, markers and drop-ins carry `napalm` or the
|
||||
driver's name. A drop-in written under an older name is still recognised and replaced.
|
||||
|
||||
Layout: `napalm_unix/{posix,linux,freebsd,openssh,transport}/`.
|
||||
|
||||
## What moves to the application
|
||||
|
||||
| Today | Why it is not a driver's |
|
||||
|---|---|
|
||||
| `health_metrics`, `_ucd_metrics` | Polling over SNMP is something the application does to a device; the driver is not even connected while it runs. |
|
||||
| `ping_sweep` | Looping over addresses, capping the targets and reporting progress is a scan. A device with a faster batch keeps it as a capability of its own. |
|
||||
| `rules_max_weight` | Scoring is how one application ranks drivers. The rules themselves stay with the driver. |
|
||||
| `provisioning` (`QEMU_GUEST_AGENTS`, cloud-init network-config, decompressors) | Which agent a guest gets and how its network is configured is a guest profile. The hypervisor driver is handed the packages and commands. |
|
||||
| `TYPE_LABEL`, `REBOOT_SETTLE_SECONDS` | A display name and a polling delay. |
|
||||
| firmware catalogues (napalm-netgear's `netgear_plus`) | What a vendor publishes is not something the device reports. |
|
||||
| `run_device_action` | Replaced by the templates above. |
|
||||
|
||||
`OSDriver.ROLE` stops defaulting to `"linux"`. Each OS driver states its own role.
|
||||
|
||||
## Rules, enforced by tests in this package
|
||||
|
||||
1. Roles and capabilities own no runtime method; a template runs only declared steps.
|
||||
2. No module imports `paramiko`, `socket`, `subprocess` or `asyncio`, and none defines a
|
||||
`*_COMMAND`.
|
||||
3. No module, string or default names an application.
|
||||
4. Role methods return TypedDicts, not `Any`.
|
||||
|
||||
Each rule is added with the step that makes it pass.
|
||||
|
||||
## Migration
|
||||
|
||||
This package cannot keep re-exports pointing at napalm-unix: the contract package would
|
||||
then depend on its implementation. So a moved name is first deprecated here and then
|
||||
removed, and nothing breaks in between.
|
||||
|
||||
| Step | Change |
|
||||
|---|---|
|
||||
| 1 | The application moves to 4.9 and the drivers that need it; napalm-bsd stops passing stdin to a privileged command while a sudo password is set. |
|
||||
| 2 | This document; the application's architecture docs and this README follow it. |
|
||||
| 3 | `SnmpHealthProfile`. The application polls on its own; the four switch and phone drivers declare their OIDs as data. `HealthMetricsMixin` is deprecated. |
|
||||
| 4 | The fix templates. napalm-linux and napalm-bsd implement the steps; the application calls the templates with its values. `run_device_action` is deprecated. The other drivers follow, one issue each. |
|
||||
| 5 | Ping sweep, scoring, provisioning, `TYPE_LABEL` and `REBOOT_SETTLE_SECONDS` move to the application and are deprecated here. Drivers report facts instead of warnings. |
|
||||
| 6 | napalm-unix, with the moved code and its tests, the unified hooks and the frame helper. napalm-linux and napalm-bsd import from it; the moved names are deprecated here. |
|
||||
| 7 | 5.0 removes everything deprecated. It also splits `models.py` by area, types the remaining `Dict[str, Any]` role returns, declares what drivers already offer undeclared (`get_networks`, `search_packages`, `first_boot_pending`, the base-system update entry), and makes `install_package` declare the result drivers return. |
|
||||
| 8 | The README shrinks to the concepts; the reader details move with the readers. |
|
||||
|
||||
## Proposals this replaces
|
||||
|
||||
Several open issues proposed adding shared driver code here. Under this structure:
|
||||
|
||||
| Issue | Goes to |
|
||||
|---|---|
|
||||
| #8 netmiko session | not here: a CLI-driver helper package, or left in the drivers |
|
||||
| #9 config mode for CLI switches | not here: a CLI-driver helper package, or left in the drivers |
|
||||
| #10 LLDP rows | the projection of rows into NAPALM's two shapes stays here; the `lldpctl` parser goes to napalm-unix |
|
||||
| #11 CLI text helpers | not here; the natural port sort, which `lag` already has, may become public |
|
||||
| #12 ping | the reply builder stays here; the BusyBox and iputils parser goes to napalm-unix |
|
||||
| #13 Linux userland | napalm-unix; the snmpd part becomes the SNMP template's steps |
|
||||
| #14 `run_device_action` | replaced by the fix templates |
|
||||
| #22 hooks onto `run_command` | step 6 |
|
||||
Reference in new issue
Block a user