napalm-device-types/docs/STRUCTURE.md

Structure

Status: agreed plan. Nothing here is implemented yet; the migration at the end says in which order it will be.

Principle

napalm-device-types is vendor-, OS-, protocol- and product-agnostic. It holds the contracts every driver implements and the generic logic that works unchanged for every driver of a role. It knows no shell, no paramiko, no package manager, no container CLI, no SNMP OID, no cloud-init and no application that uses the drivers.

That rules out three kinds of code that are here today:

• How a kind of host is read. Shell commands and their parsers for POSIX hosts move to a new package, napalm-unix, which napalm-linux and napalm-bsd depend on.
• What an application does with a device. Polling a device over SNMP, sweeping a range with pings, scoring discovery evidence, choosing what a provisioned guest installs, and deciding what counts as a problem belong to the application, not to the drivers.
• Product naming. Files, markers and messages a driver leaves on a device carry the driver's name or none, never the name of the application that called it.

napalm-device-types        roles, capabilities, templates, models, reconcilers  <- every driver
        ^
napalm-unix                POSIX host readers, SSH exec transport, privilege    <- napalm-linux, napalm-bsd
        ^
napalm-linux, napalm-bsd   connection, platform getters, platform steps

application                SNMP polling, ping sweeps, discovery scoring, guest
                           profiles, warnings, and the values every fix is given

What stays in napalm-device-types

Kind	Rule	Modules
Role	Declares only, under TYPE_CHECKING. No runtime method.	access_point, switch, firewall, hypervisor, os, storage, phone, media, residential_gateway, management_controller
Capability	Declares only.	packages, services, updates, nat_vpn, mac_acl, config_lifecycle, interface_filter, host_reboot
Template	A concrete method that runs a fixed sequence of abstract steps. A driver implements the steps, or overrides the whole method when its device does it in one go. All values come from the caller.	the fix routines below
Contract types	Interfaces and data shapes other packages implement or fill.	CommandChannelMixin, CommandResult, ByteStream; the declaration of ContainerEngineMixin; the reader mixins' public method declarations; the SNMP profile type; models
Reconciler	Concrete and vendor-neutral, without I/O of its own.	firewall_rules and dhcp diff/apply, _matching, lag
Introspection	Concrete, without I/O.	roles; base (FingerprintRule, PortSpec, the fingerprint attributes)

Fix routines are templates

A change the application wants made on a device, for example "run an SNMP agent that answers us", is one template per purpose. The template fixes the order of the steps; the driver knows how each step is done on its device; the caller supplies every value.

class SnmpAgentMixin:
    def configure_snmp_agent(self, settings: SnmpAgentSettingsDict) -> FixResultDict:
        steps = [self._install_snmp_agent, self._write_snmp_agent_config,
                 self._enable_snmp_agent, self._verify_snmp_agent]
        ...                                   # runs them in order, stops at the first failure

    # declared only; napalm-unix implements them for net-snmp
    def _install_snmp_agent(self) -> FixStepDict: ...
    def _write_snmp_agent_config(self, settings: SnmpAgentSettingsDict) -> FixStepDict: ...
    def _enable_snmp_agent(self) -> FixStepDict: ...
    def _verify_snmp_agent(self, settings: SnmpAgentSettingsDict) -> FixStepDict: ...

napalm-unix implements the four steps for net-snmp, for Linux and the BSDs alike. A REST driver such as OPNsense overrides configure_snmp_agent as a whole, because its device takes the whole configuration in one call. The community, the allowed sources, contact and location are the caller's.

The same shape replaces every action in today's run_device_action:

Today	Template
fix_snmp	configure_snmp_agent(settings)
fix_apt_proxy	configure_package_proxy(url)
fix_docker_permissions	grant_container_engine_access(user)
install_lldpd, install_auc	ensure_package(name)

hasattr(driver, "configure_snmp_agent") stays a truthful answer: the mixin is mixed in by the driver, or by the platform package, not by a role base.

SNMP: drivers declare data, the application polls

The drivers do not collect over SNMP. A driver whose device answers the standard UCD-MIB and IF-MIB declares nothing. A driver whose device publishes the same values under proprietary OIDs declares them as data, in a class attribute of a type defined here:

class ProcurveDriver(SwitchDriver):
    SNMP_HEALTH_PROFILE: ClassVar[SnmpHealthProfile] = {
        "uptime": "1.3.6.1.4.1.11.2.14.11.1.2.6.1.0", ...
    }

Polling, the standard OIDs, the counter arithmetic and the interfaces worth skipping are the application's.

Warnings are facts

A driver reports facts: the proxy a host is configured with, a DSL line's SNR, the firmware version it runs and the one it is offered. It does not compare them with a setting of the application or with a threshold, and it does not name a warning.

What moves to napalm-unix

Module	Knows
host_status	apt, dnf, dpkg, systemd timers, df
kernel	/proc, /lib/modules, gzip, base64
listening	ss, netstat, /proc/<pid>/cgroup
systemd	systemctl
virtual_ips	vip-manager, keepalived, awk
system_identity	SMBIOS via sysfs and dmidecode
package_updates	apt and dnf output
ssh_server	OpenSSH (sshd -T, sshd -t, sshd_config); which algorithms to drop stays the caller's
terminal	ANSI escape codes in shell output
channel: run_on_transport, open_stream_on_transport, ParamikoExecStream	paramiko exec channels
container_engine: ContainerEngineConnection, engine probing	the Docker CLI, system dial-stdio

It also takes what napalm-linux and napalm-bsd each carry today:

• Privilege: one sudo -S / sudo -n / root layer, including the rule that a privileged command never gets stdin while a sudo password is set, and one way to write a file as root (stage with mktemp, then install). Two copies of this have already drifted apart.
• Parsers: /etc/passwd with /etc/group, crontab lines, ps with lstart.
• Fix steps: net-snmp, the package proxy, container engine group membership.

The FreeBSD part of host_status (freebsd-version -k against -r) moves to napalm-bsd.

In napalm-unix every reader follows one pattern:

• Shape: FOO_COMMAND, parse_foo(output), and FooMixin.get_foo().
• Reaching the host: the mixin uses run_command(command, privileged=...) from CommandChannelMixin. The six per-reader _run_* hooks, with four different signatures, go away.
• Framing: one frame helper writes and parses the BEGIN/END markers for every reader.
• Errors: one exception for "cannot read" and one for "not on this host". A read that fails or is cut short raises; it never reads as "nothing there".
• Names on the host: temporary files, markers and drop-ins carry napalm or the driver's name. A drop-in written under an older name is still recognised and replaced.

Layout: napalm_unix/{posix,linux,freebsd,openssh,transport}/.

What moves to the application

Today	Why it is not a driver's
health_metrics, _ucd_metrics	Polling over SNMP is something the application does to a device; the driver is not even connected while it runs.
ping_sweep	Looping over addresses, capping the targets and reporting progress is a scan. A device with a faster batch keeps it as a capability of its own.
rules_max_weight	Scoring is how one application ranks drivers. The rules themselves stay with the driver.
provisioning (QEMU_GUEST_AGENTS, cloud-init network-config, decompressors)	Which agent a guest gets and how its network is configured is a guest profile. The hypervisor driver is handed the packages and commands.
TYPE_LABEL, REBOOT_SETTLE_SECONDS	A display name and a polling delay.
run_device_action	Replaced by the templates above.

OSDriver.ROLE stops defaulting to "linux". Each OS driver states its own role.

Rules, enforced by tests in this package

1. Roles and capabilities own no runtime method; a template runs only declared steps.
2. No module imports paramiko, socket, subprocess or asyncio, and none defines a *_COMMAND.
3. No module, string or default names an application.
4. Role methods return TypedDicts, not Any.

Each rule is added with the step that makes it pass.

Migration

This package cannot keep re-exports pointing at napalm-unix: the contract package would then depend on its implementation. So a moved name is first deprecated here and then removed, and nothing breaks in between.

Step	Change
1	The application moves to 4.9 and the drivers that need it; napalm-bsd stops passing stdin to a privileged command while a sudo password is set.
2	This document; the application's architecture docs and this README follow it.
3	SnmpHealthProfile. The application polls on its own; the four switch and phone drivers declare their OIDs as data. HealthMetricsMixin is deprecated.
4	The fix templates. napalm-linux and napalm-bsd implement the steps; the application calls the templates with its values. run_device_action is deprecated. The other drivers follow, one issue each.
5	Ping sweep, scoring, provisioning, TYPE_LABEL and REBOOT_SETTLE_SECONDS move to the application and are deprecated here. Drivers report facts instead of warnings.
6	napalm-unix, with the moved code and its tests, the unified hooks and the frame helper. napalm-linux and napalm-bsd import from it; the moved names are deprecated here.
7	5.0 removes everything deprecated. It also splits models.py by area, types the remaining Dict[str, Any] role returns, declares what drivers already offer undeclared (get_networks, search_packages, first_boot_pending, the base-system update entry), and makes install_package declare the result drivers return.
8	The README shrinks to the concepts; the reader details move with the readers.

Proposals this replaces

Several open issues proposed adding shared driver code here. Under this structure:

Issue	Goes to
#8 netmiko session	not here: a CLI-driver helper package, or left in the drivers
#9 config mode for CLI switches	not here: a CLI-driver helper package, or left in the drivers
#10 LLDP rows	the projection of rows into NAPALM's two shapes stays here; the lldpctl parser goes to napalm-unix
#11 CLI text helpers	not here; the natural port sort, which lag already has, may become public
#12 ping	the reply builder stays here; the BusyBox and iputils parser goes to napalm-unix
#13 Linux userland	napalm-unix; the snmpd part becomes the SNMP template's steps
#14 run_device_action	replaced by the fix templates
#22 hooks onto run_command	step 6