apply_dhcp_reservationset wrote all adds first, then the updates in the order of desired. Kea keeps addresses unique at each write, not only at the commit. With host A on 192.0.2.10 and B on 192.0.2.11, and desired B = 192.0.2.10, A = 192.0.2.20, B came first and Kea refused it: {'result': 'failed', 'validations': {'reservation.ip_address': 'Duplicate entry exists.'}}. In the other order both go through. An add for an address an update frees was refused the same way, because adds went first.
Fix
A new reconciler module, _dhcp_order.py, with no I/O, orders the changes so that an address is released before it is taken:
No deletes to order. The diff never deletes (by design, see diff_dhcp_reservations), so there is no "deletes first" step.
Updates that free an address go first. An update that moves a reservation off its address goes before the change that takes that address. Along a chain (A takes B's, B takes C's, C takes a free one), it is written from the free end: C, B, A. Otherwise the desired order is kept.
Adds come after every update. Adds free nothing, and an update may free their address.
An update that keeps its address frees nothing. A hostname change does not release the address.
Addresses are compared by ipaddress, so 2001:DB8::10 and 2001:db8::10 count as the same address.
Swaps: reported, not worked around with a temporary step. In a swap (A and B trade addresses) or a longer rotation, no order frees an address first. The changes are still tried, in desired order with the cycle broken at its first member. A server that checks only at the commit gets them, so a backend that already handled swaps keeps working. A server that refuses them refuses each with its own message. The [failed] line then adds -- an address swap with <the other MACs>: no order frees an address first; move one of them to a free address in a run of its own.
I chose not to use a temporary step because the contract has no way to pick a safe temporary address. Pools, leases, and reservations nobody manages all hold addresses the base class cannot see, and there is no delete in the contract. If the second write then failed, the reservation would be left on the temporary address, which is a broken state the caller never asked for.
Kept from 4.35.1. Each refused change gets a [failed] line, every other change is still written and committed, and DhcpApplyError (refused of attempted) is raised after the commit.
One read. The diff is computed by a module function shared with diff_dhcp_reservations, so the apply reads the reservations once and orders against that same reading. diff_dhcp_reservations behaves as before. The quality baseline entry for it (54 lines) is gone because the method is now short. apply_dhcp_reservationset stays within 50 lines.
apply_dhcp_subnetset does not have this problem. Subnets match on their CIDR, which is their identity: an update never changes it, and an add is a CIDR not served yet. No change releases something another change takes. It does have a different gap, which this PR does not touch: it has no 4.35.1-style tolerance, so one refused subnet aborts the run before the commit (reported separately).
Docs: README (DHCP paragraph), docs/STRUCTURE.md (the reconciler row names _dhcp_order), CHANGELOG [Unreleased] → Fixed. The version is not bumped: that is left to the release (a PATCH, 4.37.1).
Tests
Fixtures only. tests/test_dhcp_diff_apply.py adds a Kea-like fake that refuses a write taking an address another reservation still holds, with Kea's message.
TestApplyOrderReleasesBeforeTaking:
the issue's case (exact progress lines, both moved);
a chain of three written from its free end;
an add after the update that frees its address;
an update that keeps its address frees nothing;
addresses matched however they are spelled.
TestApplyOrderWithASwap:
a refused swap: both [failed] lines name the swap partner, the unrelated add is written, one commit, DhcpApplyError "2 of 3", and nothing moved;
a server that accepts the swap gets both;
a rotation of three is reported as a swap with the two others.
Red before the fix: the issue case, the chain, the add-after-update, the refused swap and the rotation failed (the spelling test was added with the fix). All existing tests pass unchanged.
Full suite: 1632 passed, 3 skipped, coverage 96.9 %. ruff format and check are clean, mypy is clean, and napalm-standard check and ratchet are green.
## What was wrong
`apply_dhcp_reservationset` wrote all adds first, then the updates in the order of `desired`. Kea keeps addresses unique at each write, not only at the commit. With host A on 192.0.2.10 and B on 192.0.2.11, and desired B = 192.0.2.10, A = 192.0.2.20, B came first and Kea refused it: `{'result': 'failed', 'validations': {'reservation.ip_address': 'Duplicate entry exists.'}}`. In the other order both go through. An add for an address an update frees was refused the same way, because adds went first.
## Fix
A new reconciler module, `_dhcp_order.py`, with no I/O, orders the changes so that an address is released before it is taken:
- **No deletes to order.** The diff never deletes (by design, see `diff_dhcp_reservations`), so there is no "deletes first" step.
- **Updates that free an address go first.** An update that moves a reservation off its address goes before the change that takes that address. Along a chain (A takes B's, B takes C's, C takes a free one), it is written from the free end: C, B, A. Otherwise the desired order is kept.
- **Adds come after every update.** Adds free nothing, and an update may free their address.
- **An update that keeps its address frees nothing.** A hostname change does not release the address.
- **Addresses are compared by `ipaddress`,** so `2001:DB8::10` and `2001:db8::10` count as the same address.
**Swaps: reported, not worked around with a temporary step.** In a swap (A and B trade addresses) or a longer rotation, no order frees an address first. The changes are still tried, in desired order with the cycle broken at its first member. A server that checks only at the commit gets them, so a backend that already handled swaps keeps working. A server that refuses them refuses each with its own message. The `[failed]` line then adds `-- an address swap with <the other MACs>: no order frees an address first; move one of them to a free address in a run of its own`.
I chose not to use a temporary step because the contract has no way to pick a safe temporary address. Pools, leases, and reservations nobody manages all hold addresses the base class cannot see, and there is no delete in the contract. If the second write then failed, the reservation would be left on the temporary address, which is a broken state the caller never asked for.
**Kept from 4.35.1.** Each refused change gets a `[failed]` line, every other change is still written and committed, and `DhcpApplyError` (refused of attempted) is raised after the commit.
**One read.** The diff is computed by a module function shared with `diff_dhcp_reservations`, so the apply reads the reservations once and orders against that same reading. `diff_dhcp_reservations` behaves as before. The quality baseline entry for it (54 lines) is gone because the method is now short. `apply_dhcp_reservationset` stays within 50 lines.
**`apply_dhcp_subnetset` does not have this problem.** Subnets match on their CIDR, which is their identity: an update never changes it, and an add is a CIDR not served yet. No change releases something another change takes. It does have a different gap, which this PR does not touch: it has no 4.35.1-style tolerance, so one refused subnet aborts the run before the commit (reported separately).
Docs: README (DHCP paragraph), `docs/STRUCTURE.md` (the reconciler row names `_dhcp_order`), CHANGELOG `[Unreleased]` → Fixed. The version is not bumped: that is left to the release (a PATCH, 4.37.1).
## Tests
Fixtures only. `tests/test_dhcp_diff_apply.py` adds a Kea-like fake that refuses a write taking an address another reservation still holds, with Kea's message.
- `TestApplyOrderReleasesBeforeTaking`:
- the issue's case (exact progress lines, both moved);
- a chain of three written from its free end;
- an add after the update that frees its address;
- an update that keeps its address frees nothing;
- addresses matched however they are spelled.
- `TestApplyOrderWithASwap`:
- a refused swap: both `[failed]` lines name the swap partner, the unrelated add is written, one commit, `DhcpApplyError` "2 of 3", and nothing moved;
- a server that accepts the swap gets both;
- a rotation of three is reported as a swap with the two others.
Red before the fix: the issue case, the chain, the add-after-update, the refused swap and the rotation failed (the spelling test was added with the fix). All existing tests pass unchanged.
Full suite: 1632 passed, 3 skipped, coverage 96.9 %. ruff format and check are clean, mypy is clean, and `napalm-standard check` and `ratchet` are green.
Closes #106
apply_dhcp_reservationset wrote adds first, then updates in the order of
the desired set. Kea keeps addresses unique at each write, so when an
address moved from one reservation to another, the reservation taking it
was refused as a duplicate whenever it came first.
The changes are now ordered (_dhcp_order): an update that moves a
reservation off its address goes before the change that takes that
address, along a chain from its free end, and adds come after every
update. A swap or rotation of addresses has no such order; its changes
are still tried, so a server that checks only at the commit takes them,
and a refused one's [failed] line says it was a swap and with which
reservations. Nothing is moved to a temporary address: the contract
knows no free one, and a failed second write would leave it there.
The diff is computed against one read of the reservations, shared with
diff_dhcp_reservations. Refused rows still do not stop the others, the
rest is committed and DhcpApplyError follows, as since 4.35.1.
Closes#106
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
What was wrong
apply_dhcp_reservationsetwrote all adds first, then the updates in the order ofdesired. Kea keeps addresses unique at each write, not only at the commit. With host A on 192.0.2.10 and B on 192.0.2.11, and desired B = 192.0.2.10, A = 192.0.2.20, B came first and Kea refused it:{'result': 'failed', 'validations': {'reservation.ip_address': 'Duplicate entry exists.'}}. In the other order both go through. An add for an address an update frees was refused the same way, because adds went first.Fix
A new reconciler module,
_dhcp_order.py, with no I/O, orders the changes so that an address is released before it is taken:diff_dhcp_reservations), so there is no "deletes first" step.ipaddress, so2001:DB8::10and2001:db8::10count as the same address.Swaps: reported, not worked around with a temporary step. In a swap (A and B trade addresses) or a longer rotation, no order frees an address first. The changes are still tried, in desired order with the cycle broken at its first member. A server that checks only at the commit gets them, so a backend that already handled swaps keeps working. A server that refuses them refuses each with its own message. The
[failed]line then adds-- an address swap with <the other MACs>: no order frees an address first; move one of them to a free address in a run of its own.I chose not to use a temporary step because the contract has no way to pick a safe temporary address. Pools, leases, and reservations nobody manages all hold addresses the base class cannot see, and there is no delete in the contract. If the second write then failed, the reservation would be left on the temporary address, which is a broken state the caller never asked for.
Kept from 4.35.1. Each refused change gets a
[failed]line, every other change is still written and committed, andDhcpApplyError(refused of attempted) is raised after the commit.One read. The diff is computed by a module function shared with
diff_dhcp_reservations, so the apply reads the reservations once and orders against that same reading.diff_dhcp_reservationsbehaves as before. The quality baseline entry for it (54 lines) is gone because the method is now short.apply_dhcp_reservationsetstays within 50 lines.apply_dhcp_subnetsetdoes not have this problem. Subnets match on their CIDR, which is their identity: an update never changes it, and an add is a CIDR not served yet. No change releases something another change takes. It does have a different gap, which this PR does not touch: it has no 4.35.1-style tolerance, so one refused subnet aborts the run before the commit (reported separately).Docs: README (DHCP paragraph),
docs/STRUCTURE.md(the reconciler row names_dhcp_order), CHANGELOG[Unreleased]→ Fixed. The version is not bumped: that is left to the release (a PATCH, 4.37.1).Tests
Fixtures only.
tests/test_dhcp_diff_apply.pyadds a Kea-like fake that refuses a write taking an address another reservation still holds, with Kea's message.TestApplyOrderReleasesBeforeTaking:TestApplyOrderWithASwap:[failed]lines name the swap partner, the unrelated add is written, one commit,DhcpApplyError"2 of 3", and nothing moved;Red before the fix: the issue case, the chain, the add-after-update, the refused swap and the rotation failed (the spelling test was added with the fix). All existing tests pass unchanged.
Full suite: 1632 passed, 3 skipped, coverage 96.9 %. ruff format and check are clean, mypy is clean, and
napalm-standard checkandratchetare green.Closes #106