feat(apt): send https sources through the cache in the form it stores (netork#2555) #122

Merged
christianmanivong merged 1 commits from fix/apt-https-cache into main 2026-11-15 19:23:47 +00:00
Owner

For NetOrk/netork#2555.

apt-cacher-ng tunnels a source fetched over https and stores nothing of it. It fetches http://HTTPS///<host>/<path> over https itself, with the certificate verified, and stores it, where port 443 is allowed (AllowUserPorts=80 443, which netOrk's compose sets in the same change).

Added: apt_sources

  • cacheable_https_hosts(report): the hosts of the https sources the rewrite would send through the cache. Left out are:
    • hosts apt fetches DIRECT,
    • hosts on another port,
    • hosts with a login in the URL,
    • hosts left on https for their credentials (not_cached).
  • rewrite_sources_command(hosts, proxy_file) and revert_sources_command(): the root commands, for .list (deb/deb-src) and deb822 .sources (URIs:). A file without the host is not touched.
  • A host apt keeps credentials for in auth.conf{,.d} stays on https. apt finds credentials by the host's name, and the cache is shared. Only root reads those files, so the rewrite names such a host in the proxy file as Package-Proxy::Not-Cached. apt ignores the key; apt-config dump shows it, and the report reads it as not_cached.

Changed: configure_package_proxy

  • url: a step cache before https. It rewrites, fetches one rewritten InRelease through url, and turns all sources back to https where url cannot (a cache without port 443 answers 403 ... prohibited port). The fetch step proves the proxy with a plain http source only. The https DIRECT fallback keeps the not-cached hosts in 00proxy.
  • "": step sources turns the sources back to https before the proxy file goes. Without the cache apt looks for a host called HTTPS.

Tested

  • tests/test_apt_sources.py runs the commands for real, against files that apt-config -o points apt at.
  • By hand in an Ubuntu 24.04 container, with real apt against the pinned acng image (3.7.5) with AllowUserPorts=80 443:
    • the rewrite of a Docker .list,
    • apt-get update through the cache,
    • the credentials host left on https and named,
    • the report reading the marker,
    • the revert, which gave the identical file back.
  • Full suite with coverage: 1711 passed, 97%. Ratchet: within the baseline. ruff and mypy clean.
For NetOrk/netork#2555. apt-cacher-ng tunnels a source fetched over https and stores nothing of it. It fetches `http://HTTPS///<host>/<path>` over https itself, with the certificate verified, and stores it, where port 443 is allowed (`AllowUserPorts=80 443`, which netOrk's compose sets in the same change). **Added: `apt_sources`** - `cacheable_https_hosts(report)`: the hosts of the https sources the rewrite would send through the cache. Left out are: - hosts apt fetches `DIRECT`, - hosts on another port, - hosts with a login in the URL, - hosts left on https for their credentials (`not_cached`). - `rewrite_sources_command(hosts, proxy_file)` and `revert_sources_command()`: the root commands, for `.list` (`deb`/`deb-src`) and deb822 `.sources` (`URIs:`). A file without the host is not touched. - A host apt keeps credentials for in `auth.conf{,.d}` stays on https. apt finds credentials by the host's name, and the cache is shared. Only root reads those files, so the rewrite names such a host in the proxy file as `Package-Proxy::Not-Cached`. apt ignores the key; `apt-config dump` shows it, and the report reads it as `not_cached`. **Changed: `configure_package_proxy`** - `url`: a step `cache` before `https`. It rewrites, fetches one rewritten `InRelease` through *url*, and turns all sources back to https where *url* cannot (a cache without port 443 answers `403 ... prohibited port`). The `fetch` step proves the proxy with a plain http source only. The `https` DIRECT fallback keeps the not-cached hosts in `00proxy`. - `""`: step `sources` turns the sources back to https before the proxy file goes. Without the cache apt looks for a host called `HTTPS`. **Tested** - `tests/test_apt_sources.py` runs the commands for real, against files that `apt-config -o` points apt at. - By hand in an Ubuntu 24.04 container, with real apt against the pinned acng image (3.7.5) with `AllowUserPorts=80 443`: - the rewrite of a Docker `.list`, - `apt-get update` through the cache, - the credentials host left on https and named, - the report reading the marker, - the revert, which gave the identical file back. - Full suite with coverage: 1711 passed, 97%. Ratchet: within the baseline. ruff and mypy clean.
christianmanivong added 1 commit 2026-11-15 19:07:41 +00:00
feat(apt): send https sources through the cache in the form it stores (NetOrk/netork#2555)
driver / lint (pull_request) Successful in 15s
driver / standard (pull_request) Successful in 19s
driver / types (pull_request) Successful in 58s
driver / test (3.13) (pull_request) Successful in 1m1s
driver / test (3.12) (pull_request) Successful in 1m8s
driver / lowest (pull_request) Successful in 35s
driver / build (pull_request) Successful in 29s
driver / security (pull_request) Successful in 1m9s
CI / driver (pull_request) Successful in 2m29s
driver / publish (pull_request) Skipped
cb0f1cc05c
apt-cacher-ng tunnels a source fetched over https and stores nothing of it. It
fetches http://HTTPS///<host>/<path> over https itself, with the certificate
verified, and stores it, where port 443 is allowed (AllowUserPorts=80 443).

- apt_sources: cacheable_https_hosts(report), and the root commands that rewrite
  the https sources of .list and deb822 .sources files to that form and back.
  A host apt has credentials for in auth.conf stays on https; the rewrite names
  it in the proxy file as Package-Proxy::Not-Cached, which the report reads.
- configure_package_proxy(url): a step "cache" before "https". It rewrites the
  sources, fetches one of them through url, and turns them all back to https
  where url cannot. The "fetch" step proves the proxy with a plain http source.
- configure_package_proxy(""): the sources go back to https before the proxy
  file goes; without the cache apt would look for a host called HTTPS.
christianmanivong merged commit 7c0e4fffc5 into main 2026-11-15 19:23:47 +00:00
christianmanivong deleted branch fix/apt-https-cache 2026-11-15 19:23:47 +00:00
Sign in to join this conversation.