Christian Manivong 4bf71b7e1c
CI / check (pull_request) Successful in 17s
feat: refuse a tag without an image for the host's platform
From 0.34.0 on, netOrk's releases (X.Y.Z, latest) are published for
linux/amd64 and linux/arm64, so a Raspberry Pi with a 64-bit OS can be a
target (NetOrk/netork#934). The deploy itself needed no change for that:
every step runs docker on the target host, which pulls its own platform,
and the images on the host (postgres, redis, apt-cacher-ng, signal-api)
exist for arm64 as well.

What did not work is a tag without arm64: latest-dev, main-<sha>,
feature-<branch> and every release before 0.34.0. On a Pi those failed at
the first pull with "no matching manifest for linux/arm64/v8", which names
neither the cause nor a tag that would work.

Now, right after the registry login, the host's architecture (docker
version) is compared with the platforms the registry lists for the
engine image, and on UI servers the ui image (docker manifest inspect).
A definite mismatch stops the deploy before anything on the host changes,
and says which tags carry arm64. When the registry's answer can't be read,
the pull decides, as before.
2026-10-11 08:38:39 +02:00

netOrk deploy

Deploys netOrk to one or more Docker hosts in parallel, from pre-built images in a container registry.

The machine running the deploy needs this repository, ssh, and a deploy.env. It does not need a netOrk checkout. Each server pulls the images itself. The compose files come out of the engine image for the tag being deployed, so they always match the images they start, and database migrations run from that same image.

Requirements

On the machine running the deploy:

  • bash
  • ssh with key-based access to every target

On every target server:

  • at least 4 GB of RAM. On a test server with 74 devices, netOrk's containers took about 2.4 GB, and at least 0.9 GB stayed free. A large task needs that room. The breakdown per container is in netOrk's docs/TECHNICAL_DESIGN.md, "Worker memory".
  • Docker with the compose plugin
  • a directory ~/netork/ with netOrk's .env, which holds the database and application settings the compose file reads (env_file: .env)

Images: netork/engine and netork/ui must be published in the registry under the tag you deploy. The engine image must carry its compose files under /app/deploy/. Tags built before that change are refused with a clear message.

Raspberry Pi and other arm64 hosts: a Raspberry Pi 3, 4, 5 or Zero 2 W with a 64-bit OS can be a target. Nothing changes in deploy.env: each host pulls the variant for its own platform. netOrk publishes arm64 images for releases only, from 0.34.0 on (latest, X.Y.Z). latest-dev, main-<sha>, feature-<branch> and older releases are amd64 only, and a deploy of such a tag to an arm64 host is refused before the host is touched. The images on the host (postgres, redis, apt-cacher-ng, signal-api) exist for arm64 as well.

Setup

cp deploy.env.example deploy.env
$EDITOR deploy.env

deploy.env is gitignored. Point DEPLOY_ENV_FILE at another file to keep it elsewhere.

Variable Meaning
DEPLOY_SERVERS Space-separated default targets
UI_SERVER Targets that also run netork-ui
REGISTRY_HOST Registry to pull from; required
NETORK_VERSION Tag to deploy (default latest)
REGISTRY_USER, REGISTRY_PASSWORD Registry login used on every server
REGISTRY_USER_<server>, REGISTRY_PASSWORD_<server>, NETORK_VERSION_<server> Per-server overrides. <server> has its dots replaced by underscores, e.g. _10_0_0_2
DEPLOY_LOCK_WAIT Seconds to wait for another deploy to the same host (default 900)

Usage

./deploy.sh                          # every server in DEPLOY_SERVERS
./deploy.sh 10.0.0.1                 # one server
./deploy.sh 10.0.0.1 10.0.0.2        # several, in parallel
./deploy.sh --version=main-1a2b3c4 10.0.0.1   # one-off tag, this run only

Only tags netOrk's CI publishes are accepted:

  • latest
  • latest-dev
  • main-<sha>
  • feature-<branch>
  • X.Y.Z

Anything else is refused before any server is touched.

If you deploy a branch build, wait until its CI run has published the images. A deploy started earlier pulls whatever image the registry held before, which is stale.

What a deploy does, per server

  1. Takes the host's deploy lock, flock on ~/netork/.deploy.lock, and holds it until the deploy ends. A second deploy to the same host waits for it and says who holds it, since when, and which tag they are deploying. It gives up after 15 minutes (DEPLOY_LOCK_WAIT, in seconds) without touching the host. The lock belongs to an ssh session, so a deploy that fails, is interrupted or loses its connection frees it on its own. A host without flock is deployed without the lock, with a warning.
  2. Logs in to the registry. The password travels over ssh's stdin, never on a command line. Then it checks that the tag has an image for the host's platform (docker version, docker manifest inspect): engine, and on UI servers ui. If not, for example latest-dev on a Raspberry Pi, it stops here and says which tags would work. If the registry's answer can't be read, the pull below decides.
  3. Pulls the engine image and copies docker-compose.yml and docker-compose.registry.yml out of it into ~/netork/.
  4. Pulls the netOrk images, then force-recreates the API, the workers, netork-beat, flower and, on UI servers, netork-ui. If the recreate fails, it is tried once more after 5 s (DEPLOY_RECREATE_RETRY_DELAY). Compose's parallel recreate can lose a container it just renamed and leave the rest stopped. If the second attempt fails too, the container states are printed and the deploy fails.
  5. Reconciles apt-cacher-ng and signal-api: each is recreated only if its definition changed. It never touches postgres or redis. It then removes services netOrk no longer ships, container and volume, where a host still has them: today the bundled registry (NetOrk/netork#763). It reports only what it actually removed; on a host that was cleaned earlier it says nothing.
  6. Verifies that the containers run exactly the image that was pulled. If they don't, the deploy fails.
  7. Records REGISTRY_HOST and NETORK_VERSION in ~/netork/.env, so that a hand-typed docker compose on the host uses the same images.
  8. Runs alembic upgrade head inside netork-api.
  9. Prunes unused images.

A failing step stops that server's deploy with a non-zero exit, and the other servers carry on. The script exits non-zero if any server failed and names those servers.

Operating notes

  • Never delete these Docker volumes:
    • celerybeat_schedule holds the scheduler state.
    • signal_cli_data holds netOrk's Signal device link. Losing it means pairing again by QR code.
  • netork-beat always rolls out together with the workers. The script does this for you; keep it that way if you deploy by hand.
  • Always deploy with this script. Do not point a local Docker client at a remote host (DOCKER_HOST=ssh://…): it resolves volume paths locally and breaks the remote containers.

Development

pip install pytest ruff shellcheck-py
shellcheck deploy.sh
ruff format --check . && ruff check .
pytest

The tests are static and behavioural checks of deploy.sh. None of them reach a real host.

License

MIT, see LICENSE.

S
Description
Deploys netOrk to Docker hosts from pre-built registry images
Readme MIT
207 KiB
0 Stars 1 Watchers 0 Forks
Languages
Python 62.8%
Shell 37.2%