From e43e68553f4034b1e9e90ca742b72d1aa103b976 Mon Sep 17 00:00:00 2001 From: Christian Manivong Date: Sun, 18 Oct 2026 21:15:23 +0200 Subject: [PATCH] docs: write the README for people installing netOrk themselves The repository is meant for customers who run netOrk on their own hosts. Drop the development section, the internal build tags (latest-dev, main-, feature-), CI wording and issue references from the README and deploy.env.example; only latest and release numbers are documented. --- README.md | 124 ++++++++++++++------------------------------- deploy.env.example | 4 +- 2 files changed, 41 insertions(+), 87 deletions(-) diff --git a/README.md b/README.md index 19b6dd9..42079a4 100644 --- a/README.md +++ b/README.md @@ -1,12 +1,12 @@ # netOrk deploy -Deploys [netOrk](https://git.netork.io/NetOrk/netork) to one or more Docker hosts in -parallel, from pre-built images in a container registry. +Installs and updates [netOrk](https://git.netork.io/NetOrk/netork) on one or more of your +own Docker hosts, from netOrk's published images. -The machine running the deploy needs this repository, `ssh`, and a `deploy.env`. It does -not need a netOrk checkout. Each server pulls the images itself. The compose files come -out of the engine image for the tag being deployed, so they always match the images they -start, and database migrations run from that same image. +The machine you run the deploy from needs this repository, `ssh` and a `deploy.env`. +Each server pulls the images itself. The compose files come out of the image for the +version being deployed, so they always match the images they start, and database +migrations run from that same image. ## Requirements @@ -15,24 +15,16 @@ start, and database migrations run from that same image. - `ssh` with key-based access to every target **On every target server:** -- at least 4 GB of RAM. On a test server with 74 devices, netOrk's containers took about - 2.4 GB, and at least 0.9 GB stayed free. A large task needs that room. The breakdown - per container is in netOrk's `docs/TECHNICAL_DESIGN.md`, "Worker memory". +- at least 4 GB of RAM. On a server with 74 devices, netOrk's containers took about + 2.4 GB, and at least 0.9 GB stayed free. Larger tasks need that room. - Docker with the compose plugin - a directory `~/netork/` with netOrk's `.env`, which holds the database and application - settings the compose file reads (`env_file: .env`) - -**Images:** `netork/engine` and `netork/ui` must be published in the registry under the -tag you deploy. The engine image must carry its compose files under `/app/deploy/`. -Tags built before that change are refused with a clear message. + settings **Raspberry Pi and other arm64 hosts:** a Raspberry Pi 3, 4, 5 or Zero 2 W with a 64-bit -OS can be a target. Nothing changes in `deploy.env`: each host pulls the variant for its -own platform. netOrk publishes arm64 images for releases only, from 0.34.0 on (`latest`, -`X.Y.Z`). `latest-dev`, `main-`, `feature-` and older releases are amd64 only, -and a deploy of such a tag to an arm64 host is refused before the host is touched. The -images on the host (`postgres`, `redis`, `apt-cacher-ng`, `signal-api`) exist for arm64 -as well. +OS can be a target, from netOrk 0.34.0 on. Nothing changes in `deploy.env`: each host +pulls the variant for its own platform. A version that has no arm64 image is refused +before the host is touched. ## Setup @@ -41,15 +33,14 @@ cp deploy.env.example deploy.env $EDITOR deploy.env ``` -`deploy.env` is gitignored. Point `DEPLOY_ENV_FILE` at another file to keep it -elsewhere. +Set `DEPLOY_ENV_FILE` to keep the file somewhere else. | Variable | Meaning | |---|---| | `DEPLOY_SERVERS` | Space-separated default targets | -| `UI_SERVER` | Targets that also run `netork-ui` | +| `UI_SERVER` | Targets that also run the web interface (`netork-ui`) | | `REGISTRY_HOST` | Registry to pull from; required | -| `NETORK_VERSION` | Tag to deploy (default `latest`) | +| `NETORK_VERSION` | Version to deploy (default `latest`) | | `REGISTRY_USER`, `REGISTRY_PASSWORD` | Registry login used on every server | | `REGISTRY_USER_`, `REGISTRY_PASSWORD_`, `NETORK_VERSION_` | Per-server overrides. `` has its dots replaced by underscores, e.g. `_10_0_0_2` | | `DEPLOY_LOCK_WAIT` | Seconds to wait for another deploy to the same host (default 900) | @@ -60,60 +51,37 @@ elsewhere. ./deploy.sh # every server in DEPLOY_SERVERS ./deploy.sh 10.0.0.1 # one server ./deploy.sh 10.0.0.1 10.0.0.2 # several, in parallel -./deploy.sh --version=main-1a2b3c4 10.0.0.1 # one-off tag, this run only +./deploy.sh --version=0.37.0 10.0.0.1 # a specific version, this run only ``` -Only tags netOrk's CI publishes are accepted: -- `latest` -- `latest-dev` -- `main-` -- `feature-`, a branch build: `feature/` and `feat/` become - `feature-`, and `fix/` becomes `feature-fix-`. Any further `/` in - the branch name becomes `-` as well. -- `X.Y.Z` +`NETORK_VERSION` and `--version` take `latest` for the newest release or a release +number such as `0.37.0`. -Anything else is refused before any server is touched. - -If you deploy a branch build, wait until its CI run has published the images. A deploy -started earlier pulls whatever image the registry held before, which is stale. +The same command installs netOrk the first time and updates it later. ## What a deploy does, per server -0. Takes the host's deploy lock, `flock` on `~/netork/.deploy.lock`, and holds it until - the deploy ends. A second deploy to the same host waits for it and says who holds - it, since when, and which tag they are deploying. It gives up after 15 minutes - (`DEPLOY_LOCK_WAIT`, in seconds) without touching the host. The lock belongs to an - ssh session, so a deploy that fails, is interrupted or loses its connection frees it - on its own. A host without `flock` is deployed without the lock, with a warning. -1. Logs in to the registry. The password travels over ssh's stdin, never on a command - line. Then it checks that the tag has an image for the host's platform (`docker - version`, `docker manifest inspect`): engine, and on UI servers ui. If not, for - example `latest-dev` on a Raspberry Pi, it stops here and says which tags would work. - If the registry's answer can't be read, the pull below decides. -2. Pulls the engine image and copies `docker-compose.yml` and - `docker-compose.registry.yml` out of it into `~/netork/`. -3. Pulls the netOrk images. -4. Records `REGISTRY_HOST` and `NETORK_VERSION` in `~/netork/.env`, so that a - hand-typed `docker compose` on the host uses the same images. It does this before - any container starts, because the services read `.env` into their environment. - A failed pull leaves `.env` as it was. -5. Force-recreates the API, the workers, `netork-beat`, `flower` and, on UI servers, - `netork-ui`. If the recreate fails, it is tried once more after 5 s - (`DEPLOY_RECREATE_RETRY_DELAY`). Compose's parallel recreate can lose a container it - just renamed and leave the rest stopped. If the second attempt fails too, the - container states are printed and the deploy fails. -6. Reconciles `apt-cacher-ng` and `signal-api`: each is recreated only if its - definition changed. It never touches `postgres` or `redis`. - It then removes services netOrk no longer ships, container and volume, where a host - still has them: today the bundled `registry` (NetOrk/netork#763). It reports only - what it actually removed; on a host that was cleaned earlier it says nothing. -7. Verifies that the containers run exactly the image that was pulled. If they don't, the - deploy fails. -8. Runs `alembic upgrade head` inside `netork-api`. -9. Prunes unused images. +1. Makes sure only one deploy runs on the host at a time. A second deploy to the same + host waits, says who is deploying which version, and gives up after 15 minutes + (`DEPLOY_LOCK_WAIT`) without touching the host. +2. Logs in to the registry. The password travels over ssh's stdin, never on a command + line. Then it checks that the version has an image for the host's platform and stops + if it doesn't. +3. Pulls the images and copies the matching compose files into `~/netork/`. +4. Records the registry and version in `~/netork/.env`, so that a `docker compose` + command typed on the host uses the same images. A failed pull leaves `.env` as it + was. +5. Restarts netOrk's containers on the new images. If that fails, it tries once more; + if it fails again, it prints the container states and stops. +6. Updates the helper services (`apt-cacher-ng`, `signal-api`) only when their + definition changed, and removes services netOrk no longer ships. It never touches + the database (`postgres`) or `redis`. +7. Verifies that the containers run exactly the images that were pulled. +8. Applies database migrations. +9. Removes unused images. -A failing step stops that server's deploy with a non-zero exit, and the other servers -carry on. The script exits non-zero if any server failed and names those servers. +If a step fails, that server's deploy stops and the other servers carry on. The script +exits non-zero if any server failed and names those servers. ## Operating notes @@ -121,24 +89,10 @@ carry on. The script exits non-zero if any server failed and names those servers - `celerybeat_schedule` holds the scheduler state. - `signal_cli_data` holds netOrk's Signal device link. Losing it means pairing again by QR code. -- `netork-beat` always rolls out together with the workers. The script does this for - you; keep it that way if you deploy by hand. - Always deploy with this script. Do not point a local Docker client at a remote host (`DOCKER_HOST=ssh://…`): it resolves volume paths locally and breaks the remote containers. -## Development - -```bash -pip install pytest ruff shellcheck-py -shellcheck deploy.sh -ruff format --check . && ruff check . -pytest -``` - -The tests are static and behavioural checks of `deploy.sh`. None of them reach a real -host. - ## License MIT, see [LICENSE](LICENSE). diff --git a/deploy.env.example b/deploy.env.example index 2336256..7c613c5 100644 --- a/deploy.env.example +++ b/deploy.env.example @@ -10,7 +10,7 @@ UI_SERVER="10.0.0.1" # Registry the servers pull netOrk's images from. REGISTRY_HOST=registry.example.com -# Image tag to deploy: latest, latest-dev, main-, feature- or X.Y.Z. +# Version to deploy: latest for the newest release, or a release number such as 0.37.0. NETORK_VERSION=latest # Registry credentials used on every server without its own entry below. @@ -21,4 +21,4 @@ REGISTRY_PASSWORD= # with dots replaced by underscores. REGISTRY_USER_10_0_0_2= REGISTRY_PASSWORD_10_0_0_2= -NETORK_VERSION_10_0_0_2=latest-dev +NETORK_VERSION_10_0_0_2=0.37.0