fix: one deploy per host at a time, under a lock the host frees on its own
CI / check (pull_request) Successful in 17s
CI / check (pull_request) Successful in 17s
Several sessions deploy to the same test server, and two runs used to overlap. On 2026-10-05 two `up -d --force-recreate` runs recreated each other's containers and the API was down for a minute (#1). On 2026-10-06 two runs renamed each other's *.new compose files and one broke off; with two different tags, one tag's compose files could have started the other's images (#3). - Each deploy first takes flock on ~/netork/.deploy.lock on the host. An ssh session holds it: the remote side takes the lock on fd 9, reports LOCKED and waits on its stdin, so ending the session frees it, whether the deploy finished, failed, was interrupted or lost its connection. Checked over real ssh on .50, including a client killed with -9. - A second deploy prints who holds the lock, since when and with which tag, and waits up to DEPLOY_LOCK_WAIT seconds (default 900); then it gives up without touching the host. - A host without flock is deployed without the lock, with a warning. - deploy_server is now the lock around deploy_steps, the old body. Closes #1 Closes #3
This commit is contained in:
@@ -25,9 +25,11 @@ import pytest
|
||||
DEPLOY = Path(__file__).resolve().parent.parent / "deploy.sh"
|
||||
|
||||
_FAKE_SSH = """#!/bin/sh
|
||||
for arg in "$@"; do cmd="$arg"; done
|
||||
# The host lock's holder runs for real, here (test_deploy_host_lock.py).
|
||||
case "$cmd" in *.deploy.lock*) exec sh -c "$cmd" ;; esac
|
||||
# Swallow stdin (docker login, the .env heredoc), then answer by command.
|
||||
cat > /dev/null
|
||||
for arg in "$@"; do cmd="$arg"; done
|
||||
case "$cmd" in
|
||||
*--force-recreate*)
|
||||
n=$(cat "{state}/recreates" 2>/dev/null || echo 0)
|
||||
|
||||
Reference in New Issue
Block a user