Access to container engines: a public command/stream channel and open_container_engine() for netOrk's runtime driver #17

New Issue

2026-10-07T14:39:38Z

christianmanivong commented

2026-10-07 14:39:38 +00:00

Decided 2026-10-07, not built yet. This is the transport half of NetOrk/netork#765. netOrk gets a container-runtime driver (Docker, later Podman) that speaks the Docker Engine API. The OS drivers provide only the channel to the host, the same split SystemdServicesMixin already has: the logic sits in the mixin, and the driver carries the commands (_run_service_command).

What is needed

A public, documented channel on the base driver, implemented by each OS driver:

Run a command: run_command(command, *, privileged=False, timeout=…) -> (stdout, stderr, exit_code). It generalises _run_service_command and replaces netOrk's use of napalm-linux' private _send.
Open a byte stream: open_stream(command) -> stream with read, write and close, for docker system dial-stdio. The runtime driver speaks HTTP to the Engine API over it.

Per driver

napalm-linux (and the drivers built on it: OpenMediaVault, Proxmox, QNAP): SSH; a paramiko exec channel is bidirectional. privileged goes through the existing sudo handling.
napalm-windows: to be settled. WinRM/PSRP has no raw bidirectional stream. The options are Windows' OpenSSH server, or only run_command there and a CLI-JSON fallback in the runtime driver.

Guards

The contract is tested against a fake transport, as the systemd mixin's are.
privileged never puts a password on a command line (sudo -S over stdin, as today).
Without a channel, open_stream raises NotImplementedError, so the runtime driver can fall back or report it.

Prio: P3. Important for NetOrk/netork#765, not urgent.

**Decided 2026-10-07, not built yet.** This is the transport half of NetOrk/netork#765. netOrk gets a container-runtime driver (Docker, later Podman) that speaks the Docker Engine API. The OS drivers provide only the channel to the host, the same split `SystemdServicesMixin` already has: the logic sits in the mixin, and the driver carries the commands (`_run_service_command`). ## What is needed A **public**, documented channel on the base driver, implemented by each OS driver: - **Run a command:** `run_command(command, *, privileged=False, timeout=…) -> (stdout, stderr, exit_code)`. It generalises `_run_service_command` and replaces netOrk's use of napalm-linux' private `_send`. - **Open a byte stream:** `open_stream(command) -> stream` with read, write and close, for `docker system dial-stdio`. The runtime driver speaks HTTP to the Engine API over it. ## Per driver - **napalm-linux** (and the drivers built on it: OpenMediaVault, Proxmox, QNAP): SSH; a paramiko exec channel is bidirectional. `privileged` goes through the existing sudo handling. - **napalm-windows:** to be settled. WinRM/PSRP has no raw bidirectional stream. The options are Windows' OpenSSH server, or only `run_command` there and a CLI-JSON fallback in the runtime driver. ## Guards - The contract is tested against a fake transport, as the systemd mixin's are. - `privileged` never puts a password on a command line (`sudo -S` over stdin, as today). - Without a channel, `open_stream` raises `NotImplementedError`, so the runtime driver can fall back or report it. Prio: P3. Important for NetOrk/netork#765, not urgent.

christianmanivong commented

2026-10-07 14:41:27 +00:00

Decided in addition (2026-10-07): the container model belongs here, not only the channel.

Docker is one model of containers. The vendor-neutral vocabulary lives in this package; the logic that speaks the Docker Engine API lives in netOrk (NetOrk/netork#765).

Model

The model replaces DockerContainerDict, DockerImageDict, DockerVolumeDict, DockerNetworkDict and DockerInfoDict (models.py). Those are shaped by the CLI: ports, size, status and timestamps are text, and the runtime is implicitly Docker.

ContainerRuntimeDict:
- available, permission_denied;
- runtime (docker, podman, an appliance's name), version, api_version;
- containers, images, volumes, networks.
ContainerDict:
- id, name, state (fixed vocabulary: running, exited, paused, restarting, created, dead), created (ISO 8601);
- image_ref (the stable reference the container was created from), image_id (what actually runs), ports (list of {host_ip, host_port, container_port, protocol}), labels (dict), compose_project/compose_service when known.
ContainerImageDict:
- id, repository, tag;
- digests (repo digests), platform (os/arch[/variant]);
- size_bytes, created, version (OCI label).
Volume and network are structured the same way.

Role

A "container host" role with get_container_runtime() -> ContainerRuntimeDict. Capability by hasattr, as with the other roles.

Appliance drivers whose containers are reachable only through the device's own API (QNAP Container Station, Synology, TrueNAS) implement it there.
For ordinary hosts, netOrk's runtime driver fills the same model through the channel above.

Boundary: LXC (Proxmox) are system containers and stay VMs; this model is about application containers.

Order: model and role first, then the channel, then napalm-linux' get_docker_info retires in favour of netOrk's runtime driver.

**Decided in addition (2026-10-07): the container model belongs here, not only the channel.** Docker is one model of containers. The vendor-neutral vocabulary lives in this package; the logic that speaks the Docker Engine API lives in netOrk (NetOrk/netork#765). ## Model The model replaces `DockerContainerDict`, `DockerImageDict`, `DockerVolumeDict`, `DockerNetworkDict` and `DockerInfoDict` (`models.py`). Those are shaped by the CLI: ports, size, status and timestamps are text, and the runtime is implicitly Docker. - `ContainerRuntimeDict`: - `available`, `permission_denied`; - `runtime` (`docker`, `podman`, an appliance's name), `version`, `api_version`; - `containers`, `images`, `volumes`, `networks`. - `ContainerDict`: - `id`, `name`, `state` (fixed vocabulary: `running`, `exited`, `paused`, `restarting`, `created`, `dead`), `created` (ISO 8601); - `image_ref` (the stable reference the container was created from), `image_id` (what actually runs), `ports` (list of `{host_ip, host_port, container_port, protocol}`), `labels` (dict), `compose_project`/`compose_service` when known. - `ContainerImageDict`: - `id`, `repository`, `tag`; - `digests` (repo digests), `platform` (`os/arch[/variant]`); - `size_bytes`, `created`, `version` (OCI label). - Volume and network are structured the same way. ## Role A **"container host" role** with `get_container_runtime() -> ContainerRuntimeDict`. Capability by `hasattr`, as with the other roles. - Appliance drivers whose containers are reachable only through the device's own API (QNAP Container Station, Synology, TrueNAS) implement it there. - For ordinary hosts, netOrk's runtime driver fills the same model through the channel above. **Boundary:** LXC (Proxmox) are system containers and stay VMs; this model is about application containers. **Order:** model and role first, then the channel, then napalm-linux' `get_docker_info` retires in favour of netOrk's runtime driver.

christianmanivong changed title from ~~A public command and stream channel on the base driver, for netOrk's container-runtime driver~~ to A neutral container model, a container-host role, and a public command/stream channel for netOrk's runtime driver

2026-10-07 14:41:27 +00:00

christianmanivong commented

2026-10-07 14:48:52 +00:00

Revised 2026-10-07; this supersedes the comment above about the model. This package and the drivers built on it provide only access to a container engine. No model, no parsing, no logic: above the connection everything is specific to the service and lives in netOrk (NetOrk/netork#765).

What this issue now covers

The channel (unchanged): run_command(command, *, privileged=False, timeout=…) and open_stream(command).
Engine access:
- container_engines() -> list[...]: which engines the device offers, for example {"engine": "docker", "api": "docker-engine"}. An appliance may name its own API as api.
- open_container_engine(engine) -> stream | connection: a connection to that engine's API.
  - Generic default in the base, for drivers with a channel: open_stream("docker system dial-stdio"), with the privilege handling.
  - A driver overrides only the device-specific part of getting there. Examples: QNAP's docker binary under Container Station's path, or a socket in a non-standard place.
  - An appliance whose engine speaks something other than the Docker Engine API returns a connection to that API. What is said over it is decided by netOrk.

Withdrawn

The neutral container model and the "container host" role get_container_runtime() from the comment above. The model lives in netOrk. DockerInfoDict and the other Docker*Dict types here retire once napalm-linux' get_docker_info is replaced by netOrk's runtime driver.

Drivers that need this: napalm-linux and those built on it (OpenMediaVault, Proxmox, QNAP), later napalm-windows (channel to be settled).

**Revised 2026-10-07; this supersedes the comment above about the model.** This package and the drivers built on it provide **only access to a container engine**. No model, no parsing, no logic: above the connection everything is specific to the service and lives in netOrk (NetOrk/netork#765). ## What this issue now covers 1. **The channel** (unchanged): `run_command(command, *, privileged=False, timeout=…)` and `open_stream(command)`. 2. **Engine access:** - `container_engines() -> list[...]`: which engines the device offers, for example `{"engine": "docker", "api": "docker-engine"}`. An appliance may name its own API as `api`. - `open_container_engine(engine) -> stream | connection`: a connection to that engine's API. - Generic default in the base, for drivers with a channel: `open_stream("docker system dial-stdio")`, with the privilege handling. - A driver overrides only the device-specific part of *getting there*. Examples: QNAP's `docker` binary under Container Station's path, or a socket in a non-standard place. - An appliance whose engine speaks something other than the Docker Engine API returns a connection to that API. What is said over it is decided by netOrk. ## Withdrawn The neutral container model and the "container host" role `get_container_runtime()` from the comment above. The model lives in netOrk. `DockerInfoDict` and the other `Docker*Dict` types here retire once napalm-linux' `get_docker_info` is replaced by netOrk's runtime driver. Drivers that need this: napalm-linux and those built on it (OpenMediaVault, Proxmox, QNAP), later napalm-windows (channel to be settled).

christianmanivong changed title from ~~A neutral container model, a container-host role, and a public command/stream channel for netOrk's runtime driver~~ to Access to container engines: a public command/stream channel and open_container_engine() for netOrk's runtime driver

2026-10-07 14:48:52 +00:00

christianmanivong commented

2026-10-07 15:29:26 +00:00

Refined at plan review (2026-10-07, Christian): this package abstracts the connection, the driver builds it, and netOrk does the talking.

open_container_engine(engine) returns a ContainerEngineConnection, not a bare stream. It has three methods:
- open_api() -> ByteStream: the stream to the engine's API (default: <binary> system dial-stdio);
- run_cli(args: list[str], *, stdin: bytes | None = None, timeout) -> CommandResult;
- stream_cli(args: list[str]) -> ByteStream.
netOrk passes only arguments. Examples: ["compose", "-f", f, "--project-directory", d, "up", "-d", svc], ["pull", ref], ["buildx", "imagetools", "inspect", "--raw", ref].
The driver owns the binary path, quoting and privileges. The hook is _container_engine_binary(engine), defaulting to the engine name; QNAP returns its Container Station path. netOrk never sees the path.
container_engines() stays [{engine, api}].
The CLI is needed where the Engine API has no equivalent: compose, pulls that use the host's registry login, and private-registry digests.

This is Phase 1 of the plan in NetOrk/netork#765.

**Refined at plan review (2026-10-07, Christian):** this package abstracts the connection, the driver builds it, and netOrk does the talking. - `open_container_engine(engine)` returns a **`ContainerEngineConnection`**, not a bare stream. It has three methods: - `open_api() -> ByteStream`: the stream to the engine's API (default: `<binary> system dial-stdio`); - `run_cli(args: list[str], *, stdin: bytes | None = None, timeout) -> CommandResult`; - `stream_cli(args: list[str]) -> ByteStream`. - **netOrk passes only arguments.** Examples: `["compose", "-f", f, "--project-directory", d, "up", "-d", svc]`, `["pull", ref]`, `["buildx", "imagetools", "inspect", "--raw", ref]`. - **The driver owns the binary path, quoting and privileges.** The hook is `_container_engine_binary(engine)`, defaulting to the engine name; QNAP returns its Container Station path. **netOrk never sees the path.** - `container_engines()` stays `[{engine, api}]`. - The CLI is needed where the Engine API has no equivalent: compose, pulls that use the host's registry login, and private-registry digests. This is Phase 1 of the plan in NetOrk/netork#765.

christianmanivong referenced this issue from a commit

2026-10-07 15:58:58 +00:00

feat: a public command channel and container engine access

christianmanivong referenced a pull request that will close this issue

2026-10-07 15:59:08 +00:00

feat: a public command channel and container engine access #18

christianmanivong referenced this issue from a commit

2026-10-07 15:59:34 +00:00

feat: run commands and streams on an exec channel, reach the container engine

christianmanivong referenced this issue from NAPALM/napalm-linux

2026-10-07 15:59:44 +00:00

feat: run commands and streams on an exec channel, reach the container engine #18