feat: initial QNAP QTS driver scaffold
QTS is a Linux distribution, so the driver inherits the OS surface from LinuxDriver and adds QNAP's storage, QPKG and virtualisation layers. What is here: the class with its discovery fingerprints, per-session detection of the QTS major version and of the QPKG-local docker and virsh binaries, and explicit resolution of the MRO collisions that StorageDriver creates over LinuxDriver. What is not: the storage, QPKG and VM parsers. Those need real command output from QTS 4 and QTS 5 hardware to be written against, which is what tools/harvest.sh collects and tools/sanitize.py anonymises. Two tests are marked xfail(strict) as the specification for that work.
This commit is contained in:
+13
@@ -0,0 +1,13 @@
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*.egg-info/
|
||||
build/
|
||||
dist/
|
||||
.venv/
|
||||
.pytest_cache/
|
||||
.mypy_cache/
|
||||
.ruff_cache/
|
||||
.coverage
|
||||
htmlcov/
|
||||
# Raw, unsanitised harvest output — never commit device data.
|
||||
tools/harvest-out/
|
||||
@@ -0,0 +1,25 @@
|
||||
# Changelog
|
||||
|
||||
All notable changes to this project are documented here.
|
||||
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
- Initial driver scaffold: `QnapQtsDriver(StorageDriver, LinuxDriver)` with
|
||||
discovery fingerprints (SNMP enterprise OID 1.3.6.1.4.1.24681, HTTP, port
|
||||
specs) and the `qnap_qts` entry point.
|
||||
- Explicit `DEVICE_CLASS = "storage"` so netOrk does not classify a QNAP as a
|
||||
plain Linux host and hide its Storage tab.
|
||||
- QTS major-version detection, Container Station docker path discovery and
|
||||
Virtualization Station virsh discovery, all resolved once per session in
|
||||
`open()`.
|
||||
- MRO collision resolution for `get_services`, which `StorageDriver` would
|
||||
otherwise shadow.
|
||||
- `tools/harvest.sh` and `tools/sanitize.py` for capturing and anonymising
|
||||
command output from real hardware.
|
||||
|
||||
### Pending
|
||||
- Storage, QPKG and VM parsers — blocked on the fixture harvest.
|
||||
@@ -0,0 +1,132 @@
|
||||
# napalm-qnap-qts
|
||||
|
||||
NAPALM driver for QNAP NAS systems running QTS, over SSH.
|
||||
|
||||
QTS is a Linux distribution, so this driver inherits the whole OS surface from
|
||||
[`napalm-linux`](https://git.netork.io/christianmanivong/napalm-linux) —
|
||||
packages, services, users, processes, Docker — and adds QNAP's storage, QPKG
|
||||
and virtualisation layers on top.
|
||||
|
||||
Tested against QTS 4.x and QTS 5.x.
|
||||
|
||||
## Status
|
||||
|
||||
The class scaffold, discovery fingerprints and version/tool detection are in
|
||||
place. The storage, QPKG and VM parsers are **not written yet**: they are
|
||||
blocked on capturing real command output from hardware (see
|
||||
[Harvesting fixtures](#harvesting-fixtures)). Writing parsers against guessed
|
||||
output is how a driver ends up passing its own tests and failing on a real NAS.
|
||||
|
||||
| Method | Source | Status |
|
||||
|---|---|---|
|
||||
| `get_facts` | `getcfg`, `getsysinfo`, `/proc/uptime` | pending harvest |
|
||||
| `get_interfaces`, `get_interfaces_ip` | `ip` / `ifconfig` | pending harvest |
|
||||
| `get_disks` | `qcli_storage -d`, `get_hd_smartinfo` | pending harvest |
|
||||
| `get_disk_pools` | `qcli_storage -p`, `/proc/mdstat` | pending harvest |
|
||||
| `get_volumes` | `qcli_storage -v`, `df` | pending harvest |
|
||||
| `get_shares` | `/etc/config/smb.conf`, `/etc/exports` | pending harvest |
|
||||
| `get_storage_services` | `getcfg`, `ss -lntup` | pending harvest |
|
||||
| `get_disk_smart` | `get_hd_smartinfo` | pending harvest |
|
||||
| `get_packages` | QPKG (`qpkg.conf` / `qpkg_cli`) | pending harvest |
|
||||
| `get_vms` | `virsh` (Virtualization Station) | pending harvest |
|
||||
| `start_vm`, `stop_vm`, `reboot_vm` | `virsh` | pending harvest |
|
||||
| `set_service_enabled` | `setcfg`, `/etc/init.d` | pending harvest |
|
||||
| `get_device_warnings` | derived from the above | pending harvest |
|
||||
| `get_docker_info` | inherited, via `_docker_bin()` | ✅ |
|
||||
| `get_services`, `get_users`, `get_processes` | inherited from `LinuxDriver` | ✅ |
|
||||
| `get_health_metrics` | inherited (UCD-MIB over SNMP) | ✅ |
|
||||
| `ping`, `ping_sweep` | inherited from `LinuxDriver` | ✅ |
|
||||
| Snapshots, quotas, replication, QPKG install/remove | — | out of scope for v1 |
|
||||
|
||||
## Requirements
|
||||
|
||||
SSH must be enabled on the NAS: **Control Panel → Telnet/SSH → Allow SSH
|
||||
connection**. The driver connects on port 22 by default; the QTS web UI on
|
||||
443/8080 is not used.
|
||||
|
||||
## Install
|
||||
|
||||
```bash
|
||||
pip install -e vendor/napalm-device-types/ -e vendor/napalm-linux/ -e vendor/napalm-qnap-qts/
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
```python
|
||||
from napalm_qnap_qts import QnapQtsDriver
|
||||
|
||||
driver = QnapQtsDriver("nas.example.lan", "admin", "secret")
|
||||
driver.open()
|
||||
print(driver.get_facts())
|
||||
driver.close()
|
||||
```
|
||||
|
||||
Recognised `optional_args`: everything `napalm-linux` accepts (`port`,
|
||||
`sudo_password`, `secret`, …). Unknown keys are ignored.
|
||||
|
||||
## Design notes
|
||||
|
||||
### Inheritance order
|
||||
|
||||
```python
|
||||
class QnapQtsDriver(StorageDriver, LinuxDriver):
|
||||
```
|
||||
|
||||
`StorageDriver` precedes `LinuxDriver` in the MRO, so its
|
||||
`NotImplementedError` stubs shadow LinuxDriver's working implementations
|
||||
wherever the names collide — `get_services`, `get_packages`,
|
||||
`install_package`. Each collision is resolved with an explicit forwarding
|
||||
method; `TestMroForwarding` guards that they stay resolved.
|
||||
|
||||
NAS services are exposed as `get_storage_services()`, not `get_services()`:
|
||||
netOrk's poller reads the former for the storage snapshot and the latter for
|
||||
the OS service list. Same convention as `napalm-openmediavault`.
|
||||
|
||||
### Device class
|
||||
|
||||
The driver sets `DEVICE_CLASS = "storage"` explicitly. Without it netOrk's
|
||||
`issubclass` chain reaches `OSDriver` before `StorageDriver` and would file a
|
||||
QNAP under "linux", hiding its Storage tab. VMs and containers stay visible
|
||||
through capability introspection (`supports_vms`), not through this key — a
|
||||
QNAP running Virtualization Station is a NAS *and* a hypervisor, and
|
||||
`device_class` only holds one of those.
|
||||
|
||||
### Docker
|
||||
|
||||
Container Station does not put `docker` on `PATH`; it lives under
|
||||
`/share/<pool>/.qpkg/container-station/`. The Docker *logic* stays in
|
||||
`LinuxDriver` and only the path is overridden here, via the `_docker_bin()`
|
||||
hook.
|
||||
|
||||
## Harvesting fixtures
|
||||
|
||||
```bash
|
||||
./tools/harvest.sh admin@nas4.example.lan qts4
|
||||
./tools/harvest.sh admin@nas5.example.lan qts5
|
||||
./tools/sanitize.py tools/harvest-out/qts5 --extra-host mynas
|
||||
```
|
||||
|
||||
`harvest.sh` runs the command set this driver parses over a single SSH session
|
||||
and writes one file per command. `sanitize.py` replaces serial numbers, MACs,
|
||||
IP addresses and hostnames with stable placeholders — cross-references between
|
||||
files survive, so the fixtures still describe one coherent device.
|
||||
|
||||
**Read the sanitised output before committing it.** The sanitiser catches
|
||||
patterns, not judgement, and fixtures live in git forever. Raw harvest output
|
||||
is gitignored and must never be committed.
|
||||
|
||||
Where QTS 4 and QTS 5 differ, keep both fixtures and parametrise the test over
|
||||
the pair, so the version divergence is part of the test matrix rather than a
|
||||
later surprise.
|
||||
|
||||
## Development
|
||||
|
||||
```bash
|
||||
pip install -e ".[dev]"
|
||||
python -m pytest -q
|
||||
ruff check .
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
Apache-2.0
|
||||
@@ -0,0 +1,5 @@
|
||||
"""napalm-qnap-qts – NAPALM driver for QNAP NAS systems running QTS."""
|
||||
|
||||
from napalm_qnap_qts.qnap_qts import QnapQtsDriver
|
||||
|
||||
__all__ = ["QnapQtsDriver"]
|
||||
@@ -0,0 +1,156 @@
|
||||
# Licensed under the Apache License, Version 2.0
|
||||
|
||||
"""NAPALM driver for QNAP NAS systems running QTS.
|
||||
|
||||
QTS is a Linux distribution, so this driver inherits the whole OS surface from
|
||||
:class:`napalm_linux.linux.LinuxDriver` — packages, services, users, processes,
|
||||
Docker — and adds QNAP's own storage, QPKG and virtualisation layers on top:
|
||||
|
||||
* Physical disk inventory and SMART via ``qcli_storage`` and ``get_hd_smartinfo``
|
||||
* Storage pools and volumes via ``qcli_storage`` and ``/proc/mdstat``
|
||||
* SMB/NFS/AFP/FTP shares from ``/etc/config/smb.conf`` and ``/etc/exports``
|
||||
* QPKG packages instead of a distribution package manager
|
||||
* Virtualization Station guests via ``virsh``
|
||||
* Container Station via LinuxDriver's Docker support, redirected to the
|
||||
QPKG-local ``docker`` binary
|
||||
|
||||
Connects via SSH. SSH has to be enabled on the NAS first
|
||||
(Control Panel → Telnet/SSH). Tested against QTS 4.x and QTS 5.x.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from typing import Any
|
||||
|
||||
from napalm_device_types import FingerprintRule, PortSpec, StorageDriver
|
||||
from napalm_linux.linux import LinuxDriver
|
||||
|
||||
#: QNAP Systems' IANA enterprise number. Used by discovery to recognise a NAS
|
||||
#: from its SNMP sysObjectID before anyone has supplied credentials.
|
||||
QNAP_ENTERPRISE_OID = "1.3.6.1.4.1.24681"
|
||||
|
||||
#: Where Container Station puts the docker binary. It is not on PATH, and the
|
||||
#: volume name varies with which pool the app was installed on, so this is a
|
||||
#: glob evaluated on the device rather than a fixed path.
|
||||
_DOCKER_GLOBS = (
|
||||
"/share/*/.qpkg/container-station/bin/docker",
|
||||
"/share/*/.qpkg/container-station/usr/bin/docker",
|
||||
)
|
||||
|
||||
_VERSION_RE = re.compile(r"(\d+)\.")
|
||||
|
||||
|
||||
class QnapQtsDriver(StorageDriver, LinuxDriver):
|
||||
"""NAPALM driver for QNAP NAS systems running QTS.
|
||||
|
||||
**Inheritance order matters.** ``StorageDriver`` precedes ``LinuxDriver`` in
|
||||
the MRO, so its ``NotImplementedError`` stubs shadow LinuxDriver's working
|
||||
implementations wherever the names collide — ``get_services``,
|
||||
``get_packages`` and ``install_package``. Each collision is resolved
|
||||
explicitly below rather than left to the MRO; see ``TestMroForwarding``.
|
||||
|
||||
NAS services are exposed as ``get_storage_services()``, not
|
||||
``get_services()``, because netOrk's poller reads the former for the storage
|
||||
snapshot and the latter for the OS service list. Same convention as
|
||||
napalm-openmediavault.
|
||||
"""
|
||||
|
||||
TYPE_LABEL = "Storage"
|
||||
# Declared outright: the driver inherits LinuxDriver for the OS surface, so
|
||||
# netOrk's issubclass chain would reach OSDriver first and classify a QNAP
|
||||
# as "linux", hiding its Storage tab. VMs and containers stay visible
|
||||
# through capability introspection, not through this key.
|
||||
DEVICE_CLASS = "storage"
|
||||
VENDOR = "QNAP"
|
||||
DRIVER_NAME = "qnap_qts"
|
||||
driver_name = "qnap_qts"
|
||||
NETMIKO_DEVICE_TYPE = "linux"
|
||||
|
||||
SNMP_OBJECT_ID_PREFIX = QNAP_ENTERPRISE_OID
|
||||
SNMP_FINGERPRINT = [
|
||||
FingerprintRule("qnap", weight=9.0),
|
||||
FingerprintRule("nas", weight=1.0),
|
||||
]
|
||||
# QTS answers with a stock OpenSSH banner, so SSH alone cannot identify a
|
||||
# QNAP — it only confirms the transport this driver needs.
|
||||
SSH_FINGERPRINT = [
|
||||
FingerprintRule("openssh", weight=1.0),
|
||||
]
|
||||
# Mandatory: without it every unidentified NAS web UI would score as a QNAP.
|
||||
HTTP_FINGERPRINT = [
|
||||
FingerprintRule("qnap", weight=9.0, mandatory=True),
|
||||
FingerprintRule("qts", weight=4.0),
|
||||
]
|
||||
PORT_SPECS = [
|
||||
PortSpec("https", 443),
|
||||
PortSpec("http", 8080),
|
||||
]
|
||||
|
||||
# ── Lifecycle ─────────────────────────────────────────────────────────────
|
||||
|
||||
def open(self) -> None:
|
||||
"""Connect, then resolve the facts that decide which code paths run.
|
||||
|
||||
QTS 4 and QTS 5 differ in the output of several tools, and the QPKG apps
|
||||
that provide docker and virsh live on whichever storage pool they were
|
||||
installed on. Resolving all of that once per session keeps every getter
|
||||
below free of discovery round trips.
|
||||
"""
|
||||
super().open()
|
||||
self._qts_major = self._detect_qts_major()
|
||||
self._docker_path = self._discover_docker_path()
|
||||
self._virsh_path = self._discover_virsh_path()
|
||||
|
||||
def _detect_qts_major(self) -> int | None:
|
||||
"""Return the QTS major version, or None when it cannot be read.
|
||||
|
||||
Deliberately not fatal: a NAS that answers nothing useful here is still
|
||||
worth polling, and the newer code path is the better default.
|
||||
"""
|
||||
try:
|
||||
raw = self._send("getcfg System Version").strip()
|
||||
except Exception:
|
||||
return None
|
||||
match = _VERSION_RE.match(raw)
|
||||
return int(match.group(1)) if match else None
|
||||
|
||||
def _discover_docker_path(self) -> str:
|
||||
"""Locate Container Station's docker binary, falling back to PATH."""
|
||||
try:
|
||||
found = self._send(f"ls {' '.join(_DOCKER_GLOBS)} 2>/dev/null | head -1").strip()
|
||||
except Exception:
|
||||
return "docker"
|
||||
return found.splitlines()[0].strip() if found else "docker"
|
||||
|
||||
def _discover_virsh_path(self) -> str | None:
|
||||
"""Locate Virtualization Station's virsh, or None when it is not installed.
|
||||
|
||||
None is a normal outcome — plenty of QNAP models never run VMs — and it
|
||||
is what makes ``get_vms`` return an empty list instead of raising.
|
||||
"""
|
||||
try:
|
||||
found = self._send(
|
||||
"ls /share/*/.qpkg/QKVM/usr/bin/virsh /share/*/.qpkg/*/bin/virsh "
|
||||
"2>/dev/null | head -1"
|
||||
).strip()
|
||||
except Exception:
|
||||
return None
|
||||
return found.splitlines()[0].strip() or None if found else None
|
||||
|
||||
def _docker_bin(self) -> str:
|
||||
"""Override LinuxDriver's hook: docker is not on PATH under QTS."""
|
||||
return getattr(self, "_docker_path", None) or "docker"
|
||||
|
||||
# ── MRO collision resolution ──────────────────────────────────────────────
|
||||
#
|
||||
# StorageDriver comes first in the MRO and its stubs would otherwise win.
|
||||
|
||||
def get_services(self) -> Any:
|
||||
"""OS services, not NAS services — this is what netOrk's poller reads.
|
||||
|
||||
Forwarded explicitly past ``StorageDriver.get_services``, which shadows
|
||||
it and returns a different shape (dict of NAS services vs. list of OS
|
||||
services).
|
||||
"""
|
||||
return LinuxDriver.get_services(self)
|
||||
@@ -0,0 +1,81 @@
|
||||
[build-system]
|
||||
requires = ["setuptools>=68", "wheel"]
|
||||
build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "napalm-qnap-qts"
|
||||
version = "0.1.0"
|
||||
description = "NAPALM driver for QNAP NAS systems running QTS via SSH"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.9"
|
||||
license = { text = "Apache-2.0" }
|
||||
authors = [
|
||||
{ name = "Christian Manivong", email = "christian@manivong.de" },
|
||||
]
|
||||
keywords = [
|
||||
"napalm",
|
||||
"network",
|
||||
"automation",
|
||||
"qnap",
|
||||
"qts",
|
||||
"nas",
|
||||
"storage",
|
||||
"ssh",
|
||||
"driver",
|
||||
]
|
||||
classifiers = [
|
||||
"Development Status :: 3 - Alpha",
|
||||
"Intended Audience :: Developers",
|
||||
"Intended Audience :: System Administrators",
|
||||
"License :: OSI Approved :: Apache Software License",
|
||||
"Operating System :: OS Independent",
|
||||
"Programming Language :: Python :: 3",
|
||||
"Programming Language :: Python :: 3.9",
|
||||
"Programming Language :: Python :: 3.10",
|
||||
"Programming Language :: Python :: 3.11",
|
||||
"Programming Language :: Python :: 3.12",
|
||||
"Topic :: System :: Networking",
|
||||
"Topic :: System :: Systems Administration",
|
||||
"Typing :: Typed",
|
||||
]
|
||||
# napalm-linux is a hard dependency, not an optional one: QTS is a Linux system
|
||||
# and this driver inherits the whole OS surface (packages, services, users,
|
||||
# processes, Docker) from LinuxDriver rather than reimplementing it.
|
||||
dependencies = [
|
||||
"napalm>=4.0",
|
||||
"napalm-device-types>=0.5.0",
|
||||
"napalm-linux>=0.1.0",
|
||||
"netmiko>=4.0.0",
|
||||
"paramiko>=5.0.0", # CVE-2026-44405
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
dev = [
|
||||
"pytest",
|
||||
"pytest-cov",
|
||||
"black",
|
||||
"ruff",
|
||||
"mypy",
|
||||
]
|
||||
|
||||
[project.entry-points."napalm.drivers"]
|
||||
qnap_qts = "napalm_qnap_qts:QnapQtsDriver"
|
||||
|
||||
[project.urls]
|
||||
Repository = "https://git.netork.io/christianmanivong/napalm-qnap-qts"
|
||||
|
||||
[tool.setuptools.packages.find]
|
||||
where = ["."]
|
||||
include = ["napalm_qnap_qts*"]
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
testpaths = ["tests"]
|
||||
|
||||
[tool.ruff]
|
||||
line-length = 100
|
||||
target-version = "py39"
|
||||
|
||||
# Same rule set the netOrk repo gates on, so this driver is held to the
|
||||
# standard of the project that consumes it rather than to ruff's defaults.
|
||||
[tool.ruff.lint]
|
||||
select = ["E", "F", "I", "UP"]
|
||||
@@ -0,0 +1,165 @@
|
||||
"""Unit tests for the QNAP QTS driver.
|
||||
|
||||
Command-output fixtures are captured from real hardware via tools/harvest.sh —
|
||||
one QTS 4 box and one QTS 5 box — and pasted in as module constants. Parsers are
|
||||
written against those, never against guessed output.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from napalm_qnap_qts import QnapQtsDriver
|
||||
|
||||
#: Parsers need real command output to be written against. These tests are the
|
||||
#: specification for the work that tools/harvest.sh unblocks. strict=True means
|
||||
#: the suite goes red the moment one starts passing, which is the reminder to
|
||||
#: delete the marker rather than leave it lying around.
|
||||
_PENDING_HARVEST = "blocked on fixture harvest from real QTS 4 / QTS 5 hardware"
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def driver():
|
||||
"""A driver with its transport mocked out, bypassing __init__.
|
||||
|
||||
Same shape as napalm-linux's fixture: nothing in __init__ needs patching,
|
||||
so constructing the object by hand is cheaper and clearer than mocking
|
||||
ConnectHandler.
|
||||
"""
|
||||
d = QnapQtsDriver.__new__(QnapQtsDriver)
|
||||
d.hostname = "testnas"
|
||||
d.username = "admin"
|
||||
d.password = "pass" # noqa: S105
|
||||
d.timeout = 60
|
||||
d.port = 22
|
||||
d._secret = "pass" # noqa: S105
|
||||
d._forced_pkg_manager = None
|
||||
d._pkg_manager = None
|
||||
d._sudo_password = None
|
||||
d.netmiko_optional_args = {}
|
||||
d._device = MagicMock()
|
||||
d._qts_major = 5
|
||||
d._docker_path = "docker"
|
||||
d._virsh_path = None
|
||||
return d
|
||||
|
||||
|
||||
class TestDriverIdentity:
|
||||
"""Read without a connection by netOrk's discovery and /drivers endpoint."""
|
||||
|
||||
def test_driver_name(self):
|
||||
assert QnapQtsDriver.DRIVER_NAME == "qnap_qts"
|
||||
|
||||
def test_lowercase_driver_name_alias_matches(self):
|
||||
"""netOrk's register_driver path reads the lowercase attribute."""
|
||||
assert QnapQtsDriver.driver_name == QnapQtsDriver.DRIVER_NAME
|
||||
|
||||
def test_vendor(self):
|
||||
assert QnapQtsDriver.VENDOR == "QNAP"
|
||||
|
||||
def test_type_label_is_storage(self):
|
||||
assert QnapQtsDriver.TYPE_LABEL == "Storage"
|
||||
|
||||
def test_declares_device_class_explicitly(self):
|
||||
"""Inheriting LinuxDriver would otherwise get it classified as "linux",
|
||||
and netOrk would hide the Storage tab."""
|
||||
assert QnapQtsDriver.DEVICE_CLASS == "storage"
|
||||
|
||||
def test_declares_at_least_one_fingerprint_source(self):
|
||||
"""Discovery silently skips a driver that declares no fingerprint data."""
|
||||
assert (
|
||||
QnapQtsDriver.HTTP_FINGERPRINT
|
||||
or QnapQtsDriver.SNMP_FINGERPRINT
|
||||
or QnapQtsDriver.SSH_FINGERPRINT
|
||||
)
|
||||
|
||||
def test_snmp_object_id_is_the_qnap_enterprise_oid(self):
|
||||
assert QnapQtsDriver.SNMP_OBJECT_ID_PREFIX == "1.3.6.1.4.1.24681"
|
||||
|
||||
def test_http_fingerprint_is_mandatory_to_avoid_matching_any_nas(self):
|
||||
patterns = {r.pattern: r for r in QnapQtsDriver.HTTP_FINGERPRINT}
|
||||
assert patterns["qnap"].mandatory is True
|
||||
|
||||
|
||||
class TestMroForwarding:
|
||||
"""StorageDriver precedes LinuxDriver in the MRO, so its NotImplementedError
|
||||
stubs shadow LinuxDriver's working implementations. Every collision has to be
|
||||
resolved deliberately — this is the class of bug that makes a driver look
|
||||
fine until it runs against hardware.
|
||||
"""
|
||||
|
||||
def test_get_services_returns_the_linux_os_service_list(self, driver):
|
||||
"""netOrk's poller expects a list here (OS services). StorageDriver's
|
||||
stub would return a dict of NAS services, if it returned anything."""
|
||||
from napalm_linux.linux import LinuxDriver
|
||||
|
||||
with patch.object(LinuxDriver, "get_services", return_value=[{"name": "sshd"}]) as m:
|
||||
result = driver.get_services()
|
||||
|
||||
assert m.called
|
||||
assert result == [{"name": "sshd"}]
|
||||
|
||||
@pytest.mark.xfail(strict=True, reason=_PENDING_HARVEST)
|
||||
def test_nas_services_live_under_a_separate_name(self):
|
||||
"""get_storage_services is what netOrk's _collect.py actually reads for
|
||||
the storage snapshot — get_services is the OS list."""
|
||||
assert hasattr(QnapQtsDriver, "get_storage_services")
|
||||
|
||||
@pytest.mark.xfail(strict=True, reason=_PENDING_HARVEST)
|
||||
def test_get_packages_is_not_the_storage_stub(self):
|
||||
from napalm_device_types import StorageDriver
|
||||
|
||||
assert QnapQtsDriver.get_packages is not StorageDriver.get_packages
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("method", "args"),
|
||||
[
|
||||
("install_package", ("qpkg-name",)),
|
||||
("remove_package", ("qpkg-name",)),
|
||||
("snapshot_create", ("DataVol1", "snap1")),
|
||||
],
|
||||
)
|
||||
def test_out_of_scope_writers_still_raise(self, driver, method, args):
|
||||
"""v1 is read-only plus safe actions. These must fail loudly rather than
|
||||
appear supported — netOrk's poller catches NotImplementedError."""
|
||||
with pytest.raises(NotImplementedError):
|
||||
getattr(driver, method)(*args)
|
||||
|
||||
|
||||
class TestQtsVersionDetection:
|
||||
def test_parses_major_version(self, driver):
|
||||
with patch.object(driver, "_send", return_value="5.1.5"):
|
||||
assert driver._detect_qts_major() == 5
|
||||
|
||||
def test_parses_qts_four(self, driver):
|
||||
with patch.object(driver, "_send", return_value="4.5.4"):
|
||||
assert driver._detect_qts_major() == 4
|
||||
|
||||
def test_tolerates_a_build_suffix(self, driver):
|
||||
with patch.object(driver, "_send", return_value="5.2.0.2782 (2026/03/14)"):
|
||||
assert driver._detect_qts_major() == 5
|
||||
|
||||
def test_unreadable_version_does_not_raise(self, driver):
|
||||
"""An unknown version must degrade to the newer code path, not abort the
|
||||
poll — a NAS answering nothing useful here is still worth polling."""
|
||||
with patch.object(driver, "_send", return_value=""):
|
||||
assert driver._detect_qts_major() is None
|
||||
|
||||
|
||||
class TestDockerBinDiscovery:
|
||||
"""Container Station does not put docker on PATH."""
|
||||
|
||||
def test_uses_the_container_station_path_when_present(self, driver):
|
||||
found = "/share/CACHEDEV1_DATA/.qpkg/container-station/bin/docker"
|
||||
with patch.object(driver, "_send", return_value=found):
|
||||
assert driver._discover_docker_path() == found
|
||||
|
||||
def test_falls_back_to_plain_docker_when_nothing_is_found(self, driver):
|
||||
with patch.object(driver, "_send", return_value=""):
|
||||
assert driver._discover_docker_path() == "docker"
|
||||
|
||||
def test_docker_bin_hook_returns_the_discovered_path(self, driver):
|
||||
driver._docker_path = "/opt/docker"
|
||||
assert driver._docker_bin() == "/opt/docker"
|
||||
@@ -0,0 +1,84 @@
|
||||
"""The harvest sanitiser (tools/sanitize.py).
|
||||
|
||||
Fixtures are committed forever, so a miss here puts a real NAS's serial numbers
|
||||
and addresses into git history. Worth testing even though it is a dev tool.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib.util
|
||||
import pathlib
|
||||
|
||||
import pytest
|
||||
|
||||
_SPEC = importlib.util.spec_from_file_location(
|
||||
"qnap_sanitize", pathlib.Path(__file__).parent.parent / "tools" / "sanitize.py"
|
||||
)
|
||||
assert _SPEC and _SPEC.loader
|
||||
sanitize = importlib.util.module_from_spec(_SPEC)
|
||||
_SPEC.loader.exec_module(sanitize)
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def scrubber():
|
||||
return sanitize.Scrubber()
|
||||
|
||||
|
||||
class TestSerials:
|
||||
def test_labelled_serial_is_replaced(self, scrubber):
|
||||
out = scrubber.scrub("Serial Number: WD-WCC4N7RTKZ9P")
|
||||
assert "WD-WCC4N7RTKZ9P" not in out
|
||||
assert "SERIAL001XXXX" in out
|
||||
|
||||
def test_lsblk_serial_field_is_replaced(self, scrubber):
|
||||
out = scrubber.scrub('NAME="sda" MODEL="WD40EFRX" SERIAL="WD-WCC4N7RTKZ9P"')
|
||||
assert "WD-WCC4N7RTKZ9P" not in out
|
||||
assert 'MODEL="WD40EFRX"' in out, "model names are not identifying and must survive"
|
||||
|
||||
def test_same_serial_maps_to_the_same_placeholder(self, scrubber):
|
||||
"""A disk serial appears in several harvested files; the cross-reference
|
||||
has to survive or the fixtures stop describing one coherent device."""
|
||||
first = scrubber.scrub("serial: ABC123456")
|
||||
second = scrubber.scrub('SERIAL="ABC123456"')
|
||||
assert "SERIAL001XXXX" in first
|
||||
assert "SERIAL001XXXX" in second
|
||||
|
||||
def test_distinct_serials_get_distinct_placeholders(self, scrubber):
|
||||
out = scrubber.scrub("sn: AAA111222\nsn: BBB333444")
|
||||
assert "SERIAL001XXXX" in out
|
||||
assert "SERIAL002XXXX" in out
|
||||
|
||||
|
||||
class TestNetworkIdentifiers:
|
||||
def test_mac_is_replaced(self, scrubber):
|
||||
out = scrubber.scrub("link/ether 24:5e:be:11:22:33 brd ff:ff:ff:ff:ff:ff")
|
||||
assert "24:5e:be:11:22:33" not in out
|
||||
assert "00:11:22:33:44:" in out
|
||||
|
||||
def test_ipv4_is_replaced(self, scrubber):
|
||||
out = scrubber.scrub("inet 10.7.224.12/24")
|
||||
assert "10.7.224.12" not in out
|
||||
assert "192.0.2.1" in out
|
||||
|
||||
def test_loopback_and_wildcard_survive(self, scrubber):
|
||||
"""Replacing these makes fixtures unreadable and reveals nothing."""
|
||||
out = scrubber.scrub("0.0.0.0:445 127.0.0.1:22")
|
||||
assert "0.0.0.0" in out
|
||||
assert "127.0.0.1" in out
|
||||
|
||||
def test_same_ip_maps_consistently(self, scrubber):
|
||||
out = scrubber.scrub("gw 10.0.0.1\nvia 10.0.0.1")
|
||||
assert out.count("192.0.2.1") == 2
|
||||
|
||||
|
||||
class TestHostnames:
|
||||
def test_extra_host_is_replaced_case_insensitively(self):
|
||||
s = sanitize.Scrubber(["MyNAS"])
|
||||
out = s.scrub("Server Name = mynas\nhost MYNAS ok")
|
||||
assert "mynas" not in out.lower().replace("testnas", "")
|
||||
assert out.count("testnas") == 2
|
||||
|
||||
def test_empty_host_entry_is_ignored(self):
|
||||
"""An unset --extra-host must not turn into a regex that matches everything."""
|
||||
s = sanitize.Scrubber(["", None])
|
||||
assert s.scrub("untouched text") == "untouched text"
|
||||
Executable
+105
@@ -0,0 +1,105 @@
|
||||
#!/usr/bin/env bash
|
||||
# Collect the command outputs this driver parses, from a real QNAP NAS.
|
||||
#
|
||||
# Run once against a QTS 4 box and once against a QTS 5 box; the resulting
|
||||
# files become the test fixtures. Writing parsers against guessed output is how
|
||||
# a driver ends up passing its own tests and failing on hardware, so this runs
|
||||
# before any parser code.
|
||||
#
|
||||
# ./tools/harvest.sh admin@nas1.example.lan qts4
|
||||
# ./tools/harvest.sh admin@nas2.example.lan qts5
|
||||
#
|
||||
# Output lands in tools/harvest-out/<label>/ (gitignored). Sanitise with
|
||||
# ./tools/sanitize.py before pasting anything into a test file.
|
||||
set -uo pipefail
|
||||
|
||||
TARGET="${1:?usage: harvest.sh <user@host> <label> [ssh-port]}"
|
||||
LABEL="${2:?usage: harvest.sh <user@host> <label> [ssh-port]}"
|
||||
PORT="${3:-22}"
|
||||
OUT="$(dirname "$0")/harvest-out/$LABEL"
|
||||
mkdir -p "$OUT"
|
||||
|
||||
# One SSH session for the whole run: QTS is slow to authenticate, and opening a
|
||||
# connection per command turns a 30-second harvest into several minutes.
|
||||
CTL="$(mktemp -u /tmp/qnap-harvest-XXXXXX)"
|
||||
cleanup() { ssh -O exit -o ControlPath="$CTL" "$TARGET" 2>/dev/null || true; }
|
||||
trap cleanup EXIT
|
||||
|
||||
ssh -M -o ControlPath="$CTL" -o ControlPersist=120 -p "$PORT" -N "$TARGET" &
|
||||
sleep 3
|
||||
|
||||
run() {
|
||||
local name="$1"; shift
|
||||
local cmd="$*"
|
||||
printf ' %-28s %s\n' "$name" "$cmd"
|
||||
{
|
||||
echo "### command: $cmd"
|
||||
ssh -o ControlPath="$CTL" -p "$PORT" "$TARGET" "$cmd" 2>&1
|
||||
echo "### exit: $?"
|
||||
} > "$OUT/$name.txt"
|
||||
}
|
||||
|
||||
echo "Harvesting $TARGET -> $OUT"
|
||||
|
||||
# ── Identity / facts ──────────────────────────────────────────────────────────
|
||||
run version 'getcfg System Version'
|
||||
run build 'getcfg System "Build Number"'
|
||||
run model 'getcfg System Model'
|
||||
run servername 'getcfg System "Server Name"'
|
||||
run getsysinfo_model 'getsysinfo model'
|
||||
run getsysinfo_serial 'getsysinfo serial'
|
||||
run uptime 'cat /proc/uptime'
|
||||
run uname 'uname -a'
|
||||
run ulinux_system 'sed -n "/^\[System\]/,/^\[/p" /etc/config/uLinux.conf'
|
||||
|
||||
# ── Interfaces ────────────────────────────────────────────────────────────────
|
||||
run ip_link 'ip -o link show'
|
||||
run ip_addr 'ip -o addr show'
|
||||
run ifconfig 'ifconfig -a'
|
||||
run ulinux_network 'sed -n "/^\[Network\]/,/^\[/p" /etc/config/uLinux.conf'
|
||||
|
||||
# ── Storage ───────────────────────────────────────────────────────────────────
|
||||
run qcli_storage 'qcli_storage'
|
||||
run qcli_storage_d 'qcli_storage -d'
|
||||
run qcli_storage_v 'qcli_storage -v'
|
||||
run qcli_storage_p 'qcli_storage -p'
|
||||
run qcli_storage_t 'qcli_storage -t'
|
||||
run mdstat 'cat /proc/mdstat'
|
||||
run volume_conf 'cat /etc/volume.conf'
|
||||
run enclosure_conf 'cat /etc/enclosure_0.conf'
|
||||
run df 'df -B1 -P'
|
||||
run hdnum 'getsysinfo hdnum'
|
||||
run hdtmp_all 'for i in $(seq 1 8); do echo "== slot $i"; getsysinfo hdtmp $i; done'
|
||||
run hdstatus_all 'for i in $(seq 1 8); do echo "== slot $i"; getsysinfo hdstatus $i; done'
|
||||
run smartinfo_1 'get_hd_smartinfo -d 1'
|
||||
run smartinfo_2 'get_hd_smartinfo -d 2'
|
||||
run lsblk 'lsblk -b -P -o NAME,MODEL,SERIAL,SIZE,ROTA,TYPE'
|
||||
|
||||
# ── Shares & services ─────────────────────────────────────────────────────────
|
||||
run smb_conf 'cat /etc/config/smb.conf'
|
||||
run exports 'cat /etc/exports'
|
||||
run ss_listen 'ss -lntup'
|
||||
run svc_samba 'getcfg Samba Enable'
|
||||
run svc_nfs 'getcfg NFS Enable'
|
||||
run svc_ftp 'getcfg FTP Enable'
|
||||
run svc_afp 'getcfg AFP Enable'
|
||||
run svc_iscsi 'getcfg iSCSI Enable'
|
||||
run init_d 'ls /etc/init.d/'
|
||||
|
||||
# ── Packages (QPKG) ───────────────────────────────────────────────────────────
|
||||
run qpkg_conf 'cat /etc/config/qpkg.conf'
|
||||
run qpkg_cli 'qpkg_cli -L'
|
||||
run qpkg_dir 'ls -d /share/*/.qpkg/*'
|
||||
|
||||
# ── Virtualisation & containers ───────────────────────────────────────────────
|
||||
run virsh_which 'ls /share/*/.qpkg/QKVM/usr/bin/virsh /share/*/.qpkg/*/bin/virsh 2>&1'
|
||||
run virsh_list 'virsh list --all'
|
||||
run docker_which 'ls /share/*/.qpkg/container-station/bin/docker /share/*/.qpkg/container-station/usr/bin/docker 2>&1'
|
||||
run docker_ps 'docker ps -a --no-trunc --format "{{json .}}"'
|
||||
|
||||
# ── Firmware / updates ────────────────────────────────────────────────────────
|
||||
run fw_update_cfg 'sed -n "/^\[FIRMWARE UPDATE\]/,/^\[/p" /etc/config/uLinux.conf'
|
||||
|
||||
echo
|
||||
echo "Done. $(ls "$OUT" | wc -l) files in $OUT"
|
||||
echo "Next: ./tools/sanitize.py $OUT"
|
||||
Executable
+127
@@ -0,0 +1,127 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Scrub identifying data out of harvested QNAP output before it becomes a fixture.
|
||||
|
||||
Fixtures live in a git repo forever. Serial numbers, MACs, public IPs and
|
||||
hostnames from a real NAS have no business in one, and a reviewer cannot be
|
||||
expected to spot every one of them by eye.
|
||||
|
||||
Replacements are stable within a run — the same serial always becomes the same
|
||||
placeholder — so cross-references between files (a disk serial appearing in both
|
||||
qcli_storage and get_hd_smartinfo) survive and the fixtures stay coherent.
|
||||
|
||||
./tools/sanitize.py tools/harvest-out/qts5
|
||||
./tools/sanitize.py tools/harvest-out/qts5 --extra-host mynas --in-place
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import pathlib
|
||||
import re
|
||||
import sys
|
||||
|
||||
# QNAP serials are alphanumeric runs; matching them generically would eat model
|
||||
# names, so they are found via their labelled context instead.
|
||||
SERIAL_CONTEXT = re.compile(
|
||||
r"(?i)\b(serial(?:\s*number)?|serial_no|sn)\b(\s*[:=]\s*|\s+)([A-Z0-9][A-Z0-9\-]{5,})"
|
||||
)
|
||||
DISK_SERIAL_FIELD = re.compile(r'(?i)\bSERIAL="([^"]*)"')
|
||||
|
||||
# One alternation rather than three passes. A MAC is also a valid match for the
|
||||
# IPv6 pattern, so separate passes let the second one rewrite what the first
|
||||
# just produced — and the placeholder MAC came back out as an IPv6 address.
|
||||
# Alternation order is the precedence: most specific first.
|
||||
ADDRESS = re.compile(
|
||||
r"(?P<mac>\b(?:[0-9A-Fa-f]{2}:){5}[0-9A-Fa-f]{2}\b)"
|
||||
r"|(?P<ipv6>\b(?:[0-9A-Fa-f]{0,4}:){3,7}[0-9A-Fa-f]{0,4}\b)"
|
||||
r"|(?P<ipv4>\b(?:\d{1,3}\.){3}\d{1,3}\b)"
|
||||
)
|
||||
|
||||
# Addresses that carry no information about the owner and make fixtures harder
|
||||
# to read if replaced.
|
||||
KEEP_IPS = {"0.0.0.0", "127.0.0.1", "255.255.255.255"}
|
||||
|
||||
|
||||
class Scrubber:
|
||||
def __init__(self, extra_hosts: list[str] | None = None) -> None:
|
||||
self._maps: dict[str, dict[str, str]] = {}
|
||||
self._extra_hosts = [h for h in (extra_hosts or []) if h]
|
||||
|
||||
def _placeholder(self, kind: str, value: str, template: str) -> str:
|
||||
bucket = self._maps.setdefault(kind, {})
|
||||
if value not in bucket:
|
||||
bucket[value] = template.format(n=len(bucket) + 1)
|
||||
return bucket[value]
|
||||
|
||||
def _serial(self, value: str) -> str:
|
||||
return self._placeholder("serial", value, "SERIAL{n:03d}XXXX")
|
||||
|
||||
def _mac(self, value: str) -> str:
|
||||
n = self._placeholder("mac", value.lower(), "{n}")
|
||||
return f"00:11:22:33:44:{int(n):02x}"
|
||||
|
||||
def _ip(self, value: str) -> str:
|
||||
return self._placeholder("ip", value, "192.0.2.{n}")
|
||||
|
||||
def _address(self, match: re.Match) -> str:
|
||||
if match.group("mac"):
|
||||
return self._mac(match.group("mac"))
|
||||
if match.group("ipv6"):
|
||||
return "2001:db8::1"
|
||||
value = match.group("ipv4")
|
||||
return value if value in KEEP_IPS else self._ip(value)
|
||||
|
||||
def scrub(self, text: str) -> str:
|
||||
text = SERIAL_CONTEXT.sub(
|
||||
lambda m: f"{m.group(1)}{m.group(2)}{self._serial(m.group(3))}", text
|
||||
)
|
||||
text = DISK_SERIAL_FIELD.sub(lambda m: f'SERIAL="{self._serial(m.group(1))}"', text)
|
||||
text = ADDRESS.sub(self._address, text)
|
||||
for host in self._extra_hosts:
|
||||
text = re.sub(rf"(?i)\b{re.escape(host)}\b", "testnas", text)
|
||||
return text
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
ap = argparse.ArgumentParser(description=__doc__)
|
||||
ap.add_argument("directory", type=pathlib.Path)
|
||||
ap.add_argument(
|
||||
"--extra-host",
|
||||
action="append",
|
||||
default=[],
|
||||
help="hostname or share name to replace with 'testnas' (repeatable)",
|
||||
)
|
||||
ap.add_argument(
|
||||
"--in-place",
|
||||
action="store_true",
|
||||
help="overwrite the harvested files instead of writing a .sanitised sibling dir",
|
||||
)
|
||||
args = ap.parse_args(argv)
|
||||
|
||||
if not args.directory.is_dir():
|
||||
print(f"not a directory: {args.directory}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
scrubber = Scrubber(args.extra_host)
|
||||
dest = (
|
||||
args.directory
|
||||
if args.in_place
|
||||
else args.directory.with_name(args.directory.name + ".sanitised")
|
||||
)
|
||||
dest.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
count = 0
|
||||
for path in sorted(args.directory.glob("*.txt")):
|
||||
cleaned = scrubber.scrub(path.read_text(errors="replace"))
|
||||
(dest / path.name).write_text(cleaned)
|
||||
count += 1
|
||||
|
||||
replaced = {k: len(v) for k, v in scrubber._maps.items()}
|
||||
print(f"sanitised {count} file(s) -> {dest}")
|
||||
print(f"replaced: {replaced or 'nothing'}")
|
||||
print("Read the output before committing — this catches patterns, not judgement.")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
Reference in New Issue
Block a user