NAPALM/napalm-proxmox
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(vm-provision): let Proxmox download cloud images into an import storage #4

Merged
christianmanivong merged 1 commits from feat/pve-import-download into master 2026-10-02 07:29:20 +00:00
Conversation 0 Commits 1 Files Changed 3
+410 -27
christianmanivong commented 2026-10-02 07:28:49 +00:00
Owner
Copy Link
Copy Source

When a node has an active storage with content type import (Proxmox 8.2+), create_vm_from_cloud_init now lets Proxmox fetch the cloud image itself, instead of downloading it over SSH into /var/lib/vz/template/netork-images on the node's root filesystem.

How it works:

  • download-url into that storage, with checksum / checksum-algorithm.
  • The image is attached as the root disk with scsi0=<storage>:0,import-from=<volid>, and the call waits for that task.

Why: on pve-dual (6.5 GB root) the SSH cache filled the root filesystem, after which Proxmox itself failed (/var/log/pve/tasks/…: No space left on device). See NetOrk/netork#480.

Details

  • Storage: the first enabled, active, node-visible storage whose content includes import. Inactive storages (an unmounted share) are skipped.
  • File name: netork-<sha256(url)[:12]>-<name>.<ext>, using only characters Proxmox keeps. An existing file is reused, because Proxmox only creates it after a successful, checksum-verified download.
  • Extension: Proxmox reads the format off the extension and has no .img. Ubuntu's qcow2 .img is therefore stored as .qcow2. If an .img were really raw, the import fails loudly; the opposite guess would silently import a qcow2 container as a raw disk.
  • Fallback: no import storage, or a type Proxmox cannot import (compressed, ISO, no extension), keeps the existing SSH path. That path is moved unchanged into _import_over_ssh.
  • Limitation: an API token without Datastore.AllocateTemplate on the storage and Sys.Audit/Sys.Modify on the node gets a 403 from download-url. There is no silent fallback to SSH in that case.

Tested

  • 15 new unit tests in tests/test_vm_import_download.py; the full suite has 259 passing.
  • Live on pve-dual (PVE 9.2.20) via netOrk feature-pve-import-storage:
    • Ubuntu 24.04 landed as software:import/netork-1f92ecd3b0ab-ubuntu-24.04-server-cloudimg-amd64.qcow2 (597 MB);
    • VM 104 (wazuh-01) booted from it on local-zfs10;
    • the old SSH cache directory stays unused.
When a node has an active storage with content type `import` (Proxmox 8.2+), `create_vm_from_cloud_init` now lets Proxmox fetch the cloud image itself, instead of downloading it over SSH into `/var/lib/vz/template/netork-images` on the node's root filesystem. How it works: - `download-url` into that storage, with `checksum` / `checksum-algorithm`. - The image is attached as the root disk with `scsi0=<storage>:0,import-from=<volid>`, and the call waits for that task. **Why:** on `pve-dual` (6.5 GB root) the SSH cache filled the root filesystem, after which Proxmox itself failed (`/var/log/pve/tasks/…: No space left on device`). See NetOrk/netork#480. **Details** - **Storage:** the first enabled, active, node-visible storage whose content includes `import`. Inactive storages (an unmounted share) are skipped. - **File name:** `netork-<sha256(url)[:12]>-<name>.<ext>`, using only characters Proxmox keeps. An existing file is reused, because Proxmox only creates it after a successful, checksum-verified download. - **Extension:** Proxmox reads the format off the extension and has no `.img`. Ubuntu's qcow2 `.img` is therefore stored as `.qcow2`. If an `.img` were really raw, the import fails loudly; the opposite guess would silently import a qcow2 container as a raw disk. - **Fallback:** no import storage, or a type Proxmox cannot import (compressed, ISO, no extension), keeps the existing SSH path. That path is moved unchanged into `_import_over_ssh`. - **Limitation:** an API token without `Datastore.AllocateTemplate` on the storage and `Sys.Audit`/`Sys.Modify` on the node gets a 403 from `download-url`. There is no silent fallback to SSH in that case. **Tested** - 15 new unit tests in `tests/test_vm_import_download.py`; the full suite has 259 passing. - Live on `pve-dual` (PVE 9.2.20) via netOrk `feature-pve-import-storage`: - Ubuntu 24.04 landed as `software:import/netork-1f92ecd3b0ab-ubuntu-24.04-server-cloudimg-amd64.qcow2` (597 MB); - VM 104 (`wazuh-01`) booted from it on `local-zfs10`; - the old SSH cache directory stays unused.
christianmanivong added 1 commit 2026-10-02 07:28:49 +00:00
feat(vm-provision): let Proxmox download cloud images into an import storage a13af149d9 
Provisioning downloaded every cloud image over SSH into
/var/lib/vz/template/netork-images, on the node's root filesystem. On a
small root that fills up and takes Proxmox down with it (netOrk #480).

When the node has an active storage with content type "import" (Proxmox
8.2+), Proxmox now does it itself: download-url with checksum
verification into that storage, then import-from as the root disk. The
file is named after a hash of the full URL and reused when present.
Proxmox takes the format from the extension and has no ".img", so
Ubuntu's qcow2 .img is stored as .qcow2 -- a wrong guess fails at import
instead of attaching a qcow2 container as a raw disk.

Without an import storage, or for an image type Proxmox cannot import,
the SSH download is used as before.
christianmanivong merged commit 6464a6c728 into master 2026-10-02 07:29:20 +00:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
christianmanivong
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: NAPALM/napalm-proxmox#4