Skip to content

NAS backup: compression, encryption, bandwidth throttle, integrity check#12898

Open
jmsperu wants to merge 1 commit intoapache:4.22from
jmsperu:fix/nasbackup-enhancements-combined
Open

NAS backup: compression, encryption, bandwidth throttle, integrity check#12898
jmsperu wants to merge 1 commit intoapache:4.22from
jmsperu:fix/nasbackup-enhancements-combined

Conversation

@jmsperu
Copy link

@jmsperu jmsperu commented Mar 26, 2026

Summary

Adds four optional, zone-scoped features to NAS backup operations on KVM, all disabled by default:

  • Compression (-c): Uses qcow2 internal compression (qemu-img convert -c) to reduce backup size
  • LUKS Encryption (-e): Encrypts backup files at rest using LUKS via qemu-img convert --object secret
  • Bandwidth Throttle (-b): Limits backup I/O — virsh blockjob --bandwidth for running VMs, qemu-img convert -r + ionice for stopped VMs
  • Integrity Check (--verify): Runs qemu-img check on each backup file after creation

Configuration Keys (Zone scope)

Setting Type Default Description
nas.backup.compression.enabled Boolean false Enable qcow2 compression for backup files
nas.backup.encryption.enabled Boolean false Enable LUKS encryption for backup files
nas.backup.encryption.passphrase String (Secure) "" Passphrase for LUKS encryption
nas.backup.bandwidth.limit.mbps Integer 0 Bandwidth limit in MiB/s (0 = unlimited)
nas.backup.integrity.check Boolean false Run qemu-img check after backup

Architecture

  1. NASBackupProvider reads zone-scoped ConfigKeys and populates a details map on TakeBackupCommand
  2. TakeBackupCommand carries the details map from management server to KVM agent
  3. LibvirtTakeBackupCommandWrapper extracts the details and translates them to nasbackup.sh CLI flags
  4. nasbackup.sh implements the actual compression, encryption, throttling, and verification logic

Files Changed

  • scripts/vm/hypervisor/kvm/nasbackup.sh — new -c, -b, -e, --verify flags with encrypt_backup() and verify_backup() functions
  • core/.../TakeBackupCommand.java — added details map (HashMap) with getter/setter/addDetail
  • plugins/backup/nas/.../NASBackupProvider.java — 5 new ConfigKeys, populate command details in takeBackup()
  • plugins/hypervisors/kvm/.../LibvirtTakeBackupCommandWrapper.java — extract details, build dynamic CLI args, temp passphrase file lifecycle

Notes

Test plan

  • Verify backup works with all four features disabled (default) — no behavioral change
  • Enable nas.backup.compression.enabled at zone scope, take backup, verify qcow2 files are compressed
  • Enable nas.backup.bandwidth.limit.mbps (e.g. 50), take backup of running VM, verify virsh blockjob bandwidth is applied
  • Enable nas.backup.bandwidth.limit.mbps, take backup of stopped VM, verify qemu-img -r rate limit is applied
  • Enable nas.backup.encryption.enabled with passphrase, take backup, verify files are LUKS encrypted (qemu-img info shows encryption)
  • Enable nas.backup.integrity.check, take backup, verify qemu-img check runs and passes
  • Test with multiple features enabled simultaneously (compression + integrity check)
  • Verify restore still works for backups created with compression/encryption
  • Test with RBD storage pools — verify bandwidth throttle applies correctly

… integrity check

Adds four optional features to NAS backup operations, configurable at
zone scope via CloudStack global settings:

- Compression (-c): qcow2 internal compression of backup files
  Config: nas.backup.compression.enabled (default: false)

- LUKS Encryption (-e): encrypt backup files at rest using qemu-img
  Config: nas.backup.encryption.enabled (default: false)
  Config: nas.backup.encryption.passphrase (Secure category)

- Bandwidth Throttle (-b): limit backup I/O bandwidth via virsh
  blockjob for running VMs or qemu-img -r for stopped VMs
  Config: nas.backup.bandwidth.limit.mbps (default: 0/unlimited)

- Integrity Check (--verify): qemu-img check after backup creation
  Config: nas.backup.integrity.check (default: false)

All features are disabled by default and fully backward compatible.
Settings are read from zone-scoped ConfigKeys in NASBackupProvider,
passed to the KVM agent via TakeBackupCommand details map, and
translated to nasbackup.sh CLI flags in LibvirtTakeBackupCommandWrapper.

Changes:
- nasbackup.sh: add -c, -b, -e, --verify flags with encrypt_backup()
  and verify_backup() helper functions
- TakeBackupCommand.java: add details map for passing config to agent
- NASBackupProvider.java: add 5 ConfigKeys, populate command details
- LibvirtTakeBackupCommandWrapper.java: extract details, build CLI args,
  handle passphrase temp file lifecycle

Combines and supersedes PRs apache#12844, apache#12846, apache#12848, apache#12845
@codecov
Copy link

codecov bot commented Mar 27, 2026

Codecov Report

❌ Patch coverage is 18.84058% with 56 lines in your changes missing coverage. Please review.
✅ Project coverage is 17.60%. Comparing base (c1af36f) to head (2c40fdd).

Files with missing lines Patch % Lines
...ource/wrapper/LibvirtTakeBackupCommandWrapper.java 0.00% 36 Missing ⚠️
...rg/apache/cloudstack/backup/NASBackupProvider.java 52.17% 7 Missing and 4 partials ⚠️
...rg/apache/cloudstack/backup/TakeBackupCommand.java 10.00% 9 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               4.22   #12898      +/-   ##
============================================
- Coverage     17.61%   17.60%   -0.01%     
+ Complexity    15676    15671       -5     
============================================
  Files          5917     5917              
  Lines        531537   531601      +64     
  Branches      64985    64997      +12     
============================================
- Hits          93610    93602       -8     
- Misses       427369   427439      +70     
- Partials      10558    10560       +2     
Flag Coverage Δ
uitests 3.70% <ø> (ø)
unittests 18.67% <18.84%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant