ROOT /

History

Enable CI/CD by adding .onedev-buildspec.yml

api/v1alpha1	Loading last commit info...
cmd/server
internal
Dockerfile
LICENSE.md
README.md
go.mod
go.sum

README.md

Volume Manager (volman)

A Kubernetes controller for automated volume backup and restore operations using S3-compatible storage.

Overview

Volume Manager watches for deployments being scaled down or deleted, automatically backs up their persistent volumes to S3-compatible storage (MinIO, AWS S3, etc.), and restores data when deployments are recreated.

Features

CRD-based Configuration: Define backup destinations and targets using Kubernetes Custom Resources
Automatic Backup: When a deployment is scaled to zero or deleted, volumes are automatically backed up
Automatic Restore: When a pod is created matching a previous backup, an init-container is injected to restore data
Selector-based Backup Naming: Backups are named based on selector labels (e.g., username=alice → backup-alice.tar.gz)
KeyDB State Management: Backup states are stored in KeyDB for persistence
Vault Integration: Secrets are injected via Vault secrets injector

Custom Resource Definitions

Destination

Defines S3-compatible storage connection details:

apiVersion: s4lieri.volman/v1alpha1
kind: Destination
metadata:
  name: my-minio
spec:
  url: "https://minio.example.com"
  bucket: "volman-backups"
  region: "us-east-1"  # Optional, required for AWS S3
  accessKeyEnvVar: "S3_ACCESS_KEY"  # Env var name for access key
  secretKeyEnvVar: "S3_SECRET_KEY"  # Env var name for secret key
  forcePathStyle: true  # Required for MinIO
  useSSL: true
  skipSSLVerify: false  # Not recommended for production
  prefix: "backups"  # Optional path prefix

Target

Defines backup rules for a namespace:

apiVersion: s4lieri.volman/v1alpha1
kind: Target
metadata:
  name: vscode-workspaces
spec:
  namespace: "vscode"  # Namespace to monitor
  backupAnnotation: "volman.s4lieri/backup"  # Annotation marking volumes for backup
  selectorLabel: "username"  # Label used to differentiate backups
  backupRule: "backup-restore"  # Options: backup-only, backup-restore
  destinationRef: "my-minio"  # Reference to Destination CR
  mountPath: "/data"  # Volume mount path
  backupImage: "amazon/aws-cli:latest"  # Image for backup/restore jobs
  retentionDays: 30  # How long to keep backup states in KeyDB
  cleanupAfterBackup: true  # Delete PVC/PV after successful backup
  customPaths:  # Optional: specific paths to backup
    - path: "workspace"
      exclude:
        - "*.tmp"
        - "node_modules"

How It Works

Backup Flow

Deployment is scaled to zero or deleted
Controller detects the change
Finds PVCs used by the deployment
Checks if deployment/PVC has backup annotation
Creates a backup job that:
- Mounts the PVC
- Creates a tar.gz archive
- Uploads to S3 with name based on selector label
Saves backup state to KeyDB
Optionally deletes PVC/PV after successful backup

Restore Flow

New pod is created with backup annotation
Mutating webhook intercepts the pod creation
Checks KeyDB for existing backup matching selector label
If found, injects an init-container that:
- Downloads the backup from S3
- Extracts it to the volume mount path
Pod continues with restored data

Installation

Prerequisites

Kubernetes 1.24+
Helm 3.x
cert-manager (for webhook certificates)
Vault with secrets injector (for credentials)

Helm Installation

# Add the chart repository (if applicable)
helm repo add shardnet https://git.shardnet.tech/charts

# Install with default values
helm install shardnet ./charts/volume-manager -n volman-system --create-namespace

# Install with custom values
helm install shardnet ./charts/volume-manager -n volman-system --create-namespace \
  --set keydb.enabled=true \
  --set webhook.enabled=true

Configuration

See charts/volume-manager/values.yaml for all available options.

Key configurations:

# Enable/disable components
keydb:
  enabled: true

webhook:
  enabled: true
  failurePolicy: Ignore  # Don't block pod creation if webhook fails

# Vault annotations for secrets injection
podAnnotations:
  vault.hashicorp.com/agent-inject: "true"
  vault.hashicorp.com/role: "volume-manager"
  vault.hashicorp.com/agent-inject-secret-s3: "secret/data/volman/s3"

Usage

1. Create a Destination

kubectl apply -f - <<EOF
apiVersion: s4lieri.volman/v1alpha1
kind: Destination
metadata:
  name: production-minio
spec:
  url: "https://minio.prod.example.com"
  bucket: "volume-backups"
  accessKeyEnvVar: "S3_ACCESS_KEY"
  secretKeyEnvVar: "S3_SECRET_KEY"
  forcePathStyle: true
  useSSL: true
EOF

2. Create a Target

kubectl apply -f - <<EOF
apiVersion: s4lieri.volman/v1alpha1
kind: Target
metadata:
  name: dev-workspaces
spec:
  namespace: "development"
  backupAnnotation: "volman.s4lieri/backup"
  selectorLabel: "user-id"
  backupRule: "backup-restore"
  destinationRef: "production-minio"
  mountPath: "/home/coder"
  retentionDays: 30
  cleanupAfterBackup: true
EOF

3. Annotate Your Deployment

kubectl annotate deployment my-workspace volman.s4lieri/backup=true

4. Add Selector Labels

kubectl label deployment my-workspace user-id=user123

Now when the deployment is scaled to zero, the volume will be backed up as backup-user123.tar.gz. When a new deployment with user-id=user123 is created, the data will be automatically restored.

Development

Building

# Build the binary
go build -o volume-manager ./cmd/server

# Build Docker image
docker build -t volume-manager:latest .

Running Locally

# Run with kubeconfig
go run ./cmd/server \
  --metrics-bind-address=:8080 \
  --health-probe-bind-address=:8081 \
  --webhook-port=9443

Running Tests

go test ./...

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                     Volume Manager                               │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  ┌──────────────────┐  ┌──────────────────┐  ┌───────────────┐ │
│  │   Deployment     │  │     Target       │  │  Destination  │ │
│  │   Controller     │  │   Controller     │  │  Controller   │ │
│  └────────┬─────────┘  └────────┬─────────┘  └───────┬───────┘ │
│           │                     │                     │         │
│           ▼                     ▼                     ▼         │
│  ┌──────────────────────────────────────────────────────────┐  │
│  │                     Kubernetes API                        │  │
│  └──────────────────────────────────────────────────────────┘  │
│                                                                  │
│  ┌──────────────────┐  ┌──────────────────┐                    │
│  │  Pod Mutating    │  │     KeyDB        │                    │
│  │    Webhook       │  │  State Store     │                    │
│  └────────┬─────────┘  └────────┬─────────┘                    │
│           │                     │                               │
└───────────┼─────────────────────┼───────────────────────────────┘
            │                     │
            ▼                     ▼
    ┌───────────────┐    ┌───────────────┐
    │  Pod Creation │    │   Backup      │
    │  (Restore)    │    │   States      │
    └───────────────┘    └───────────────┘
            │
            ▼
    ┌───────────────┐
    │   S3/MinIO    │
    │   Storage     │
    └───────────────┘

License

MIT