veza/docs/PRODUCTION_DEPLOYMENT.md

# Production Deployment Guide — Veza

## Overview

This guide documents how to securely deploy Veza to production with proper secrets management. **Never commit secrets to Git.**

---

## 🔐 Secrets Management

### Required Secrets

The following secrets **must** be injected at runtime:

| Secret | Description | Example Source |
|--------|-------------|----------------|
| `DATABASE_URL` | PostgreSQL connection string | K8s Secret, AWS RDS |
| `JWT_SECRET` | Token signing key (min 32 chars) | AWS Secrets Manager, Vault |
| `REDIS_PASSWORD` | Redis authentication | K8s Secret |
| `SENTRY_DSN` | Error tracking endpoint | Sentry dashboard |
| `SMTP_PASSWORD` | Email service password | AWS SES, SendGrid |

### Optional Secrets

| Secret | Description | Required If |
|--------|-------------|-------------|
| `RABBITMQ_URL` | Message queue connection | Using async events |
| `CLAMAV_*` | Antivirus configuration | File upload scanning enabled |

---

## 📦 Deployment Methods

### Method 1: Kubernetes Secrets

**1. Create secrets:**

```bash
kubectl create secret generic veza-backend-secrets \
  --from-literal=database-url="postgresql://user:pass@host:5432/veza" \
  --from-literal=jwt-secret="$(openssl rand -base64 32)" \
  --from-literal=redis-password="$(openssl rand -base64 16)" \
  --namespace=production
```

**2. Reference in deployment:**

```yaml
# k8s/backend-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: veza-backend
spec:
  template:
    spec:
      containers:
      - name: api
        image: veza/backend:latest
        env:
        - name: DATABASE_URL
          valueFrom:
            secretKeyRef:
              name: veza-backend-secrets
              key: database-url
        - name: JWT_SECRET
          valueFrom:
            secretKeyRef:
              name: veza-backend-secrets
              key: jwt-secret
```

---

### Method 2: AWS Secrets Manager

**1. Store secrets:**

```bash
aws secretsmanager create-secret \
  --name veza/production/database \
  --secret-string '{"url":"postgresql://..."}'

aws secretsmanager create-secret \
  --name veza/production/jwt \
  --secret-string '{"secret":"..."}'
```

**2. Inject at runtime (ECS task definition):**

```json
{
  "containerDefinitions": [{
    "name": "veza-backend",
    "secrets": [
      {
        "name": "DATABASE_URL",
        "valueFrom": "arn:aws:secretsmanager:region:account:secret:veza/production/database:url::"
      },
      {
        "name": "JWT_SECRET",
        "valueFrom": "arn:aws:secretsmanager:region:account:secret:veza/production/jwt:secret::"
      }
    ]
  }]
}
```

---

### Method 3: HashiCorp Vault

**1. Store secrets:**

```bash
vault kv put secret/veza/production \
  database_url="postgresql://..." \
  jwt_secret="..." \
  redis_password="..."
```

**2. Inject via Vault Agent:**

```hcl
# vault-agent-config.hcl
template {
  source      = "/etc/veza/.env.production.tpl"
  destination = "/etc/veza/.env.production"
}
```

**Template file:**

```bash
# /etc/veza/.env.production.tpl
DATABASE_URL={{ with secret "secret/veza/production" }}{{ .Data.data.database_url }}{{ end }}
JWT_SECRET={{ with secret "secret/veza/production" }}{{ .Data.data.jwt_secret }}{{ end }}
```

---

### Method 4: Docker Compose (Staging)

**For staging/testing only, not production:**

```yaml
# docker-compose.production.yml
services:
  backend:
    image: veza/backend:latest
    env_file:
      - .env.production.local  # NOT committed to Git
    environment:
      - DATABASE_URL=${DATABASE_URL}
      - JWT_SECRET=${JWT_SECRET}
```

**Create `.env.production.local`** (add to `.gitignore`):

```bash
DATABASE_URL=postgresql://user:pass@postgres:5432/veza
JWT_SECRET=$(openssl rand -base64 32)
REDIS_PASSWORD=$(openssl rand -base64 16)
```

---

## ✅ Security Checklist

### Before Deployment

- [ ] All secrets stored in secrets manager (not Git)
- [ ] `.env` files in `.gitignore`
- [ ] `COOKIE_SECURE=true` in production
- [ ] `COOKIE_SAME_SITE=strict` in production
- [ ] CORS origins explicitly listed (no wildcard)
- [ ] JWT secret minimum 32 characters
- [ ] Database credentials rotated
- [ ] HTTPS enabled for all endpoints
- [ ] Sentry DSN configured for error tracking

### After Deployment

- [ ] Health endpoint returns 200: `curl https://api.veza.com/api/v1/health`
- [ ] CORS headers present on OPTIONS requests
- [ ] Login/logout flow works end-to-end
- [ ] No secrets visible in logs
- [ ] Database migrations applied successfully
- [ ] Redis connection established
- [ ] Email sending works (if configured)

---

## 🔄 Secret Rotation

### JWT Secret Rotation

**Impact**: All active sessions invalidated

```bash
# 1. Generate new secret
NEW_SECRET=$(openssl rand -base64 32)

# 2. Update in secrets manager
kubectl patch secret veza-backend-secrets \
  -p "{\"data\":{\"jwt-secret\":\"$(echo -n $NEW_SECRET | base64)\"}}"

# 3. Rolling restart
kubectl rollout restart deployment/veza-backend

# 4. Monitor for errors
kubectl logs -f deployment/veza-backend
```

### Database Password Rotation

**Impact**: Requires coordinated update

```bash
# 1. Create new user with new password in PostgreSQL
# 2. Update DATABASE_URL in secrets manager
# 3. Rolling restart with zero downtime
# 4. Revoke old user after verification
```

---

## 🚨 Troubleshooting

### "Invalid JWT secret" errors

**Cause**: Secret not injected or too short

**Fix**:
```bash
# Verify secret is set
kubectl get secret veza-backend-secrets -o jsonpath='{.data.jwt-secret}' | base64 -d | wc -c
# Should be >= 32 characters
```

### "Database connection failed"

**Cause**: DATABASE_URL not set or incorrect

**Fix**:
```bash
# Test connection from pod
kubectl exec -it deployment/veza-backend -- psql $DATABASE_URL -c "SELECT 1"
```

### "CORS errors in production"

**Cause**: Frontend domain not in CORS_ALLOWED_ORIGINS

**Fix**:
```bash
# Update backend .env.production
CORS_ALLOWED_ORIGINS=https://app.veza.com,https://www.veza.com
```

---

## 📚 References

- [Kubernetes Secrets](https://kubernetes.io/docs/concepts/configuration/secret/)
- [AWS Secrets Manager](https://docs.aws.amazon.com/secretsmanager/)
- [HashiCorp Vault](https://www.vaultproject.io/docs)
- [12-Factor App Config](https://12factor.net/config)

---

**Last Updated**: 2026-01-29  
**Audit Reference**: AUDIT_TEMP_29_01_2026.md (P2.3)