# Veza V5 Ultra Deployment Guide

This guide provides step-by-step instructions for deploying Veza V5 Ultra using Ansible, Incus containers, OVN networking, HAProxy, and Let's Encrypt.

## Table of Contents

- [Prerequisites](#prerequisites)
- [Quick Start](#quick-start)
- [Step-by-Step Deployment](#step-by-step-deployment)
- [Troubleshooting](#troubleshooting)
- [Post-Deployment](#post-deployment)
- [Maintenance](#maintenance)

## Prerequisites

### Control Node (Your Machine)
- Ansible 2.16+
- SSH access to target host
- Required collections: `community.general`, `community.docker`

### Target Host (192.168.0.12)
- Debian 12 (Bookworm)
- SSH key authentication configured
- Root or sudo access
- Internet connectivity

### DNS Configuration
- Domain: `veza.talas.fr`
- A record pointing to target host IP (192.168.0.12)

## Quick Start

```bash
# 1. Clone and navigate to ansible directory
cd ansible

# 2. Install required collections
ansible-galaxy collection install community.general community.docker

# 3. Run full deployment
./deploy-veza.sh

# 4. Configure DNS and re-run HAProxy playbook
ansible-playbook -i inventory/prod/hosts.yml playbooks/30-haproxy-acme.yml -e domain=veza.talas.fr -e acme_email=ops@talas.fr

# 5. Run smoke tests
ansible-playbook -i inventory/prod/hosts.yml playbooks/50-smoke-tests.yml
```

## Step-by-Step Deployment

### Step 1: Bootstrap Target Host

```bash
ansible-playbook -i inventory/prod/hosts.yml playbooks/00-bootstrap-remote.yml
```

**What this does:**
- Installs essential packages (python3, sudo, curl, etc.)
- Configures SSH for better performance
- Sets up firewall rules for required ports
- Installs Incus dependencies

**Expected output:**
```
TASK [Install essential packages] **********************************************
ok: [edge-1]

TASK [Configure firewall for Veza ports] **************************************
ok: [edge-1]

TASK [Test connectivity] ******************************************************
ok: [edge-1]
```

### Step 2: Install Incus and OVN

```bash
ansible-playbook -i inventory/prod/hosts.yml playbooks/10-incus-ovn.yml
```

**What this does:**
- Installs Incus via snap
- Initializes Incus in standalone mode
- Creates OVN network `veza-ovn`
- Creates `veza` profile for containers

**Expected output:**
```
TASK [Install Incus via snap] *************************************************
ok: [edge-1]

TASK [Create OVN network for Veza] ********************************************
ok: [edge-1]

TASK [Verify Incus is running] ************************************************
ok: [edge-1]
```

### Step 3: Create Containers

```bash
ansible-playbook -i inventory/prod/hosts.yml playbooks/20-incus-containers.yml
```

**What this does:**
- Creates 5 containers: haproxy, backend, chat, stream, web
- Configures networking with static IPs
- Sets up proxy devices for external access
- Starts all containers

**Expected output:**
```
TASK [Create Veza containers] *************************************************
ok: [edge-1] => (item=veza-haproxy)
ok: [edge-1] => (item=veza-backend)
ok: [edge-1] => (item=veza-chat)
ok: [edge-1] => (item=veza-stream)
ok: [edge-1] => (item=veza-web)
```

### Step 4: Configure HAProxy and Let's Encrypt

```bash
ansible-playbook -i inventory/prod/hosts.yml playbooks/30-haproxy-acme.yml -e domain=veza.talas.fr -e acme_email=ops@talas.fr
```

**What this does:**
- Installs HAProxy and ACME tools in container
- Configures nginx for ACME challenges
- Sets up HAProxy with SSL termination
- Requests Let's Encrypt certificate
- Configures automatic renewal

**Expected output:**
```
TASK [Install HAProxy and ACME tools in container] ****************************
ok: [edge-1]

TASK [Request Let's Encrypt certificate] ***************************************
ok: [edge-1]

TASK [Test HAProxy configuration] **********************************************
ok: [edge-1]
```

### Step 5: Deploy Applications

```bash
ansible-playbook -i inventory/prod/hosts.yml playbooks/40-veza-apps.yml
```

**What this does:**
- Installs Go and builds backend API
- Installs Rust and builds chat server
- Installs Rust and builds stream server
- Installs Node.js and deploys web app
- Creates systemd services for all apps

**Expected output:**
```
TASK [Deploy Go Backend API] **************************************************
ok: [edge-1]

TASK [Deploy Rust Chat Server] ***********************************************
ok: [edge-1]

TASK [Deploy Rust Stream Server] **********************************************
ok: [edge-1]

TASK [Deploy React Web Application] *******************************************
ok: [edge-1]
```

### Step 6: Run Smoke Tests

```bash
ansible-playbook -i inventory/prod/hosts.yml playbooks/50-smoke-tests.yml
```

**What this does:**
- Tests all container connectivity
- Validates all service endpoints
- Checks HAProxy configuration
- Tests external access (if DNS configured)
- Generates comprehensive test report

**Expected output:**
```
TASK [Test container connectivity] *********************************************
ok: [edge-1]

TASK [Test Backend API service] ***********************************************
ok: [edge-1]

TASK [Generate smoke test summary] ********************************************
ok: [edge-1]
```

## Troubleshooting

### Common Issues

#### 1. SSH Connection Failed
```bash
# Test SSH connectivity
ssh -o ConnectTimeout=10 senke@192.168.0.12 "echo 'SSH test'"

# Check SSH config
grep -n "compressionlevel" ~/.ssh/config
```

**Solution:** Fix SSH config or ensure target host is reachable.

#### 2. Incus Installation Failed
```bash
# Check snapd status
incus exec veza-haproxy -- systemctl status snapd

# Reinstall Incus
incus exec veza-haproxy -- snap remove incus
incus exec veza-haproxy -- snap install incus --classic
```

#### 3. Container Creation Failed
```bash
# Check Incus status
incus list
incus network list
incus profile list

# Clean up and retry
incus delete veza-haproxy --force
ansible-playbook -i inventory/prod/hosts.yml playbooks/20-incus-containers.yml
```

#### 4. HAProxy Configuration Error
```bash
# Test HAProxy config
incus exec veza-haproxy -- haproxy -c -f /etc/haproxy/haproxy.cfg

# Check HAProxy logs
incus exec veza-haproxy -- journalctl -u haproxy -f
```

#### 5. Let's Encrypt Certificate Failed
```bash
# Check ACME challenges
incus exec veza-haproxy -- curl http://localhost:8888/.well-known/acme-challenge/test

# Manual certificate request
incus exec veza-haproxy -- dehydrated -c -d veza.talas.fr
```

#### 6. Application Service Failed
```bash
# Check service status
incus exec veza-backend -- systemctl status veza-backend
incus exec veza-chat -- systemctl status veza-chat
incus exec veza-stream -- systemctl status veza-stream
incus exec veza-web -- systemctl status veza-web

# Check logs
incus exec veza-backend -- journalctl -u veza-backend -f
```

### Debug Commands

```bash
# Check all container status
incus list --format=json | jq '.[] | {name: .name, status: .status, state: .state}'

# Check network configuration
incus network show veza-ovn

# Check HAProxy statistics
incus exec veza-haproxy -- curl -s http://localhost:8404/stats

# Test internal connectivity
incus exec veza-web -- curl -s http://10.10.0.101:8080/api/health
incus exec veza-web -- curl -s http://10.10.0.102:8081/health
incus exec veza-web -- curl -s http://10.10.0.103:8082/stream/health
```

## Post-Deployment

### 1. Configure DNS
Point your domain's A record to the target host IP:
```
veza.talas.fr.    IN    A    192.168.0.12
```

### 2. Re-run HAProxy Playbook
After DNS is configured, re-run the HAProxy playbook to get the Let's Encrypt certificate:
```bash
ansible-playbook -i inventory/prod/hosts.yml playbooks/30-haproxy-acme.yml -e domain=veza.talas.fr -e acme_email=ops@talas.fr
```

### 3. Verify HTTPS Access
```bash
curl -I https://veza.talas.fr
curl -I https://veza.talas.fr/api/health
```

### 4. Monitor Application Logs
```bash
# Follow all logs
incus exec veza-haproxy -- journalctl -u haproxy -f &
incus exec veza-backend -- journalctl -u veza-backend -f &
incus exec veza-chat -- journalctl -u veza-chat -f &
incus exec veza-stream -- journalctl -u veza-stream -f &
incus exec veza-web -- journalctl -u veza-web -f &
```

## Maintenance

### Certificate Renewal
Certificates are automatically renewed via cron. To check:
```bash
incus exec veza-haproxy -- crontab -l
incus exec veza-haproxy -- ls -la /etc/haproxy/certs/
```

### Container Updates
```bash
# Update container images
incus exec veza-backend -- apt update && apt upgrade -y
incus exec veza-chat -- apt update && apt upgrade -y
incus exec veza-stream -- apt update && apt upgrade -y
incus exec veza-web -- apt update && apt upgrade -y
```

### Backup
```bash
# Backup container configurations
incus export veza-haproxy /backup/veza-haproxy.tar.gz
incus export veza-backend /backup/veza-backend.tar.gz
incus export veza-chat /backup/veza-chat.tar.gz
incus export veza-stream /backup/veza-stream.tar.gz
incus export veza-web /backup/veza-web.tar.gz
```

### Scaling
To add more backend instances:
```bash
# Create additional backend container
incus launch debian/bookworm veza-backend-2 --profile veza
incus config device set veza-backend-2 eth0 ipv4.address=10.10.0.105/24
incus start veza-backend-2

# Update HAProxy configuration to include new backend
incus exec veza-haproxy -- sed -i 's/server api1 10.10.0.101:8080/server api1 10.10.0.101:8080\n    server api2 10.10.0.105:8080/' /etc/haproxy/haproxy.cfg
incus exec veza-haproxy -- systemctl reload haproxy
```

## Support

For issues or questions:
1. Check the troubleshooting section above
2. Review container logs for error messages
3. Run smoke tests to identify failing components
4. Check the Ansible playbook logs for deployment issues

## Architecture Overview

```
Internet (veza.talas.fr)
    ↓
HAProxy Container (80/443)
    ↓
OVN Network (veza-ovn)
    ↓
┌─────────┬─────────┬─────────┬─────────┐
│Backend  │ Chat    │ Stream  │ Web     │
│:8080    │ :8081   │ :8082   │ :3000   │
│(Go)     │ (Rust)  │ (Rust)  │ (Node)  │
└─────────┴─────────┴─────────┴─────────┘
```

This deployment provides a complete, production-ready Veza V5 Ultra platform with automatic SSL certificate management, load balancing, and comprehensive monitoring.