veza/veza-backend-api/docs/ARCHITECTURE.md

# Veza Backend API - Architecture Documentation

## Table of Contents

1. [Overview](#overview)
2. [System Architecture](#system-architecture)
3. [Application Architecture](#application-architecture)
4. [Code Structure](#code-structure)
5. [Design Patterns](#design-patterns)
6. [Data Flow](#data-flow)
7. [Service Integration](#service-integration)
8. [Security Architecture](#security-architecture)
9. [Performance & Scalability](#performance--scalability)
10. [Observability](#observability)
11. [Architectural Decisions](#architectural-decisions)

## Overview

The Veza Backend API is the core REST API service for the Veza platform, a collaborative audio streaming platform. It serves as the central service that orchestrates user authentication, content management, marketplace operations, and integrates with specialized services for real-time chat and audio streaming.

### Key Responsibilities

- **Authentication & Authorization**: JWT-based authentication, session management, RBAC
- **User Management**: User profiles, settings, profile completion
- **Content Management**: Audio tracks, playlists, metadata
- **Marketplace**: Products, orders, licenses, downloads
- **Social Features**: Likes, comments, sharing, collaboration
- **Analytics**: Event tracking, statistics, dashboards
- **Webhooks**: Asynchronous event delivery
- **Integration Hub**: Coordinates with Chat Server and Stream Server

### Technology Stack

- **Language**: Go 1.23+
- **Web Framework**: Gin
- **Database**: PostgreSQL (GORM + database/sql)
- **Cache**: Redis (optional)
- **Message Queue**: RabbitMQ (optional)
- **Logging**: Zap (structured logging)
- **Metrics**: Prometheus
- **Error Tracking**: Sentry (optional)

## System Architecture

### High-Level Architecture

```
┌─────────────────────────────────────────────────────────────┐
│                        Client Layer                         │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐                  │
│  │   Web    │  │  Mobile  │  │ Desktop │                  │
│  │  (React) │  │   App    │  │   App   │                  │
│  └────┬─────┘  └────┬─────┘  └────┬─────┘                  │
└───────┼─────────────┼─────────────┼────────────────────────┘
        │             │             │
        └─────────────┴─────────────┘
                      │
        ┌─────────────▼─────────────┐
        │   Load Balancer / WAF    │
        └─────────────┬─────────────┘
                      │
        ┌─────────────▼─────────────┐
        │    Backend API (Go)       │
        │    Port: 8080             │
        └─────┬───────────┬─────────┘
              │           │
    ┌─────────┘           └─────────┐
    │                               │
┌───▼────┐                    ┌───▼────┐
│  Chat  │                    │ Stream │
│ Server │                    │ Server │
│ (Rust) │                    │ (Rust) │
│ :8081  │                    │ :8082  │
└────────┘                    └────────┘
    │                               │
    └───────────┬───────────────────┘
                │
    ┌───────────▼───────────┐
    │   Data & Services     │
    │  ┌─────────┐         │
    │  │PostgreSQL│         │
    │  └─────────┘         │
    │  ┌─────────┐         │
    │  │  Redis  │         │
    │  └─────────┘         │
    │  ┌─────────┐         │
    │  │RabbitMQ │         │
    │  └─────────┘         │
    │  ┌─────────┐         │
    │  │   S3    │         │
    │  └─────────┘         │
    └──────────────────────┘
```

### Service Boundaries

The Veza platform consists of multiple services:

1. **Backend API (Go)**: REST API, business logic, data management
2. **Chat Server (Rust)**: WebSocket-based real-time chat
3. **Stream Server (Rust)**: Audio streaming (HLS/WebRTC)
4. **Frontend (React)**: Web application

### Communication Patterns

- **REST API**: Primary communication between frontend and backend
- **JWT Tokens**: Authentication and authorization
- **WebSocket**: Real-time chat (delegated to Chat Server)
- **HTTP Callbacks**: Integration with Stream Server
- **Message Queue**: Asynchronous job processing (RabbitMQ)

## Application Architecture

### Layered Architecture

The application follows a layered architecture pattern:

```
┌─────────────────────────────────────────┐
│         HTTP Layer (Gin)                 │
│  Handlers, Middleware, Routes            │
└──────────────┬──────────────────────────┘
               │
┌──────────────▼──────────────────────────┐
│         Core Layer                      │
│  Business Logic, Domain Models          │
│  - auth/                                │
│  - track/                               │
│  - marketplace/                         │
│  - social/                              │
└──────────────┬──────────────────────────┘
               │
┌──────────────▼──────────────────────────┐
│      Service Layer                      │
│  Application Services                   │
│  - UserService                          │
│  - TrackService                         │
│  - PlaylistService                      │
│  - AnalyticsService                     │
└──────────────┬──────────────────────────┘
               │
┌──────────────▼──────────────────────────┐
│      Repository Layer                   │
│  Data Access (GORM)                     │
│  - UserRepository                       │
│  - TrackRepository                      │
│  - PlaylistRepository                   │
└──────────────┬──────────────────────────┘
               │
┌──────────────▼──────────────────────────┐
│      Database Layer                     │
│  PostgreSQL                              │
└─────────────────────────────────────────┘
```

### Core Domains

The application is organized into core domains:

#### 1. Authentication Domain (`internal/core/auth/`)

- **Purpose**: User authentication and authorization
- **Components**:
  - `AuthService`: Authentication logic
  - `JWTService`: JWT token generation and validation
  - `SessionService`: Session management
  - `TwoFactorService`: 2FA implementation
- **Models**: `User`, `Session`, `RefreshToken`

#### 2. Track Domain (`internal/core/track/`)

- **Purpose**: Audio track management
- **Components**:
  - `TrackService`: Track business logic
  - `TrackHandler`: HTTP handlers
  - `TrackUploadService`: Upload processing
  - `TrackSearchService`: Search functionality
- **Models**: `Track`, `TrackVersion`, `TrackLike`

#### 3. Marketplace Domain (`internal/core/marketplace/`)

- **Purpose**: Marketplace operations
- **Components**:
  - `MarketplaceService`: Product and order management
  - `LicenseService`: License management
- **Models**: `Product`, `Order`, `OrderItem`, `License`

#### 4. Social Domain (`internal/core/social/`)

- **Purpose**: Social features
- **Components**: Comments, likes, sharing
- **Models**: `Comment`, `Like`, `Share`

## Code Structure

### Directory Organization

```
veza-backend-api/
├── cmd/
│   └── api/
│       └── main.go              # Application entry point
├── internal/
│   ├── api/                      # HTTP layer
│   │   ├── router.go            # Route configuration
│   │   └── middleware.go        # Middleware setup
│   ├── core/                     # Business logic (domains)
│   │   ├── auth/                # Authentication domain
│   │   ├── track/               # Track domain
│   │   ├── marketplace/         # Marketplace domain
│   │   └── social/              # Social domain
│   ├── handlers/                # HTTP handlers (legacy)
│   ├── services/                # Application services
│   ├── repositories/            # Data access layer
│   ├── models/                  # Data models (GORM)
│   ├── middleware/              # HTTP middleware
│   ├── config/                  # Configuration management
│   ├── database/                # Database connection & migrations
│   ├── workers/                 # Background job workers
│   ├── validators/              # Input validation
│   ├── errors/                  # Error handling
│   ├── metrics/                 # Prometheus metrics
│   └── logging/                 # Structured logging
├── migrations/                  # Database migrations
├── tests/                       # Integration tests
└── docs/                        # Documentation
```

### Package Organization Principles

1. **Domain-Driven**: Core business logic organized by domain
2. **Separation of Concerns**: Clear boundaries between layers
3. **Dependency Inversion**: High-level modules don't depend on low-level modules
4. **Single Responsibility**: Each package has a single, well-defined purpose

## Design Patterns

### 1. Repository Pattern

Data access is abstracted through repositories:

```go
type UserRepository interface {
    GetByID(ctx context.Context, id uuid.UUID) (*models.User, error)
    GetByEmail(ctx context.Context, email string) (*models.User, error)
    Create(ctx context.Context, user *models.User) error
    Update(ctx context.Context, user *models.User) error
}
```

### 2. Service Layer Pattern

Business logic is encapsulated in services:

```go
type UserService struct {
    repo UserRepository
    // ... other dependencies
}

func (s *UserService) CreateUser(ctx context.Context, req CreateUserRequest) (*User, error) {
    // Business logic here
}
```

### 3. Handler Pattern

HTTP handlers delegate to services:

```go
func (h *UserHandler) CreateUser(c *gin.Context) {
    var req CreateUserRequest
    if err := c.ShouldBindJSON(&req); err != nil {
        // Handle error
    }
    
    user, err := h.userService.CreateUser(c.Request.Context(), req)
    // Handle response
}
```

### 4. Middleware Pattern

Cross-cutting concerns handled via middleware:

- Authentication
- Authorization
- CORS
- Rate Limiting
- Request Timeout
- Error Recovery
- Metrics Collection
- Logging

### 5. Worker Pattern

Background jobs processed by workers:

```go
type JobWorker struct {
    queue   JobQueue
    workers []Worker
}

func (w *JobWorker) EnqueueAnalyticsJob(eventName string, userID *uuid.UUID, payload interface{}) {
    // Enqueue job
}
```

## Data Flow

### Request Flow

```
1. HTTP Request
   ↓
2. Middleware Chain
   - CORS
   - Rate Limiting
   - Authentication
   - Request Timeout
   ↓
3. Route Handler
   ↓
4. Service Layer
   - Business Logic
   - Validation
   ↓
5. Repository Layer
   - Data Access
   ↓
6. Database
   ↓
7. Response
```

### Authentication Flow

```
1. User submits credentials
   ↓
2. AuthService.ValidateCredentials()
   ↓
3. JWTService.GenerateToken()
   ↓
4. SessionService.CreateSession()
   ↓
5. Return JWT + Refresh Token
```

### Upload Flow

```
1. User uploads file
   ↓
2. UploadValidator.Validate() (ClamAV scan)
   ↓
3. TrackUploadService.ProcessUpload()
   ↓
4. TrackService.CreateTrack()
   ↓
5. Repository.Save()
   ↓
6. Background job: Process audio metadata
   ↓
7. Notify Stream Server
```

### Marketplace Purchase Flow

```
1. User creates order
   ↓
2. MarketplaceService.ValidateOrder()
   ↓
3. Payment processing (Stripe)
   ↓
4. OrderService.CreateOrder() (transaction)
   ↓
5. LicenseService.GenerateLicense()
   ↓
6. OrderService.CompleteOrder()
   ↓
7. Webhook: Order completed event
```

## Service Integration

### Chat Server Integration

The Backend API generates JWT tokens for the Chat Server:

```go
// Endpoint: POST /api/v1/chat/token
func (h *ChatHandler) GetToken(c *gin.Context) {
    userID := getUserIDFromContext(c)
    token := h.chatService.GenerateToken(userID)
    // Return token
}
```

**Flow**:
1. Frontend requests chat token from Backend API
2. Backend API generates short-lived JWT
3. Frontend uses token to connect to Chat Server
4. Chat Server validates token and establishes WebSocket connection

### Stream Server Integration

The Backend API notifies the Stream Server when tracks are ready:

```go
// Endpoint: POST /api/v1/internal/tracks/:id/stream-ready
func (h *StreamHandler) NotifyStreamReady(c *gin.Context) {
    trackID := c.Param("id")
    h.streamService.NotifyReady(trackID)
}
```

**Flow**:
1. Track upload completes
2. Backend API processes track
3. Backend API calls Stream Server to prepare streaming
4. Stream Server generates HLS segments
5. Stream Server notifies Backend API when ready
6. Backend API updates track status

### External Services

- **Stripe**: Payment processing for marketplace
- **S3/MinIO**: File storage for audio tracks
- **ClamAV**: Antivirus scanning for uploads
- **Sentry**: Error tracking and monitoring
- **Email Service**: Transactional emails

## Security Architecture

### Authentication

- **JWT Tokens**: HS256 algorithm
- **Token Expiration**: Configurable (default 24h)
- **Refresh Tokens**: Long-lived tokens for token renewal
- **Token Versioning**: Supports token revocation

### Authorization

- **RBAC**: Role-Based Access Control
- **Ownership Checks**: Resource ownership validation
- **Permission Middleware**: Route-level permission checks

### Security Layers

1. **Network Layer**: HTTPS/TLS
2. **Application Layer**: 
   - CORS protection
   - CSRF tokens
   - Rate limiting
   - Input validation
   - SQL injection prevention (GORM)
3. **Data Layer**:
   - Password hashing (bcrypt)
   - Encrypted sensitive data
   - Secure session storage

### Security Features

- **2FA Support**: TOTP-based two-factor authentication
- **Password Policies**: Strong password requirements
- **Rate Limiting**: Prevents brute force attacks
- **ClamAV Integration**: Virus scanning for uploads
- **Audit Logging**: Security event tracking

## Performance & Scalability

### Database Optimization

- **Connection Pooling**: Configurable pool size
- **Query Optimization**: Indexed queries, prepared statements
- **Caching**: Redis for frequently accessed data
- **Read Replicas**: Support for read replicas (future)

### Caching Strategy

- **Redis Cache**: 
  - User sessions
  - Rate limiting counters
  - Frequently accessed data
  - API response caching

### Scalability Patterns

- **Horizontal Scaling**: Stateless API design
- **Load Balancing**: Multiple API instances
- **Database Sharding**: Future consideration
- **CDN Integration**: Static asset delivery

### Performance Optimizations

- **Async Processing**: Background jobs for heavy operations
- **Chunked Uploads**: Large file upload support
- **Pagination**: Efficient data retrieval
- **Lazy Loading**: On-demand data loading

## Observability

### Logging

- **Structured Logging**: JSON format (production)
- **Log Levels**: DEBUG, INFO, WARN, ERROR
- **Context Propagation**: Request IDs, user IDs
- **Log Aggregation**: Centralized log collection

### Metrics

- **Prometheus Metrics**: 
  - HTTP request duration
  - Request count by endpoint
  - Error rates
  - Database query duration
  - Cache hit/miss rates

### Tracing

- **Request Tracing**: Request ID propagation
- **Distributed Tracing**: Future integration with OpenTelemetry

### Health Checks

- **Liveness Probe**: `/healthz`
- **Readiness Probe**: `/readyz`
- **Dependency Checks**: Database, Redis, external services

## Architectural Decisions

### ADR-001: Go as Primary Language

**Decision**: Use Go 1.23+ for the backend API

**Rationale**:
- Strong typing and compile-time safety
- Excellent performance
- Great concurrency support
- Strong ecosystem for web services
- Easy deployment (single binary)

**Alternatives Considered**:
- Node.js: Runtime errors, callback hell
- Python: Performance concerns
- Rust: Steeper learning curve

### ADR-002: Gin Web Framework

**Decision**: Use Gin as the web framework

**Rationale**:
- High performance
- Simple API
- Good middleware support
- Active community
- Well-documented

**Alternatives Considered**:
- Echo: Similar features, smaller community
- Fiber: Less mature
- Standard library: Too low-level

### ADR-003: GORM for Database Access

**Decision**: Use GORM as the ORM

**Rationale**:
- Type-safe queries
- Migration support
- Relationship management
- Active Record pattern

**Alternatives Considered**:
- sqlx: More control, less convenience
- Ent: More complex
- Raw SQL: Too verbose

### ADR-004: Layered Architecture

**Decision**: Use layered architecture with domain separation

**Rationale**:
- Clear separation of concerns
- Easy to test
- Maintainable
- Scalable

**Alternatives Considered**:
- Microservices: Too complex for MVP
- Monolithic: Already using this approach

### ADR-005: JWT for Authentication

**Decision**: Use JWT tokens for authentication

**Rationale**:
- Stateless
- Scalable
- Industry standard
- Easy to implement

**Alternatives Considered**:
- Session-based: Requires session storage
- OAuth2: More complex for MVP

### ADR-006: PostgreSQL as Primary Database

**Decision**: Use PostgreSQL as the primary database

**Rationale**:
- ACID compliance
- Rich feature set
- Excellent performance
- Strong ecosystem

**Alternatives Considered**:
- MySQL: Less features
- MongoDB: NoSQL not suitable for relational data

### ADR-007: Redis for Caching

**Decision**: Use Redis for caching (optional)

**Rationale**:
- High performance
- Rich data structures
- Pub/Sub support
- Widely used

**Alternatives Considered**:
- Memcached: Less features
- In-memory cache: Not shared across instances

### ADR-008: RabbitMQ for Message Queue

**Decision**: Use RabbitMQ for asynchronous job processing (optional)

**Rationale**:
- Reliable message delivery
- Good performance
- Rich features
- Easy to set up

**Alternatives Considered**:
- Kafka: Too complex for MVP
- NATS: Less features
- In-process queue: Not suitable for distributed systems

## Future Considerations

### Planned Improvements

1. **Microservices Migration**: Split into domain-specific services
2. **Event Sourcing**: For audit trail and state reconstruction
3. **CQRS**: Separate read and write models
4. **GraphQL API**: Alternative to REST
5. **gRPC**: For inter-service communication
6. **Service Mesh**: For service discovery and load balancing

### Scalability Roadmap

1. **Phase 1**: Current monolithic architecture
2. **Phase 2**: Extract chat and streaming to separate services
3. **Phase 3**: Domain-based microservices
4. **Phase 4**: Full microservices with service mesh

## Additional Resources

- **API Documentation**: See `docs/API_DOCUMENTATION.md`
- **Development Guide**: See `docs/DEVELOPMENT_SETUP_GUIDE.md`
- **Deployment Guide**: See `docs/DEPLOYMENT_GUIDE.md`
- **Code Examples**: See `internal/core/` for domain implementations