senke/veza
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
veza/veza-backend-api/docs/ARCHITECTURE.md

20 KiB
Raw Blame History

Veza Backend API - Architecture Documentation

Table of Contents

  1. Overview
  2. System Architecture
  3. Application Architecture
  4. Code Structure
  5. Design Patterns
  6. Data Flow
  7. Service Integration
  8. Security Architecture
  9. Performance & Scalability
  10. Observability
  11. Architectural Decisions

Overview

The Veza Backend API is the core REST API service for the Veza platform, a collaborative audio streaming platform. It serves as the central service that orchestrates user authentication, content management, marketplace operations, and integrates with specialized services for real-time chat and audio streaming.

Key Responsibilities

  • Authentication & Authorization: JWT-based authentication, session management, RBAC
  • User Management: User profiles, settings, profile completion
  • Content Management: Audio tracks, playlists, metadata
  • Marketplace: Products, orders, licenses, downloads
  • Social Features: Likes, comments, sharing, collaboration
  • Analytics: Event tracking, statistics, dashboards
  • Webhooks: Asynchronous event delivery
  • Integration Hub: Coordinates with Chat Server and Stream Server

Technology Stack

  • Language: Go 1.23+
  • Web Framework: Gin
  • Database: PostgreSQL (GORM + database/sql)
  • Cache: Redis (optional)
  • Message Queue: RabbitMQ (optional)
  • Logging: Zap (structured logging)
  • Metrics: Prometheus
  • Error Tracking: Sentry (optional)

System Architecture

High-Level Architecture

┌─────────────────────────────────────────────────────────────┐
│                        Client Layer                         │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐                  │
│  │   Web    │  │  Mobile  │  │ Desktop │                  │
│  │  (React) │  │   App    │  │   App   │                  │
│  └────┬─────┘  └────┬─────┘  └────┬─────┘                  │
└───────┼─────────────┼─────────────┼────────────────────────┘
        │             │             │
        └─────────────┴─────────────┘
                      │
        ┌─────────────▼─────────────┐
        │   Load Balancer / WAF    │
        └─────────────┬─────────────┘
                      │
        ┌─────────────▼─────────────┐
        │    Backend API (Go)       │
        │    Port: 8080             │
        └─────┬───────────┬─────────┘
              │           │
    ┌─────────┘           └─────────┐
    │                               │
┌───▼────┐                    ┌───▼────┐
│  Chat  │                    │ Stream │
│ Server │                    │ Server │
│ (Rust) │                    │ (Rust) │
│ :8081  │                    │ :8082  │
└────────┘                    └────────┘
    │                               │
    └───────────┬───────────────────┘
                │
    ┌───────────▼───────────┐
    │   Data & Services     │
    │  ┌─────────┐         │
    │  │PostgreSQL│         │
    │  └─────────┘         │
    │  ┌─────────┐         │
    │  │  Redis  │         │
    │  └─────────┘         │
    │  ┌─────────┐         │
    │  │RabbitMQ │         │
    │  └─────────┘         │
    │  ┌─────────┐         │
    │  │   S3    │         │
    │  └─────────┘         │
    └──────────────────────┘

Service Boundaries

The Veza platform consists of multiple services:

  1. Backend API (Go): REST API, business logic, data management
  2. Chat Server (Rust): WebSocket-based real-time chat
  3. Stream Server (Rust): Audio streaming (HLS/WebRTC)
  4. Frontend (React): Web application

Communication Patterns

  • REST API: Primary communication between frontend and backend
  • JWT Tokens: Authentication and authorization
  • WebSocket: Real-time chat (delegated to Chat Server)
  • HTTP Callbacks: Integration with Stream Server
  • Message Queue: Asynchronous job processing (RabbitMQ)

Application Architecture

Layered Architecture

The application follows a layered architecture pattern:

┌─────────────────────────────────────────┐
│         HTTP Layer (Gin)                 │
│  Handlers, Middleware, Routes            │
└──────────────┬──────────────────────────┘
               │
┌──────────────▼──────────────────────────┐
│         Core Layer                      │
│  Business Logic, Domain Models          │
│  - auth/                                │
│  - track/                               │
│  - marketplace/                         │
│  - social/                              │
└──────────────┬──────────────────────────┘
               │
┌──────────────▼──────────────────────────┐
│      Service Layer                      │
│  Application Services                   │
│  - UserService                          │
│  - TrackService                         │
│  - PlaylistService                      │
│  - AnalyticsService                     │
└──────────────┬──────────────────────────┘
               │
┌──────────────▼──────────────────────────┐
│      Repository Layer                   │
│  Data Access (GORM)                     │
│  - UserRepository                       │
│  - TrackRepository                      │
│  - PlaylistRepository                   │
└──────────────┬──────────────────────────┘
               │
┌──────────────▼──────────────────────────┐
│      Database Layer                     │
│  PostgreSQL                              │
└─────────────────────────────────────────┘

Core Domains

The application is organized into core domains:

1. Authentication Domain (internal/core/auth/)

  • Purpose: User authentication and authorization
  • Components:
    • AuthService: Authentication logic
    • JWTService: JWT token generation and validation
    • SessionService: Session management
    • TwoFactorService: 2FA implementation
  • Models: User, Session, RefreshToken

2. Track Domain (internal/core/track/)

  • Purpose: Audio track management
  • Components:
    • TrackService: Track business logic
    • TrackHandler: HTTP handlers
    • TrackUploadService: Upload processing
    • TrackSearchService: Search functionality
  • Models: Track, TrackVersion, TrackLike

3. Marketplace Domain (internal/core/marketplace/)

  • Purpose: Marketplace operations
  • Components:
    • MarketplaceService: Product and order management
    • LicenseService: License management
  • Models: Product, Order, OrderItem, License

4. Social Domain (internal/core/social/)

  • Purpose: Social features
  • Components: Comments, likes, sharing
  • Models: Comment, Like, Share

Code Structure

Directory Organization

veza-backend-api/
├── cmd/
│   └── api/
│       └── main.go              # Application entry point
├── internal/
│   ├── api/                      # HTTP layer
│   │   ├── router.go            # Route configuration
│   │   └── middleware.go        # Middleware setup
│   ├── core/                     # Business logic (domains)
│   │   ├── auth/                # Authentication domain
│   │   ├── track/               # Track domain
│   │   ├── marketplace/         # Marketplace domain
│   │   └── social/              # Social domain
│   ├── handlers/                # HTTP handlers (legacy)
│   ├── services/                # Application services
│   ├── repositories/            # Data access layer
│   ├── models/                  # Data models (GORM)
│   ├── middleware/              # HTTP middleware
│   ├── config/                  # Configuration management
│   ├── database/                # Database connection & migrations
│   ├── workers/                 # Background job workers
│   ├── validators/              # Input validation
│   ├── errors/                  # Error handling
│   ├── metrics/                 # Prometheus metrics
│   └── logging/                 # Structured logging
├── migrations/                  # Database migrations
├── tests/                       # Integration tests
└── docs/                        # Documentation

Package Organization Principles

  1. Domain-Driven: Core business logic organized by domain
  2. Separation of Concerns: Clear boundaries between layers
  3. Dependency Inversion: High-level modules don't depend on low-level modules
  4. Single Responsibility: Each package has a single, well-defined purpose

Design Patterns

1. Repository Pattern

Data access is abstracted through repositories:

type UserRepository interface {
    GetByID(ctx context.Context, id uuid.UUID) (*models.User, error)
    GetByEmail(ctx context.Context, email string) (*models.User, error)
    Create(ctx context.Context, user *models.User) error
    Update(ctx context.Context, user *models.User) error
}

2. Service Layer Pattern

Business logic is encapsulated in services:

type UserService struct {
    repo UserRepository
    // ... other dependencies
}

func (s *UserService) CreateUser(ctx context.Context, req CreateUserRequest) (*User, error) {
    // Business logic here
}

3. Handler Pattern

HTTP handlers delegate to services:

func (h *UserHandler) CreateUser(c *gin.Context) {
    var req CreateUserRequest
    if err := c.ShouldBindJSON(&req); err != nil {
        // Handle error
    }
    
    user, err := h.userService.CreateUser(c.Request.Context(), req)
    // Handle response
}

4. Middleware Pattern

Cross-cutting concerns handled via middleware:

  • Authentication
  • Authorization
  • CORS
  • Rate Limiting
  • Request Timeout
  • Error Recovery
  • Metrics Collection
  • Logging

5. Worker Pattern

Background jobs processed by workers:

type JobWorker struct {
    queue   JobQueue
    workers []Worker
}

func (w *JobWorker) EnqueueAnalyticsJob(eventName string, userID *uuid.UUID, payload interface{}) {
    // Enqueue job
}

Data Flow

Request Flow

1. HTTP Request
   ↓
2. Middleware Chain
   - CORS
   - Rate Limiting
   - Authentication
   - Request Timeout
   ↓
3. Route Handler
   ↓
4. Service Layer
   - Business Logic
   - Validation
   ↓
5. Repository Layer
   - Data Access
   ↓
6. Database
   ↓
7. Response

Authentication Flow

1. User submits credentials
   ↓
2. AuthService.ValidateCredentials()
   ↓
3. JWTService.GenerateToken()
   ↓
4. SessionService.CreateSession()
   ↓
5. Return JWT + Refresh Token

Upload Flow

1. User uploads file
   ↓
2. UploadValidator.Validate() (ClamAV scan)
   ↓
3. TrackUploadService.ProcessUpload()
   ↓
4. TrackService.CreateTrack()
   ↓
5. Repository.Save()
   ↓
6. Background job: Process audio metadata
   ↓
7. Notify Stream Server

Marketplace Purchase Flow

1. User creates order
   ↓
2. MarketplaceService.ValidateOrder()
   ↓
3. Payment processing (Stripe)
   ↓
4. OrderService.CreateOrder() (transaction)
   ↓
5. LicenseService.GenerateLicense()
   ↓
6. OrderService.CompleteOrder()
   ↓
7. Webhook: Order completed event

Service Integration

Chat Server Integration

The Backend API generates JWT tokens for the Chat Server:

// Endpoint: POST /api/v1/chat/token
func (h *ChatHandler) GetToken(c *gin.Context) {
    userID := getUserIDFromContext(c)
    token := h.chatService.GenerateToken(userID)
    // Return token
}

Flow:

  1. Frontend requests chat token from Backend API
  2. Backend API generates short-lived JWT
  3. Frontend uses token to connect to Chat Server
  4. Chat Server validates token and establishes WebSocket connection

Stream Server Integration

The Backend API notifies the Stream Server when tracks are ready:

// Endpoint: POST /api/v1/internal/tracks/:id/stream-ready
func (h *StreamHandler) NotifyStreamReady(c *gin.Context) {
    trackID := c.Param("id")
    h.streamService.NotifyReady(trackID)
}

Flow:

  1. Track upload completes
  2. Backend API processes track
  3. Backend API calls Stream Server to prepare streaming
  4. Stream Server generates HLS segments
  5. Stream Server notifies Backend API when ready
  6. Backend API updates track status

External Services

  • Stripe: Payment processing for marketplace
  • S3/MinIO: File storage for audio tracks
  • ClamAV: Antivirus scanning for uploads
  • Sentry: Error tracking and monitoring
  • Email Service: Transactional emails

Security Architecture

Authentication

  • JWT Tokens: HS256 algorithm
  • Token Expiration: Configurable (default 24h)
  • Refresh Tokens: Long-lived tokens for token renewal
  • Token Versioning: Supports token revocation

Authorization

  • RBAC: Role-Based Access Control
  • Ownership Checks: Resource ownership validation
  • Permission Middleware: Route-level permission checks

Security Layers

  1. Network Layer: HTTPS/TLS
  2. Application Layer:
    • CORS protection
    • CSRF tokens
    • Rate limiting
    • Input validation
    • SQL injection prevention (GORM)
  3. Data Layer:
    • Password hashing (bcrypt)
    • Encrypted sensitive data
    • Secure session storage

Security Features

  • 2FA Support: TOTP-based two-factor authentication
  • Password Policies: Strong password requirements
  • Rate Limiting: Prevents brute force attacks
  • ClamAV Integration: Virus scanning for uploads
  • Audit Logging: Security event tracking

Performance & Scalability

Database Optimization

  • Connection Pooling: Configurable pool size
  • Query Optimization: Indexed queries, prepared statements
  • Caching: Redis for frequently accessed data
  • Read Replicas: Support for read replicas (future)

Caching Strategy

  • Redis Cache:
    • User sessions
    • Rate limiting counters
    • Frequently accessed data
    • API response caching

Scalability Patterns

  • Horizontal Scaling: Stateless API design
  • Load Balancing: Multiple API instances
  • Database Sharding: Future consideration
  • CDN Integration: Static asset delivery

Performance Optimizations

  • Async Processing: Background jobs for heavy operations
  • Chunked Uploads: Large file upload support
  • Pagination: Efficient data retrieval
  • Lazy Loading: On-demand data loading

Observability

Logging

  • Structured Logging: JSON format (production)
  • Log Levels: DEBUG, INFO, WARN, ERROR
  • Context Propagation: Request IDs, user IDs
  • Log Aggregation: Centralized log collection

Metrics

  • Prometheus Metrics:
    • HTTP request duration
    • Request count by endpoint
    • Error rates
    • Database query duration
    • Cache hit/miss rates

Tracing

  • Request Tracing: Request ID propagation
  • Distributed Tracing: Future integration with OpenTelemetry

Health Checks

  • Liveness Probe: /healthz
  • Readiness Probe: /readyz
  • Dependency Checks: Database, Redis, external services

Architectural Decisions

ADR-001: Go as Primary Language

Decision: Use Go 1.23+ for the backend API

Rationale:

  • Strong typing and compile-time safety
  • Excellent performance
  • Great concurrency support
  • Strong ecosystem for web services
  • Easy deployment (single binary)

Alternatives Considered:

  • Node.js: Runtime errors, callback hell
  • Python: Performance concerns
  • Rust: Steeper learning curve

ADR-002: Gin Web Framework

Decision: Use Gin as the web framework

Rationale:

  • High performance
  • Simple API
  • Good middleware support
  • Active community
  • Well-documented

Alternatives Considered:

  • Echo: Similar features, smaller community
  • Fiber: Less mature
  • Standard library: Too low-level

ADR-003: GORM for Database Access

Decision: Use GORM as the ORM

Rationale:

  • Type-safe queries
  • Migration support
  • Relationship management
  • Active Record pattern

Alternatives Considered:

  • sqlx: More control, less convenience
  • Ent: More complex
  • Raw SQL: Too verbose

ADR-004: Layered Architecture

Decision: Use layered architecture with domain separation

Rationale:

  • Clear separation of concerns
  • Easy to test
  • Maintainable
  • Scalable

Alternatives Considered:

  • Microservices: Too complex for MVP
  • Monolithic: Already using this approach

ADR-005: JWT for Authentication

Decision: Use JWT tokens for authentication

Rationale:

  • Stateless
  • Scalable
  • Industry standard
  • Easy to implement

Alternatives Considered:

  • Session-based: Requires session storage
  • OAuth2: More complex for MVP

ADR-006: PostgreSQL as Primary Database

Decision: Use PostgreSQL as the primary database

Rationale:

  • ACID compliance
  • Rich feature set
  • Excellent performance
  • Strong ecosystem

Alternatives Considered:

  • MySQL: Less features
  • MongoDB: NoSQL not suitable for relational data

ADR-007: Redis for Caching

Decision: Use Redis for caching (optional)

Rationale:

  • High performance
  • Rich data structures
  • Pub/Sub support
  • Widely used

Alternatives Considered:

  • Memcached: Less features
  • In-memory cache: Not shared across instances

ADR-008: RabbitMQ for Message Queue

Decision: Use RabbitMQ for asynchronous job processing (optional)

Rationale:

  • Reliable message delivery
  • Good performance
  • Rich features
  • Easy to set up

Alternatives Considered:

  • Kafka: Too complex for MVP
  • NATS: Less features
  • In-process queue: Not suitable for distributed systems

Future Considerations

Planned Improvements

  1. Microservices Migration: Split into domain-specific services
  2. Event Sourcing: For audit trail and state reconstruction
  3. CQRS: Separate read and write models
  4. GraphQL API: Alternative to REST
  5. gRPC: For inter-service communication
  6. Service Mesh: For service discovery and load balancing

Scalability Roadmap

  1. Phase 1: Current monolithic architecture
  2. Phase 2: Extract chat and streaming to separate services
  3. Phase 3: Domain-based microservices
  4. Phase 4: Full microservices with service mesh

Additional Resources

  • API Documentation: See docs/API_DOCUMENTATION.md
  • Development Guide: See docs/DEVELOPMENT_SETUP_GUIDE.md
  • Deployment Guide: See docs/DEPLOYMENT_GUIDE.md
  • Code Examples: See internal/core/ for domain implementations