veza/veza-backend-api/internal/services/cdn_service.go
senke 15e591305e
Some checks failed
Veza CI / Rust (Stream Server) (push) Successful in 5m12s
Security Scan / Secret Scanning (gitleaks) (push) Failing after 54s
Veza CI / Backend (Go) (push) Failing after 8m38s
Veza CI / Frontend (Web) (push) Failing after 16m44s
Veza CI / Notify on failure (push) Successful in 15s
E2E Playwright / e2e (full) (push) Successful in 20m28s
feat(cdn): Bunny.net signed URLs + HLS cache headers + metric collision fix (W3 Day 13)
CDN edge in front of S3/MinIO via origin-pull. Backend signs URLs
with Bunny.net token-auth (SHA-256 over security_key + path + expires)
so edges verify before serving cached objects ; origin is never hit
on a valid token. Cloudflare CDN / R2 / CloudFront stubs kept.

- internal/services/cdn_service.go : new providers CDNProviderBunny +
  CDNProviderCloudflareR2. SecurityKey added to CDNConfig.
  generateBunnySignedURL implements the documented Bunny scheme
  (url-safe base64, no padding, expires query). HLSSegmentCacheHeaders
  + HLSPlaylistCacheHeaders helpers exported for handlers.
- internal/services/cdn_service_test.go : pin Bunny URL shape +
  base64-url charset ; assert empty SecurityKey fails fast (no
  silent fallback to unsigned URLs).
- internal/core/track/service.go : new CDNURLSigner interface +
  SetCDNService(cdn). GetStorageURL prefers CDN signed URL when
  cdnService.IsEnabled, falls back to direct S3 presign on signing
  error so a CDN partial outage doesn't block playback.
- internal/api/routes_tracks.go + routes_core.go : wire SetCDNService
  on the two TrackService construction sites that serve stream/download.
- internal/config/config.go : 4 new env vars (CDN_ENABLED, CDN_PROVIDER,
  CDN_BASE_URL, CDN_SECURITY_KEY). config.CDNService always non-nil
  after init ; IsEnabled gates the actual usage.
- internal/handlers/hls_handler.go : segments now return
  Cache-Control: public, max-age=86400, immutable (content-addressed
  filenames make this safe). Playlists at max-age=60.
- veza-backend-api/.env.template : 4 placeholder env vars.
- docs/ENV_VARIABLES.md §12 : provider matrix + Bunny vs Cloudflare
  vs R2 trade-offs.

Bug fix collateral : v1.0.9 Day 11 introduced veza_cache_hits_total
which collided in name with monitoring.CacheHitsTotal (different
label set ⇒ promauto MustRegister panic at process init). Day 13
deletes the monitoring duplicate and restores the metrics-package
counter as the single source of truth (label: subsystem). All 8
affected packages green : services, core/track, handlers, middleware,
websocket/chat, metrics, monitoring, config.

Acceptance (Day 13) : code path is wired ; verifying via real Bunny
edge requires a Pull Zone provisioned by the user (EX-? in roadmap).
On the user side : create Pull Zone w/ origin = MinIO, copy token
auth key into CDN_SECURITY_KEY, set CDN_ENABLED=true.

W3 progress : Redis Sentinel ✓ · MinIO distribué ✓ · CDN ✓ ·
DMCA  Day 14 · embed  Day 15.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 14:07:20 +02:00

347 lines
11 KiB
Go

package services
import (
"crypto/sha256"
"encoding/base64"
"strconv"
"context"
"fmt"
"strings"
"time"
"go.uber.org/zap"
)
// CDNProvider represents the type of CDN provider
type CDNProvider string
const (
CDNProviderCloudFront CDNProvider = "cloudfront"
CDNProviderCloudflare CDNProvider = "cloudflare"
CDNProviderCloudflareR2 CDNProvider = "cloudflare_r2"
CDNProviderBunny CDNProvider = "bunny" // v1.0.9 W3 Day 13 — Bunny.net Stream w/ token auth
CDNProviderGeneric CDNProvider = "generic"
CDNProviderNone CDNProvider = "none"
)
// CDNConfig represents configuration for CDN service
type CDNConfig struct {
Provider CDNProvider
BaseURL string // CDN base URL (e.g., https://d1234567890.cloudfront.net)
DistributionID string // CloudFront distribution ID (if applicable)
APIKey string // API key for cache invalidation (if applicable)
// SecurityKey is the provider-specific signing key. For Bunny.net,
// this is the Pull Zone "Token Authentication Key" (Bunny dashboard
// → Pull Zone → Security → Token Authentication). Sensitive — keep
// out of logs, encrypt at rest. (v1.0.9 W3 Day 13.)
SecurityKey string
Enabled bool
Logger *zap.Logger
}
// CDNService provides CDN integration capabilities
// BE-SVC-013: Implement CDN integration
type CDNService struct {
config CDNConfig
logger *zap.Logger
}
// NewCDNService creates a new CDN service
func NewCDNService(config CDNConfig) *CDNService {
if config.Logger == nil {
config.Logger = zap.NewNop()
}
return &CDNService{
config: config,
logger: config.Logger,
}
}
// GetURL generates a CDN URL for a given path
func (s *CDNService) GetURL(path string) string {
if !s.config.Enabled || s.config.Provider == CDNProviderNone {
return path // Return original path if CDN is disabled
}
// Remove leading slash if present
path = strings.TrimPrefix(path, "/")
// Ensure base URL doesn't have trailing slash
baseURL := strings.TrimSuffix(s.config.BaseURL, "/")
// Construct CDN URL
return fmt.Sprintf("%s/%s", baseURL, path)
}
// GetURLs generates CDN URLs for multiple paths
func (s *CDNService) GetURLs(paths []string) map[string]string {
result := make(map[string]string)
for _, path := range paths {
result[path] = s.GetURL(path)
}
return result
}
// GetAssetURL generates a CDN URL for a static asset
func (s *CDNService) GetAssetURL(assetType string, filename string) string {
path := fmt.Sprintf("assets/%s/%s", assetType, filename)
return s.GetURL(path)
}
// GetAudioURL generates a CDN URL for an audio file
func (s *CDNService) GetAudioURL(trackID string, filename string) string {
path := fmt.Sprintf("audio/%s/%s", trackID, filename)
return s.GetURL(path)
}
// GetHLSURL generates a CDN URL for an HLS stream
func (s *CDNService) GetHLSURL(trackID string, path string) string {
fullPath := fmt.Sprintf("hls/%s/%s", trackID, path)
return s.GetURL(fullPath)
}
// GetImageURL generates a CDN URL for an image
func (s *CDNService) GetImageURL(imageType string, filename string) string {
path := fmt.Sprintf("images/%s/%s", imageType, filename)
return s.GetURL(path)
}
// InvalidateCache invalidates CDN cache for given paths
func (s *CDNService) InvalidateCache(ctx context.Context, paths []string) error {
if !s.config.Enabled {
s.logger.Debug("CDN cache invalidation skipped (CDN disabled)")
return nil
}
if len(paths) == 0 {
return nil
}
switch s.config.Provider {
case CDNProviderCloudFront:
return s.invalidateCloudFront(ctx, paths)
case CDNProviderCloudflare:
return s.invalidateCloudflare(ctx, paths)
case CDNProviderGeneric:
// Generic CDN - just log, actual invalidation would need API integration
s.logger.Info("CDN cache invalidation requested (generic provider)",
zap.Strings("paths", paths),
)
return nil
default:
s.logger.Warn("CDN cache invalidation not supported for provider",
zap.String("provider", string(s.config.Provider)),
)
return nil
}
}
// invalidateCloudFront invalidates CloudFront cache
func (s *CDNService) invalidateCloudFront(ctx context.Context, paths []string) error {
// Note: Full CloudFront invalidation would require AWS SDK
// This is a placeholder implementation
s.logger.Info("CloudFront cache invalidation requested",
zap.String("distribution_id", s.config.DistributionID),
zap.Strings("paths", paths),
)
// In a real implementation, this would:
// 1. Create CloudFront invalidation request
// 2. Use AWS SDK to submit invalidation
// 3. Return invalidation ID for tracking
return nil
}
// invalidateCloudflare invalidates Cloudflare cache
func (s *CDNService) invalidateCloudflare(ctx context.Context, paths []string) error {
// Note: Full Cloudflare invalidation would require Cloudflare API
// This is a placeholder implementation
s.logger.Info("Cloudflare cache invalidation requested",
zap.Strings("paths", paths),
)
// In a real implementation, this would:
// 1. Call Cloudflare API to purge cache
// 2. Handle API authentication
// 3. Return purge result
return nil
}
// IsEnabled returns whether CDN is enabled
func (s *CDNService) IsEnabled() bool {
return s.config.Enabled && s.config.Provider != CDNProviderNone
}
// GetProvider returns the CDN provider type
func (s *CDNService) GetProvider() CDNProvider {
return s.config.Provider
}
// GetBaseURL returns the CDN base URL
func (s *CDNService) GetBaseURL() string {
return s.config.BaseURL
}
// GenerateSignedURL generates a signed URL for private content (if supported)
func (s *CDNService) GenerateSignedURL(path string, expiration time.Duration) (string, error) {
if !s.config.Enabled {
return s.GetURL(path), nil
}
switch s.config.Provider {
case CDNProviderCloudFront:
return s.generateCloudFrontSignedURL(path, expiration)
case CDNProviderBunny:
return s.generateBunnySignedURL(path, expiration)
case CDNProviderCloudflare, CDNProviderCloudflareR2:
// Cloudflare CDN/R2 doesn't expose signed URLs natively from the
// edge — use Workers if you need them. For v1.0 we treat these
// as public-cache CDNs and return the unsigned URL ; access
// control happens upstream (auth check before redirect).
return s.GetURL(path), nil
default:
return s.GetURL(path), nil
}
}
// generateBunnySignedURL signs a path with Bunny.net's Token
// Authentication scheme. Bunny edges verify the token+expires query
// pair before serving the cached object — invalid or expired tokens
// return 403 directly from the edge (origin is never hit).
//
// Format (per https://docs.bunny.net/docs/cdn-token-authentication) :
//
// token = url_safe_base64( sha256_raw( security_key + path + expires ) )
// .strip("=")
// URL = https://<base>/<path>?token=<token>&expires=<unix>
//
// Notes :
// - `path` is the URL path INCLUDING the leading slash.
// - The hash is over the raw bytes of sha256, NOT the hex digest.
// - We strip the trailing "=" padding to match Bunny's reference impl.
// - We DO NOT include client IP — that's an optional Bunny feature
// and would prevent CDN-level caching (every IP needs its own
// token URL). v1.0 trades stricter binding for cache hit ratio.
func (s *CDNService) generateBunnySignedURL(path string, expiration time.Duration) (string, error) {
if s.config.SecurityKey == "" {
return "", fmt.Errorf("bunny CDN signing requires CDN_SECURITY_KEY")
}
// Normalise to a leading slash — Bunny signs the path that the
// edge URL parses, and the edge always sees a leading slash.
if !strings.HasPrefix(path, "/") {
path = "/" + path
}
expires := strconv.FormatInt(time.Now().Add(expiration).Unix(), 10)
hasher := sha256.New()
hasher.Write([]byte(s.config.SecurityKey + path + expires))
token := strings.TrimRight(base64.URLEncoding.EncodeToString(hasher.Sum(nil)), "=")
baseURL := strings.TrimSuffix(s.config.BaseURL, "/")
return fmt.Sprintf("%s%s?token=%s&expires=%s", baseURL, path, token, expires), nil
}
// generateCloudFrontSignedURL generates a CloudFront signed URL
func (s *CDNService) generateCloudFrontSignedURL(path string, expiration time.Duration) (string, error) {
// Note: Full CloudFront signed URL generation would require AWS SDK
// This is a placeholder implementation
s.logger.Info("CloudFront signed URL generation requested",
zap.String("path", path),
zap.Duration("expiration", expiration),
)
// In a real implementation, this would:
// 1. Use AWS SDK to generate signed URL
// 2. Include expiration time
// 3. Sign with private key
// For now, return regular URL
return s.GetURL(path), nil
}
// GetCacheHeaders returns appropriate cache headers for CDN
func (s *CDNService) GetCacheHeaders() map[string]string {
headers := make(map[string]string)
if !s.config.Enabled {
headers["Cache-Control"] = "no-cache"
return headers
}
switch s.config.Provider {
case CDNProviderCloudFront:
headers["Cache-Control"] = "public, max-age=31536000, immutable"
headers["X-CDN-Provider"] = "cloudfront"
case CDNProviderCloudflare, CDNProviderCloudflareR2:
headers["Cache-Control"] = "public, max-age=31536000, immutable"
headers["X-CDN-Provider"] = "cloudflare"
case CDNProviderBunny:
headers["Cache-Control"] = "public, max-age=31536000, immutable"
headers["X-CDN-Provider"] = "bunny"
default:
headers["Cache-Control"] = "public, max-age=3600"
}
return headers
}
// HLSSegmentCacheHeaders returns the cache headers to apply to a
// /tracks/:id/hls/* response (segments + playlists). The values are
// what the v1.0.9 W3 Day 13 acceptance asks for : the segment is
// content-addressed (filename includes a hash), so a long max-age +
// `immutable` directive lets every layer (browser, CDN, origin) skip
// re-validation entirely.
//
// Use these on the BACKEND HLS response directly even when CDN is
// disabled — they're correct caching semantics in either mode.
func HLSSegmentCacheHeaders() map[string]string {
return map[string]string{
"Cache-Control": "public, max-age=86400, immutable",
"Vary": "Accept-Encoding",
}
}
// HLSPlaylistCacheHeaders is for the .m3u8 manifests. They CAN change
// (live streams, transcoder retries) so a shorter TTL is safer.
// 60s is a balance : edges cache aggressively for the common case but
// a stale manifest only blocks a viewer for a minute, not a day.
func HLSPlaylistCacheHeaders() map[string]string {
return map[string]string{
"Cache-Control": "public, max-age=60",
"Vary": "Accept-Encoding",
}
}
// BatchInvalidate invalidates multiple paths in batches (useful for rate limits)
func (s *CDNService) BatchInvalidate(ctx context.Context, paths []string, batchSize int) error {
if batchSize <= 0 {
batchSize = 10 // Default batch size
}
for i := 0; i < len(paths); i += batchSize {
end := i + batchSize
if end > len(paths) {
end = len(paths)
}
batch := paths[i:end]
if err := s.InvalidateCache(ctx, batch); err != nil {
s.logger.Warn("Failed to invalidate cache batch",
zap.Int("batch_start", i),
zap.Int("batch_end", end),
zap.Error(err),
)
// Continue with next batch even if one fails
}
// Small delay between batches to respect rate limits
if end < len(paths) {
time.Sleep(100 * time.Millisecond)
}
}
return nil
}