Commit graph

23 commits

Author SHA1 Message Date
senke
84e92a75e2 feat(observability): OTel SDK + collector + Tempo + 4 hot path spans (W2 Day 9)
Some checks failed
Veza CI / Notify on failure (push) Blocked by required conditions
Security Scan / Secret Scanning (gitleaks) (push) Waiting to run
Veza CI / Backend (Go) (push) Has been cancelled
Veza CI / Rust (Stream Server) (push) Has been cancelled
Veza CI / Frontend (Web) (push) Has been cancelled
E2E Playwright / e2e (full) (push) Has been cancelled
Wires distributed tracing end-to-end. Backend exports OTLP/gRPC to a
collector, which tail-samples (errors + slow always, 10% rest) and
ships to Tempo. Grafana service-map dashboard pivots on the 4
instrumented hot paths.

- internal/tracing/otlp_exporter.go : InitOTLPTracer + Provider.Shutdown,
  BatchSpanProcessor (5s/512 batch), ParentBased(TraceIDRatio) sampler,
  W3C trace-context + baggage propagators. OTEL_SDK_DISABLED=true
  short-circuits to a no-op. Failure to dial collector is non-fatal.
- cmd/api/main.go : init at boot, defer Shutdown(5s) on exit. appVersion
  ldflag-overridable for resource attributes.
- 4 hot paths instrumented :
    * handlers/auth.go::Login           → "auth.login"
    * core/track/track_upload_handler.go::InitiateChunkedUpload → "track.upload.initiate"
    * core/marketplace/service.go::ProcessPaymentWebhook → "payment.webhook"
    * handlers/search_handlers.go::Search → "search.query"
  PII guarded — email masked, query content not recorded (length only).
- infra/ansible/roles/otel_collector : pin v0.116.1 contrib build,
  systemd unit, tail-sampling config (errors + > 500ms always kept).
- infra/ansible/roles/tempo : pin v2.7.1 monolithic, local-disk backend
  (S3 deferred to v1.1), 14d retention.
- infra/ansible/playbooks/observability.yml : provisions both Incus
  containers + applies common baseline + roles in order.
- inventory/lab.yml : new groups observability, otel_collectors, tempo.
- config/grafana/dashboards/service-map.json : node graph + 4 hot-path
  span tables + collector throughput/queue panels.
- docs/ENV_VARIABLES.md §30 : 4 OTEL_* env vars documented.

Acceptance criterion (Day 9) : login → span visible in Tempo UI. Lab
deployment to validate with `ansible-playbook -i inventory/lab.yml
playbooks/observability.yml` once roles/postgres_ha is up.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 01:15:11 +02:00
senke
73fc6e128a fix(deps): bump x/net to v0.51.0 for GO-2026-4559
HTTP/2 frame handling panic fix in golang.org/x/net. The vuln database
added this entry between the local govulncheck run on 3d1f127ad (clean)
and the CI run on b33227a57 (GO-2026-4559 flagged). Reachable from
PlaylistHandler / SupportHandler / PlaylistExportHandler via standard
http2.* error and frame string helpers — production path, not test-only.

  golang.org/x/net    v0.50.0 → v0.51.0   (GO-2026-4559)

Local verification:
  go build ./...                OK
  go mod tidy                   OK
  govulncheck ./...             OK (no findings)
2026-04-15 15:31:35 +02:00
senke
3d1f127ad0 fix(deps): bump vulnerable modules to unblock govulncheck CI
Backend (Go) CI has been red for the entire v1.0.4 cleanup sprint (and
before it) because govulncheck reports 7 vulnerabilities in transitive
test-infrastructure deps, while the test suite itself passes cleanly.
Bump three direct dependencies to pull fixed versions of the affected
modules.

Direct bumps:
  golang.org/x/image                  v0.36.0 → v0.38.0   (GO-2026-4815)
  github.com/quic-go/quic-go          v0.54.0 → v0.57.0   (GO-2025-4233)
  github.com/testcontainers/testcontainers-go         v0.33.0 → v0.42.0
  github.com/testcontainers/testcontainers-go/modules/postgres
                                                       v0.33.0 → v0.42.0

Indirect / transitive side effects:
  - containerd/containerd v1.7.18 is REMOVED from the dependency graph.
    Newer testcontainers-go depends on containerd/errdefs + log +
    platforms sub-packages only, which do not carry GO-2025-4108 /
    GO-2025-4100 / GO-2025-3528.
  - docker/docker v27.1.1 is REMOVED from the dependency graph for the
    same reason — it was reached only via testcontainers-go, and the
    new version no longer pulls the full Moby engine. This eliminates
    GO-2026-4887 and GO-2026-4883 (the two vulns with no upstream fix)
    WITHOUT needing a govulncheck allowlist/exclude wrapper.
  - quic-go/qpack, x/crypto, x/net, x/sync, x/sys, x/text, x/tools and
    a handful of otel-* modules bumped as a coherent set.
  - Transitive opentelemetry bump (otel v1.24.0 → v1.41.0) is expected
    since testcontainers-go v0.42 pulls a newer instrumentation.

All 7 vulnerabilities previously reported are now resolved:
  GO-2026-4887  docker/docker         — vuln module removed
  GO-2026-4883  docker/docker         — vuln module removed
  GO-2026-4815  x/image               — fixed in v0.38.0
  GO-2025-4233  quic-go               — fixed in v0.57.0
  GO-2025-4108  containerd            — vuln module removed
  GO-2025-4100  containerd            — vuln module removed
  GO-2025-3528  containerd            — vuln module removed

Verification (local):
  go build ./...                                           OK
  go vet ./...                                             OK
  govulncheck ./...                                        OK (no findings)
  VEZA_SKIP_INTEGRATION=1 go test ./internal/... -short   OK

No breaking API changes observed from the testcontainers-go v0.33 →
v0.42 bump (the project only uses GenericContainer, DockerContainer
.Terminate, and modules/postgres which are stable across these
versions). The shared Redis testcontainer helper in internal/testutils
and the hard-delete worker integration test from J4 still compile and
pass.

This commit enables the v1.0.4 tag to be cut on a green CI. No J7
(release) commit is part of this change — that ships separately.

Refs: AUDIT_REPORT.md §10 P5 (test infra hygiene), CI run 98
2026-04-15 14:38:48 +02:00
senke
73eca4f6ad feat: backend, stream server & infra improvements
Backend (Go):
- Config: CORS, RabbitMQ, rate limit, main config updates
- Routes: core, distribution, tracks routing changes
- Middleware: rate limiter, endpoint limiter, response cache hardening
- Handlers: distribution, search handler fixes
- Workers: job worker improvements
- Upload validator and logging config additions
- New migrations: products, orders, performance indexes
- Seed tooling and data

Stream Server (Rust):
- Audio processing, config, routes, simple stream server updates
- Dockerfile improvements

Infrastructure:
- docker-compose.yml updates
- nginx-rtmp config changes
- Makefile improvements (config, dev, high, infra)
- Root package.json and lock file updates
- .env.example updates

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 11:36:06 +01:00
senke
171a154763 feat(v0.10.2): Recherche fulltext Elasticsearch - F361-F365
- Elasticsearch 8.x dans docker-compose.dev
- Package internal/elasticsearch: client, config, mappings, indices
- Sync PG→ES: reindex tracks/users/playlists, IndexTrack/DeleteTrack
- SearchService ES: multi_match + fuzziness (typo tolerance), highlighting
- Fallback gracieux: PostgreSQL si ELASTICSEARCH_URL absent
- Routes: GET /search, GET /search/suggestions, POST /admin/search/reindex
- Frontend: searchApi cursor/limit params (extensibilité)
- docs/ENV_VARIABLES: ELASTICSEARCH_URL, ELASTICSEARCH_INDEX, ELASTICSEARCH_AUTO_INDEX
- Roadmap v0.10.2 → DONE
2026-03-09 10:13:18 +01:00
senke
2a4de3ce21 v0.9.8 2026-03-06 19:13:16 +01:00
senke
f5bca2b642 v0.9.5 2026-03-06 10:02:53 +01:00
senke
72d40990c5 feat(v0.923): API contract tests, OpenAPI generation, CI type sync check
Some checks failed
Backend API CI / test-unit (push) Failing after 0s
Backend API CI / test-integration (push) Failing after 0s
Frontend CI / test (push) Failing after 0s
Storybook Audit / Build & audit Storybook (push) Failing after 0s
2026-02-27 20:23:10 +01:00
senke
83ed4f315b chore(release): v0.602 — Payout, Dette Technique & Tests E2E
Some checks failed
Backend API CI / test-unit (push) Failing after 0s
Backend API CI / test-integration (push) Failing after 0s
Frontend CI / test (push) Failing after 0s
Storybook Audit / Build & audit Storybook (push) Failing after 0s
- Stripe Connect: onboarding, balance, SellerDashboardView
- Interceptors: auth.ts, error.ts extracted, facade
- Grafana: dashboards enriched (p50, top endpoints, 4xx, WS, commerce)
- E2E commerce: product->order->review->invoice
- SMOKE_TEST_V0602, RETROSPECTIVE_V0602, PAYOUT_MANUAL
- Archive V0_602 scope, V0_603 placeholder, SCOPE_CONTROL v0.603
- Fix sanitizer regex (Go no backreferences)
- Marketplace test schema: product_licenses, product_images, orders, licenses
2026-02-23 22:32:01 +01:00
senke
80492a4644 refactor(websocket): replace gorilla/websocket with coder/websocket
INT-06: Migrated playback_websocket_handler.go from deprecated
gorilla/websocket to coder/websocket v1.8.14. Uses context-based
reads/writes and websocket.Accept instead of Upgrader.
2026-02-22 17:53:10 +01:00
senke
166acc6069 chore(backend): add PDF library for invoices
feat(marketplace): add invoice generation service and download endpoint
2026-02-22 16:11:42 +01:00
senke
49e3122e78 feat(notifications): N1.1-N1.3 Web Push subscription, send on events, preferences
- N1.1: POST /notifications/push/subscribe, PushService, migration 090
- N1.2: Send Web Push on follow/like/comment/message via CreateNotification
- N1.3: GET/PUT /notifications/preferences, migration 093
- Shared NotificationService with PushService for profile, track, comment handlers
- Fix MockSocialService GetGlobalFeed, GetTrendingHashtags for tests
2026-02-21 16:41:39 +01:00
senke
b103a09a25 chore: consolidate CI, E2E, backend and frontend updates
- CI: workflows updates (cd, ci), remove playwright.yml
- E2E: global-setup, auth/playlists/profile specs
- Remove playwright-report and test-results artifacts from tracking
- Backend: auth, handlers, services, workers, migrations
- Frontend: components, features, vite config
- Add e2e-results.json to gitignore
- Docs: REMEDIATION_PROGRESS, audit archive
- Rust: chat-server, stream-server updates
2026-02-17 16:43:21 +01:00
senke
afea976f57 chore: add go.work and optional monorepo orchestrator 2026-02-14 18:21:39 +01:00
senke
ae586f6134 Phase 2 stabilisation: code mort, Modal→Dialog, feature flags, tests, router split, Rust legacy
Bloc A - Code mort:
- Suppression Studio (components, views, features)
- Suppression gamification + services mock (projectService, storageService, gamificationService)
- Mise à jour Sidebar, Navbar, locales

Bloc B - Frontend:
- Suppression modal.tsx deprecated, Modal.stories (doublon Dialog)
- Feature flags: PLAYLIST_SEARCH, PLAYLIST_RECOMMENDATIONS, ROLE_MANAGEMENT = true
- Suppression 19 tests orphelins, retrait exclusions vitest.config

Bloc C - Backend:
- Extraction routes_auth.go depuis router.go

Bloc D - Rust:
- Suppression security_legacy.rs (code mort, patterns déjà dans security/)
2026-02-14 17:23:32 +01:00
senke
51869a3649 fix(deps): upgrade gin to 1.11
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-11 21:31:00 +01:00
senke
b28d0e7eac [T0-006] test(backend): Ajout tests pour frontend_log_handler
- Tests complets pour frontend_log_handler.go (12 tests)
- Tests couvrent NewFrontendLogHandler et ReceiveLog
- Tests pour tous les niveaux de log (DEBUG, INFO, WARN, ERROR)
- Tests pour gestion des erreurs et validation JSON
- Couverture actuelle: 30.6% (objectif: 80%)

Files: veza-backend-api/internal/handlers/frontend_log_handler_test.go
       VEZA_ROADMAP.json
Hours: 16 estimated, 23 actual
2026-01-04 01:44:22 +01:00
senke
6e8d797d82 [T0-001] fix(backend): Corriger erreurs compilation Go
- go build ./... réussit sans erreur
- go vet ./... retourne 0 warnings critiques
- Aucune erreur de type dans les handlers
- go mod verify et go mod tidy exécutés avec succès

Files: VEZA_ROADMAP.json, veza-backend-api/go.mod, veza-backend-api/go.sum
Hours: 6 estimated, 1 actual

Le code compile déjà correctement, aucune correction nécessaire.
Vérifications effectuées:
- go build -a ./... ✓
- go vet -all ./... ✓
- go mod verify ✓
- go mod tidy ✓
2026-01-04 01:44:20 +01:00
senke
4c652150c5 [BE-SVC-005] be-svc: Implement file storage abstraction
- Added AWS SDK v2 dependency for S3 support
- Created S3StorageService implementing S3Service interface
- Support for AWS S3 and MinIO (S3-compatible storage)
- Added S3 configuration in config.go with environment variables
- Implemented upload, delete, presigned URL, and public URL methods
- Added unit tests for service validation and URL generation
- Service integrates with existing TrackStorageService
2025-12-24 16:28:51 +01:00
senke
d61d851f65 stabilizing veza-backend-api: phase 1 2025-12-16 11:23:49 -05:00
okinrev
cfe6ed0119 STABILISATION: phase 1 & phase 2 2025-12-06 14:45:07 +01:00
okinrev
b7955a680c P0: stabilisation backend/chat/stream + nouvelle base migrations v1
Backend Go:
- Remplacement complet des anciennes migrations par la base V1 alignée sur ORIGIN.
- Durcissement global du parsing JSON (BindAndValidateJSON + RespondWithAppError).
- Sécurisation de config.go, CORS, statuts de santé et monitoring.
- Implémentation des transactions P0 (RBAC, duplication de playlists, social toggles).
- Ajout d’un job worker structuré (emails, analytics, thumbnails) + tests associés.
- Nouvelle doc backend : AUDIT_CONFIG, BACKEND_CONFIG, AUTH_PASSWORD_RESET, JOB_WORKER_*.

Chat server (Rust):
- Refonte du pipeline JWT + sécurité, audit et rate limiting avancé.
- Implémentation complète du cycle de message (read receipts, delivered, edit/delete, typing).
- Nettoyage des panics, gestion d’erreurs robuste, logs structurés.
- Migrations chat alignées sur le schéma UUID et nouvelles features.

Stream server (Rust):
- Refonte du moteur de streaming (encoding pipeline + HLS) et des modules core.
- Transactions P0 pour les jobs et segments, garanties d’atomicité.
- Documentation détaillée de la pipeline (AUDIT_STREAM_*, DESIGN_STREAM_PIPELINE, TRANSACTIONS_P0_IMPLEMENTATION).

Documentation & audits:
- TRIAGE.md et AUDIT_STABILITY.md à jour avec l’état réel des 3 services.
- Cartographie complète des migrations et des transactions (DB_MIGRATIONS_*, DB_TRANSACTION_PLAN, AUDIT_DB_TRANSACTIONS, TRANSACTION_TESTS_PHASE3).
- Scripts de reset et de cleanup pour la lab DB et la V1.

Ce commit fige l’ensemble du travail de stabilisation P0 (UUID, backend, chat et stream) avant les phases suivantes (Coherence Guardian, WS hardening, etc.).
2025-12-06 11:14:38 +01:00
okinrev
2425c15b09 adding initial backend API (Go) 2025-12-03 20:29:37 +01:00