veza/docs/CI_E2E.md

E2E CI — runbook

v1.0.8 Batch C — Playwright E2E suite running on Forgejo Actions. Workflow: .github/workflows/e2e.yml. Tests: tests/e2e/*.spec.ts. Skipped tests inventory: tests/e2e/SKIPPED_TESTS.md.

---

Triggers

Trigger	Scope	Target time	Why
PR opened / synced (against main)	@critical only	~5–7 min	Fast feedback loop, blocks merge if red
Push to main	Full suite	~25 min	Catches regressions that slipped past @critical
Nightly cron (03:00 UTC)	Full suite	~25 min	Catches infra drift independent of merges
workflow_dispatch	Full suite	manual	Re-run after a flaky failure or on a feature branch

@critical is a Playwright --grep tag — see npm run e2e:critical.

---

How a CI run works

1. actions/checkout + setup-node@20 + setup-go@1.25.
2. npm ci from repo root.
3. Adds 127.0.0.1 veza.fr to /etc/hosts so the browsers can hit the dev domain.
4. Generates dev JWT keys + SSL cert via the existing scripts.
5. Brings up postgres / redis / rabbitmq via docker compose.
6. Runs Go migrations.
7. go run ./cmd/tools/seed --ci — the lean seed: 5 test accounts
   • 10 tracks + 3 playlists, no chat/live/marketplace/analytics. ~5s.
8. Builds + starts the backend on localhost:18080, asserts /api/v1/health.
9. playwright install --with-deps chromium.
10. Runs npm run e2e:critical (PR) or npm run e2e (push/cron). CI=true is exported globally so playwright.config.ts:141,155 spawns its own Vite + backend instance instead of trying to reuse.
11. On failure: uploads the Playwright HTML report and backend.log as artifacts, retained 7 days.

---

Required secrets (Forgejo)

The workflow falls back to dev defaults so it can still run on a fresh repo without secrets configured, but production-style runs should set these in Forgejo Actions secrets:

Secret	Default fallback	Purpose
E2E_DB_PASSWORD	devpassword	Postgres password (must match docker-compose.yml)
E2E_JWT_SECRET	ci-dev-jwt-secret-32-chars-min-padding!!	HS256 signing key (32+ chars)
E2E_RABBITMQ_URL	amqp://guest:guest@localhost:5672/	RabbitMQ AMQP URL

Without these, the workflow still passes for everything that doesn't exercise WebSocket / RabbitMQ paths under load.

---

Reproducing a CI failure locally

Mirrors the workflow exactly:

# From repo root
make infra-up-dev                  # postgres + redis + rabbitmq
cd veza-backend-api
go run cmd/migrate_tool/main.go
go run ./cmd/tools/seed --ci       # 5 test accounts only
go build -o veza-api ./cmd/api/main.go
APP_ENV=test ./veza-api &

# In another shell
cd apps/web && npm run dev -- --host 127.0.0.1 --port 5174 &

# Run the same tests CI ran
cd /path/to/repo
CI=true npm run e2e:critical       # PR scope
# or
CI=true npm run e2e                # full suite

If the failure only reproduces under CI=true, suspect reuseExistingServer — set CI= (empty) to flip back to local mode and bisect.

---

Debugging a red run

1. Open the run in Forgejo Actions UI.
2. Find the failing job's "Run E2E" step. Each test failure shows the selector / assertion / screenshot inline.
3. Scroll to the artifact section: download playwright-report-<run-id>-<attempt> (the HTML report — opens in any browser, shows trace viewer + video for retry-on-fail) and backend-log-<run-id>-<attempt> (full backend stdout + stderr).
4. If the failure looks env-related (404 on a known route, 500 without a clear cause), check backend-log for panics or migration errors before assuming a test bug.
5. Cross-check tests/e2e/SKIPPED_TESTS.md — if the test is already listed as flaky, the right fix may be .skip() until the underlying app bug is tracked.

---

Adding a new E2E test

1. Drop a *.spec.ts file under tests/e2e/.
2. Tag it with @critical if it must run on every PR (be conservative — every @critical test extends the PR feedback loop).
3. Use the auth fixture from tests/e2e/fixtures/auth.fixture.ts (listenerPage / creatorPage / adminPage / moderatorPage) instead of writing UI login flows.
4. If the test needs DB state outside the --ci seed (rare), seed it from inside the test via page.request.post(...) rather than extending the seed tool — keeps the seed lean.
5. Run locally with CI=true npm run e2e:critical -- --grep "your test" before pushing.

---

Scaling considerations

• Forgejo runner pool is shared across CI workflows — keep PR runs under 10 min so we don't hold a runner during peak hours.
• docker compose up -d postgres redis rabbitmq reuses the dev compose file; if that file changes, the workflow inherits the change automatically.
• The full suite is gated to push/cron/dispatch precisely because we don't want to pay 25 min on every PR push.

---

Accessibility (@a11y axe-core WCAG 2.1 AA scan)

v1.0.10 ops item 12 — tests/e2e/24-axe-wcag.spec.ts runs @axe-core/playwright against home, login, register, dashboard, discover, and search. The test fails on any serious or critical axe violation at WCAG 2.1 AA level ; moderate / minor violations are logged but don't gate.

• Trigger : every PR + push (alongside @critical). Adds ~30s.
• Local run : npm run e2e:a11y.
• Reports : violation summary printed to stdout in the CI log ; the full Playwright HTML report is uploaded as an artifact on failure (see "Upload Playwright report" step).
• Exclusions : [data-sonner-toaster] + .toast-container (toast portal nodes occasionally flagged for transient color contrast).
• Adding a new route : append to the targets array in 24-axe-wcag.spec.ts. If it's auth-only, set requiresAuth: true.
• Fixing a violation : the test output prints rule id + impact + count of nodes. Cross-reference with https://dequeuniversity.com/rules/axe/4.10/ for the remediation guide.

---

Related

• tests/e2e/playwright.config.ts — base config, reuseExistingServer: !process.env.CI (committed in v1.0.8 C3, commit 46d21c5c).
• veza-backend-api/cmd/tools/seed/config.go — CIConfig() and the --ci flag (committed in v1.0.8 C4, commit cee850a5).
• tests/e2e/SKIPPED_TESTS.md — known flakes + tickets to resolve.
• docs/audit-2026-04/v107-plan.md — historical context for E2E coverage gaps that landed in v1.0.7.