a3f4ac6b70
- Update E2E test credentials to match actual seed users (user@veza.music, artist@veza.music, admin@veza.music, mod@veza.music) - Fix hardcoded "Suggested Accounts" in SuggestionsWidget with i18n key - Replace hardcoded amelie_dubois references with CONFIG.users.creator - Refactor auth, player, upload E2E tests for reliability - Add tmt test plans and scripts for CI integration - Simplify CI workflow Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
94 lines
3.6 KiB
Markdown
94 lines
3.6 KiB
Markdown
# Veza Unified Testing Pipeline (TMT)
|
|
|
|
> **Philosophy**: Sobriety, Frugality, Accessibility.
|
|
> **The Law**: [docs/FRUGALITY.md](../docs/FRUGALITY.md)
|
|
> **The Contract**: [docs/BUDGETS.md](../docs/BUDGETS.md)
|
|
|
|
TMT is the **single entry point** for all Veza tests.
|
|
The CI calls `tmt run`, never `go test`, `vitest`, or `cargo test` directly.
|
|
|
|
## Rules
|
|
|
|
1. **Vital tests block everything.** If a tier 1 test fails, the commit is rejected.
|
|
2. **Contractual budgets.** Resource limits from `docs/BUDGETS.md` are enforced by tests.
|
|
3. **No `|| true`.** A test that passes despite regression is worse than no test.
|
|
4. **New frontend tests are legacy by default.** Promote to vital only if it protects a critical invariant.
|
|
|
|
## Plans
|
|
|
|
| Plan | Tier | Scope | Blocking | Usage |
|
|
|------|------|-------|----------|-------|
|
|
| `/vital` | 1 | All components | Yes | Local: `make test-tmt` |
|
|
| `/vital-backend` | 1 | Go backend only | Yes | CI parallel job |
|
|
| `/vital-frontend` | 1 | Web frontend only | Yes | CI parallel job |
|
|
| `/vital-services` | 1 | Rust services only | Yes | CI parallel job |
|
|
| `/legacy` | 2 | Slow / integration tests | No (warning) | `tmt --root tmt run plan --name /legacy` |
|
|
| `/integration` | 2 | Tests needing infra (DB, Redis) | No (warning) | Requires docker-compose |
|
|
| `/nightly` | 3 | E2E Playwright, Storybook | No | Nightly / pre-release |
|
|
|
|
## Directory Structure
|
|
|
|
```
|
|
tmt/
|
|
├── .fmf/version # FMF tree root
|
|
├── plans/
|
|
│ ├── vital.fmf # Tier 1 — all components
|
|
│ ├── vital-backend.fmf # Tier 1 — Go only (CI)
|
|
│ ├── vital-frontend.fmf # Tier 1 — Web only (CI)
|
|
│ ├── vital-services.fmf # Tier 1 — Rust only (CI)
|
|
│ ├── legacy.fmf # Tier 2 — slow / secondary
|
|
│ ├── integration.fmf # Tier 2 — needs infra
|
|
│ └── nightly.fmf # Tier 3 — E2E, storybook
|
|
├── tests/
|
|
│ ├── backend/ # Go: govulncheck, vet, lint, unit, build, etc.
|
|
│ ├── frontend/ # Web: audit, lint, typecheck, unit, build, etc.
|
|
│ ├── services/ # Rust: audit, clippy, build, test
|
|
│ ├── e2e/ # Playwright (tier 3)
|
|
│ └── storybook/ # Storybook audit (tier 3)
|
|
└── README.md
|
|
```
|
|
|
|
## How to Run
|
|
|
|
```bash
|
|
# All vital tests (the standard)
|
|
make test-tmt
|
|
|
|
# Component-specific
|
|
make test-tmt-backend
|
|
make test-tmt-frontend
|
|
make test-tmt-services
|
|
|
|
# Direct TMT commands
|
|
tmt --root tmt run plan --name /vital # All vital
|
|
tmt --root tmt run plan --name /vital-backend # Backend only
|
|
tmt --root tmt run plan --name /integration # Integration (needs infra)
|
|
tmt --root tmt run plan --name /nightly # E2E + storybook
|
|
tmt --root tmt run # Everything
|
|
|
|
# Install TMT
|
|
pip install tmt
|
|
```
|
|
|
|
## Test Execution Order
|
|
|
|
Tests within each component use the `order` attribute for sequencing:
|
|
|
|
| Order | Phase | Examples |
|
|
|-------|-------|---------|
|
|
| 10 | Security audits | govulncheck, npm audit, cargo audit |
|
|
| 15 | Code generation | Types sync check |
|
|
| 20 | Static analysis | vet, lint, format, clippy, core isolation |
|
|
| 30 | Type checking | TypeScript typecheck |
|
|
| 40 | Unit tests | go test, vitest, cargo test |
|
|
| 50 | Build | go build, cargo build, npm run build |
|
|
| 60 | Post-build | Bundle size, build perf |
|
|
|
|
Fast checks fail early. Heavy checks only run if basic hygiene passes.
|
|
|
|
## Environment Variables
|
|
|
|
| Variable | Value | Purpose |
|
|
|----------|-------|---------|
|
|
| `GOMAXPROCS` | 1 | Low-power backend (set in unit.sh) |
|
|
| `RUST_BACKTRACE` | 0 | Reduce noise (set in plans) |
|