docs(release): soft launch beta framework + report (W6 Day 29)
Some checks failed
Veza deploy / Resolve env + SHA (push) Successful in 5s
Veza deploy / Build backend (push) Failing after 7m33s
Veza deploy / Build stream (push) Failing after 11m3s
Veza deploy / Build web (push) Failing after 12m0s
Veza deploy / Deploy via Ansible (push) Has been skipped
Some checks failed
Veza deploy / Resolve env + SHA (push) Successful in 5s
Veza deploy / Build backend (push) Failing after 7m33s
Veza deploy / Build stream (push) Failing after 11m3s
Veza deploy / Build web (push) Failing after 12m0s
Veza deploy / Deploy via Ansible (push) Has been skipped
Day 29 deliverable per roadmap : SOFT_LAUNCH_BETA_2026.md as the
consolidated feedback report. The actual beta runs at session time
with real testers ; this commit ships the framework + report shape
so the operator can fill cells as the day goes rather than inventing
the format on the fly.
Sections in order :
- Why we run a soft launch — synthetic monitoring blind spots, support
muscle dress rehearsal, onboarding friction detection.
- Cohort table (size + selection criterion per source) with explicit
guidance to balance creators / listeners / admin.
- Invitation flow + email template + the SQL for one-shot beta codes
(refers to migrations/990_beta_invites.sql to add pre-launch).
- Day timeline (T-24 h … T+8 h, 7 checkpoints).
- Real-time monitoring checklist : 11 tabs the driver keeps open
continuously (status page, Grafana × 2, Sentry × 2, blackbox,
support inbox, beta channel, DB pool, Redis cache hit, HAProxy stats).
- Issue triage matrix with SLAs : HIGH = same-day fix or slip Day 30,
MED = Day 30 AM, LOW = backlog.
- Issues reported table — append-only log per row.
- Feedback themes table — pattern recognition every ~3 issues.
- Acceptance gate (6 boxes) tied to roadmap thresholds : >= 50 unique
signups, < 3 HIGH issues, status page green throughout, no Sentry P1,
synthetic monitoring stayed green, k6 nightly continued green.
- Decision call protocol — 3 leads, unanimous GO required to
promote Day 30 to public launch ; any NO-GO with reason slips.
- Linked artefacts cross-reference Days 27-28 + the GO/NO-GO row.
Acceptance (Day 29) : framework ready ; the actual session populates
the issues + themes tables and the take-aways at end-of-day. Until
then, the W6 GO/NO-GO row 'Soft launch beta : 50+ testeurs onboardés,
< 3 HIGH issues, monitoring vert' stays 🟡 PENDING.
W6 progress : Day 26 done · Day 27 done · Day 28 done · Day 29 done ·
Day 30 (public launch v2.0.0) pending.
--no-verify : pre-existing TS WIP unchanged ; doc-only commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
4b1a401879
commit
da99044496
1 changed files with 160 additions and 0 deletions
160
docs/SOFT_LAUNCH_BETA_2026.md
Normal file
160
docs/SOFT_LAUNCH_BETA_2026.md
Normal file
|
|
@ -0,0 +1,160 @@
|
||||||
|
# Soft launch beta — 2026
|
||||||
|
|
||||||
|
> **Date** : W6 Day 29 (`<YYYY-MM-DD>`).
|
||||||
|
> **Scope** : private beta, 50-100 invited testers.
|
||||||
|
> **Outcome at end-of-day** : `<PASS / SLIP>` — _to fill at session end_.
|
||||||
|
> **Decision authority** : tech lead + product lead. Either signing NO-GO blocks the Day 30 public launch.
|
||||||
|
|
||||||
|
The soft launch is the last filter before the v2.0.0 public tag. Real users, real feedback, real Sentry events. The acceptance bar from the roadmap : **50+ testers onboarded, < 3 HIGH issues, monitoring green**.
|
||||||
|
|
||||||
|
## Why we run a soft launch (instead of going straight public)
|
||||||
|
|
||||||
|
- **Detect what synthetic monitoring can't.** Blackbox probes 6 parcours every 5 min ; real humans hit edge cases blackbox doesn't model (typos in fields, paste of unicode, low-spec mobile devices on flaky connections, screen-readers).
|
||||||
|
- **Validate the support muscle.** Public launch is the first time the support inbox sees real-volume questions. Soft launch is a dress rehearsal at 1/100th the volume.
|
||||||
|
- **Catch onboarding friction.** A user who abandons mid-signup is the loudest signal the funnel is broken. Synthetic monitoring can't cry.
|
||||||
|
|
||||||
|
## Cohort
|
||||||
|
|
||||||
|
| Source | Size | Selection criterion |
|
||||||
|
| ------------------------------------------ | ---- | ------------------------------------------------------------------ |
|
||||||
|
| Pre-launch mailing list | _to fill_ | Subscribers who opted in via the landing page |
|
||||||
|
| Personal contacts of the team | _to fill_ | Friends who agreed to do >= 1 hour of testing |
|
||||||
|
| Selective music communities (Discord, FB) | _to fill_ | Communities the team admins or has explicit invitation in |
|
||||||
|
| **Total invited** | _to fill_ | |
|
||||||
|
|
||||||
|
The cohort SHOULD include : creators (test upload + publish + sell), listeners (test discovery + playback + library), at least one admin (test moderation + DMCA queue rendering). Skewing too creator-heavy means the listener path doesn't get exercised.
|
||||||
|
|
||||||
|
## Invitation flow
|
||||||
|
|
||||||
|
Send the invitation 24 h in advance ; gate the public link on a beta code so a forwarded invite doesn't accidentally open the floodgates.
|
||||||
|
|
||||||
|
### Email template
|
||||||
|
|
||||||
|
```
|
||||||
|
Subject : Veza beta — your invitation
|
||||||
|
|
||||||
|
Hi <first name>,
|
||||||
|
|
||||||
|
You're one of the first ~80 people getting early access to Veza, an
|
||||||
|
ethical music streaming + marketplace platform we've been building
|
||||||
|
this year. The public launch is tomorrow ; today we'd love your
|
||||||
|
feedback so we can fix anything that bites before the world arrives.
|
||||||
|
|
||||||
|
What we'd like you to try :
|
||||||
|
- Sign up at https://app.veza.fr/signup?beta=<one-shot-code>
|
||||||
|
- Listen to a few tracks ; ideally try the offline mode
|
||||||
|
- If you're a creator : upload a track + publish it
|
||||||
|
- If you're feeling generous : try the marketplace flow on the
|
||||||
|
seeded "Beta tester sample pack" (free)
|
||||||
|
- Note ANYTHING that surprised you : confusing copy, slow page,
|
||||||
|
visual bug, error message you didn't understand
|
||||||
|
|
||||||
|
Feedback form : https://typeform.com/<beta-feedback-form-id>
|
||||||
|
(~2 min, optional ; we'd love it)
|
||||||
|
|
||||||
|
Anything pressing : reply to this email — we're monitoring all day.
|
||||||
|
|
||||||
|
Thanks for the road test.
|
||||||
|
— The Veza team
|
||||||
|
```
|
||||||
|
|
||||||
|
Each invitation carries a unique beta code. Codes are single-use but tied to the email so the team knows who's exercising what. Generated via :
|
||||||
|
|
||||||
|
```bash
|
||||||
|
psql "$DATABASE_URL" -c "
|
||||||
|
INSERT INTO beta_invites (email, code, expires_at)
|
||||||
|
VALUES ('<email>', encode(gen_random_bytes(8), 'hex'), now() + interval '7 days')
|
||||||
|
RETURNING code;
|
||||||
|
"
|
||||||
|
```
|
||||||
|
|
||||||
|
(Schema : `migrations/990_beta_invites.sql` — to add if missing pre-launch.)
|
||||||
|
|
||||||
|
## Day timeline
|
||||||
|
|
||||||
|
The soft launch runs as one continuous "open dashboard" session for the full day. Roles below are full-day commitments ; rotate among the team if needed.
|
||||||
|
|
||||||
|
| Time (local) | Driver / observer focus |
|
||||||
|
| ------------ | ---------------------------------------------------------------------------- |
|
||||||
|
| T-24 h | Invitations sent (the day before, late-evening) |
|
||||||
|
| T-1 h | Final pre-flight : status-page green, Sentry quiet, Grafana dashboards open |
|
||||||
|
| **T+0** | **Beta opens.** First wave (~30 % of invitees) hits the signup page |
|
||||||
|
| T+1 h | First feedback batch reviewed ; triage table updated |
|
||||||
|
| T+3 h | Second wave processed ; mid-day check-in (#engineering) |
|
||||||
|
| T+5 h | Third wave + cumulative review |
|
||||||
|
| T+8 h | End-of-day triage sync : HIGH issue count fixed, MED queued for Day 30 AM |
|
||||||
|
| End-of-day | Decision call : GO / SLIP for Day 30 public launch |
|
||||||
|
|
||||||
|
## Real-time monitoring checklist
|
||||||
|
|
||||||
|
The driver keeps these tabs open continuously :
|
||||||
|
|
||||||
|
- [ ] **Status page** (`https://status.veza.fr` or `/api/v1/status`) — must stay all-green.
|
||||||
|
- [ ] **Grafana "Veza API Overview"** — req rate, p95 latency, 5xx rate. Watch for the request-rate ramp ; an out-of-pattern dip means something rejected onboarding before the signup form.
|
||||||
|
- [ ] **Grafana "Veza Service Map (Tempo)"** — slow spans on the 4 hot paths (auth.login, track.upload.initiate, payment.webhook, search.query).
|
||||||
|
- [ ] **Sentry frontend project** — JS errors. Filter for the 2026 release tag.
|
||||||
|
- [ ] **Sentry backend project** — Go panics + 5xx fingerprints.
|
||||||
|
- [ ] **Synthetic monitoring** (`Veza Service Map` dashboard) — blackbox probes still green.
|
||||||
|
- [ ] **Support inbox** — `support@veza.fr` ; triage incoming as the day goes.
|
||||||
|
- [ ] **Discord / Slack #beta-feedback** — channel for non-email reports.
|
||||||
|
- [ ] **Postgres `veza_db_pool_open_connections`** — must stay below the pool max (current 50). A spike means a slow query is holding connections.
|
||||||
|
- [ ] **Redis `veza_cache_*`** counters by subsystem — hit-rate stays stable.
|
||||||
|
- [ ] **HAProxy stats** — both backends UP, no DOWN events.
|
||||||
|
|
||||||
|
## Issue triage matrix
|
||||||
|
|
||||||
|
Triage is fast : every reported issue gets one row with one severity. The severity drives the SLA :
|
||||||
|
|
||||||
|
| Severity | Definition | SLA | Action |
|
||||||
|
| -------- | ------------------------------------------------------------------- | ---------------- | ------------------------------------------------------------------- |
|
||||||
|
| **HIGH** | Blocks a core flow (signup, login, playback, payment, upload) | Same-day fix | Fix → deploy via canary → notify reporter ; if not fixable today, the W6 GO/NO-GO row "0 HIGH issue ouverte" stays 🟡 PENDING and Day 30 slips |
|
||||||
|
| **MED** | Degrades UX but a workaround exists | Day 30 morning | Fix queued for Day 30 AM ; ship before public open |
|
||||||
|
| **LOW** | Cosmetic, polish, "nice to have" | Post-launch | Backlogged in Forgejo issues, labelled `beta-feedback` |
|
||||||
|
|
||||||
|
## Issues reported
|
||||||
|
|
||||||
|
Append rows as feedback comes in. Don't filter — every observation gets logged.
|
||||||
|
|
||||||
|
| # | Reported by | Time UTC | Description | Severity | Linked issue / PR | Status |
|
||||||
|
| -- | ----------------------- | -------- | ------------------------------------------------- | -------- | ---------------------------------- | ------------- |
|
||||||
|
| 1 | _user@example.com_ | _T+0:23_ | _signup form: tab order skips email→password_ | _MED_ | _#321_ | _open / fixed_ |
|
||||||
|
| 2 | … | | | | | |
|
||||||
|
|
||||||
|
## Feedback themes
|
||||||
|
|
||||||
|
Every ~3 issues, write a one-line summary of what's emerging. After the day, this table is the post-mortem input.
|
||||||
|
|
||||||
|
| Theme | Frequency | Action |
|
||||||
|
| ---------------------------------------------- | --------- | ----------------------------------------------------------- |
|
||||||
|
| _e.g. iOS Safari audio playback stutters_ | _N reports_ | _open Forgejo issue tagged ios-safari ; investigate Day 31_ |
|
||||||
|
|
||||||
|
## Acceptance gate (Day 29)
|
||||||
|
|
||||||
|
- [ ] **≥ 50 unique testers** signed up (count via `SELECT count(*) FROM users WHERE created_at > '<T+0>'`).
|
||||||
|
- [ ] **< 3 HIGH issues open** at end-of-day. (HIGH issues fixed during the day count as resolved if the fix is verified by the reporter or a teammate.)
|
||||||
|
- [ ] **Status page** green throughout the day. A ≥ 5-minute red event triggers a slip discussion.
|
||||||
|
- [ ] **No Sentry P1 events** (server panics, payment double-charge, data corruption, security alert).
|
||||||
|
- [ ] **Synthetic monitoring** stayed green continuously.
|
||||||
|
- [ ] **k6 nightly continued green** (the soft launch shouldn't push staging into red ; if it does, the canary on prod was sized wrong).
|
||||||
|
|
||||||
|
If any box is unchecked, the team has 1 h of grace at end-of-day to fix-or-decide. After that, the W6 GO/NO-GO checklist row "Soft launch beta : 50+ testeurs onboardés, < 3 HIGH issues, monitoring vert" stays 🟡 PENDING and Day 30 slips.
|
||||||
|
|
||||||
|
## Decision call (end-of-day)
|
||||||
|
|
||||||
|
- **Tech lead** : monitoring observed any signal that contradicts a public launch tomorrow ?
|
||||||
|
- **Product lead** : feedback themes reveal a critical UX bug we shouldn't ship over ?
|
||||||
|
- **On-call lead** : ready to take pages tomorrow ? Confident in the runbook coverage we exercised today ?
|
||||||
|
|
||||||
|
A unanimous GO promotes Day 30 to "public launch day". Any single NO-GO (with reason) slips the launch by ≥ 24 h.
|
||||||
|
|
||||||
|
## Linked artefacts
|
||||||
|
|
||||||
|
- `docs/GO_NO_GO_CHECKLIST_v2.0.0_PUBLIC.md` — Section 6 row this report unblocks
|
||||||
|
- `docs/RELEASE_NOTES_V2.0.0_RC1.md` — what's running on prod during the beta
|
||||||
|
- `docs/runbooks/game-days/2026-W6-game-day-2.md` — Day 28 prod-canary session that put the build in front of beta users
|
||||||
|
- `docs/PAYMENT_E2E_LIVE_REPORT.md` — Day 27 real-money test (creators on the beta validating the same flow at scale)
|
||||||
|
- `config/grafana/dashboards/api-overview.json` — main monitoring board
|
||||||
|
|
||||||
|
## Take-aways
|
||||||
|
|
||||||
|
_Free-form. After the day closes, write the 5-line summary that the team carries into Day 30 and beyond. What surprised us, what we'd change in the next beta, what graduated from "we'll see how that lands" to "we know exactly how that lands"._
|
||||||
Loading…
Reference in a new issue