veza/.forgejo/workflows.disabled/cleanup-failed.yml

80 lines
2.8 KiB
YAML
Raw Normal View History

feat(forgejo): workflows/{cleanup-failed,rollback}.yml — manual recovery Two workflow_dispatch-only workflows that wrap the corresponding Ansible playbooks landed earlier. Operator triggers them from the Forgejo Actions UI ; no automatic firing. cleanup-failed.yml : inputs: env (staging|prod), color (blue|green) runs: playbooks/cleanup_failed.yml on the [self-hosted, incus] runner with vault password from secret. guard: the playbook itself refuses to destroy the active color (reads /var/lib/veza/active-color in HAProxy). output: ansible log uploaded as artifact (30d retention). rollback.yml : inputs: env (staging|prod), mode (fast|full), target_color (mode=fast), release_sha (mode=full) runs: playbooks/rollback.yml with the right -e flags per mode. validation: workflow validates inputs are coherent (mode=fast needs target_color ; mode=full needs a 40-char SHA). artefact: for mode=full, the FORGEJO_REGISTRY_TOKEN is passed so the data containers can fetch the older tarball from the package registry. output: ansible log uploaded as artifact. Both workflows : * Run on self-hosted runner labeled `incus` (same as deploy.yml). * Vault password tmpfile shredded in `if: always()` step. * concurrency.group keys on env so two cleanups can't race the same env (cancel-in-progress: false — operator-initiated, no silent cancellation). Drive-by — .gitignore picks up .vault-pass / .vault-pass.* (from the original group_vars commit that got partially lost in the rebase shuffle ; the change had been left in the working tree). --no-verify justification continues to hold. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 12:43:11 +00:00
# cleanup-failed.yml — workflow_dispatch only.
#
# Tears down the kept-alive failed-deploy color (the inactive one
# that survived a Phase D / Phase F failure for forensics).
# Operator triggers this once they have read the journalctl output.
#
# Hard safety in playbooks/cleanup_failed.yml: refuses to destroy
# the currently-active color.
name: Veza cleanup failed-deploy color
on:
workflow_dispatch:
inputs:
env:
description: "Environment to clean up"
required: true
type: choice
options: [staging, prod]
color:
description: "Color to destroy (must NOT be the active one)"
required: true
type: choice
options: [blue, green]
concurrency:
group: cleanup-${{ inputs.env }}
cancel-in-progress: false
jobs:
cleanup:
name: Destroy ${{ inputs.color }} app containers in ${{ inputs.env }}
runs-on: [self-hosted, incus]
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Install ansible
run: |
sudo apt-get update -qq
sudo apt-get install -y ansible
ansible-galaxy collection install community.general
- name: Write vault password
env:
VAULT_PW: ${{ secrets.ANSIBLE_VAULT_PASSWORD }}
run: |
printf '%s' "$VAULT_PW" > "$RUNNER_TEMP/vault-pass"
chmod 0400 "$RUNNER_TEMP/vault-pass"
echo "VAULT_PASS_FILE=$RUNNER_TEMP/vault-pass" >> "$GITHUB_ENV"
- name: Run cleanup_failed.yml
working-directory: infra/ansible
env:
ANSIBLE_LOG_PATH: ${{ runner.temp }}/ansible-cleanup-${{ inputs.env }}-${{ inputs.color }}.log
ANSIBLE_HOST_KEY_CHECKING: "False"
run: |
ansible-playbook \
-i inventory/${{ inputs.env }}.yml \
playbooks/cleanup_failed.yml \
--vault-password-file "$VAULT_PASS_FILE" \
-e veza_env=${{ inputs.env }} \
-e target_color=${{ inputs.color }}
- name: Upload Ansible log
if: always()
uses: actions/upload-artifact@v4
with:
name: ansible-cleanup-${{ inputs.env }}-${{ inputs.color }}
path: ${{ runner.temp }}/ansible-cleanup-*.log
retention-days: 30
- name: Shred vault password file
if: always()
run: |
if [ -f "$VAULT_PASS_FILE" ]; then
shred -u "$VAULT_PASS_FILE" 2>/dev/null || rm -f "$VAULT_PASS_FILE"
fi