273 lines
6 KiB
Markdown
273 lines
6 KiB
Markdown
|
|
# Circuit Breakers — Documentation
|
||
|
|
|
||
|
|
**Date**: 2025-01-27
|
||
|
|
**Status**: ✅ **IMPLEMENTED** - MOD-P2-007
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Vue d'ensemble
|
||
|
|
|
||
|
|
Les circuit breakers protègent l'application contre les dépendances externes lentes ou indisponibles en interrompant automatiquement les appels après un seuil d'échecs.
|
||
|
|
|
||
|
|
### Implémentation
|
||
|
|
|
||
|
|
- **Bibliothèque**: `github.com/sony/gobreaker`
|
||
|
|
- **Wrapper**: `internal/services/circuit_breaker.go`
|
||
|
|
- **Métriques**: `internal/metrics/circuit_breaker.go`
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Configuration
|
||
|
|
|
||
|
|
### Paramètres par défaut
|
||
|
|
|
||
|
|
```go
|
||
|
|
MaxRequests: 3 // Requêtes simultanées max
|
||
|
|
Interval: 60s // Réinitialisation des compteurs
|
||
|
|
Timeout: 30s // Délai avant half-open
|
||
|
|
ReadyToTrip: 5 échecs // Seuil pour ouvrir le circuit
|
||
|
|
```
|
||
|
|
|
||
|
|
### États du Circuit Breaker
|
||
|
|
|
||
|
|
1. **Closed** (Fermé): État normal, toutes les requêtes passent
|
||
|
|
2. **Open** (Ouvert): Circuit ouvert après 5 échecs consécutifs, requêtes rejetées
|
||
|
|
3. **Half-Open** (Demi-ouvert): Après 30s, permet quelques requêtes de test
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Utilisation
|
||
|
|
|
||
|
|
### Création d'un client avec circuit breaker
|
||
|
|
|
||
|
|
```go
|
||
|
|
import (
|
||
|
|
"veza-backend-api/internal/services"
|
||
|
|
"go.uber.org/zap"
|
||
|
|
)
|
||
|
|
|
||
|
|
logger := zap.NewNop()
|
||
|
|
httpClient := &http.Client{Timeout: 10 * time.Second}
|
||
|
|
cbClient := services.NewCircuitBreakerHTTPClient(
|
||
|
|
httpClient,
|
||
|
|
"my-service", // Nom du circuit breaker (pour métriques)
|
||
|
|
logger,
|
||
|
|
)
|
||
|
|
```
|
||
|
|
|
||
|
|
### Exécution d'une requête
|
||
|
|
|
||
|
|
```go
|
||
|
|
req, _ := http.NewRequest("GET", "https://api.example.com/data", nil)
|
||
|
|
resp, err := cbClient.Do(req)
|
||
|
|
if err != nil {
|
||
|
|
// Gérer l'erreur (circuit ouvert, timeout, 5xx, etc.)
|
||
|
|
return err
|
||
|
|
}
|
||
|
|
defer resp.Body.Close()
|
||
|
|
```
|
||
|
|
|
||
|
|
### Avec contexte (timeout/cancellation)
|
||
|
|
|
||
|
|
```go
|
||
|
|
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||
|
|
defer cancel()
|
||
|
|
|
||
|
|
req, _ := http.NewRequest("GET", "https://api.example.com/data", nil)
|
||
|
|
resp, err := cbClient.DoWithContext(ctx, req)
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Services Intégrés
|
||
|
|
|
||
|
|
### 1. Stream Service
|
||
|
|
|
||
|
|
**Fichier**: `internal/services/stream_service.go`
|
||
|
|
|
||
|
|
```go
|
||
|
|
circuitBreaker: NewCircuitBreakerHTTPClient(
|
||
|
|
httpClient,
|
||
|
|
"stream-service",
|
||
|
|
logger,
|
||
|
|
)
|
||
|
|
```
|
||
|
|
|
||
|
|
**Utilisation**: Appels HTTP vers le serveur de streaming pour transcodage.
|
||
|
|
|
||
|
|
### 2. OAuth Service
|
||
|
|
|
||
|
|
**Fichier**: `internal/services/oauth_service.go`
|
||
|
|
|
||
|
|
```go
|
||
|
|
circuitBreaker: NewCircuitBreakerHTTPClient(
|
||
|
|
httpClient,
|
||
|
|
"oauth-service",
|
||
|
|
logger,
|
||
|
|
)
|
||
|
|
```
|
||
|
|
|
||
|
|
**Utilisation**: Appels HTTP vers les providers OAuth (Google, GitHub, Discord).
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Métriques Prometheus
|
||
|
|
|
||
|
|
Les métriques suivantes sont exposées automatiquement:
|
||
|
|
|
||
|
|
### `veza_circuit_breaker_state`
|
||
|
|
**Type**: Gauge
|
||
|
|
**Labels**: `circuit_breaker_name`
|
||
|
|
**Valeurs**:
|
||
|
|
- `0` = Closed
|
||
|
|
- `1` = Half-Open
|
||
|
|
- `2` = Open
|
||
|
|
|
||
|
|
**Exemple**:
|
||
|
|
```
|
||
|
|
veza_circuit_breaker_state{circuit_breaker_name="stream-service"} 0
|
||
|
|
```
|
||
|
|
|
||
|
|
### `veza_circuit_breaker_requests_total`
|
||
|
|
**Type**: Counter
|
||
|
|
**Labels**: `circuit_breaker_name`, `result` (success|failure|rejected)
|
||
|
|
|
||
|
|
**Exemple**:
|
||
|
|
```
|
||
|
|
veza_circuit_breaker_requests_total{circuit_breaker_name="stream-service",result="success"} 150
|
||
|
|
veza_circuit_breaker_requests_total{circuit_breaker_name="stream-service",result="failure"} 5
|
||
|
|
veza_circuit_breaker_requests_total{circuit_breaker_name="stream-service",result="rejected"} 2
|
||
|
|
```
|
||
|
|
|
||
|
|
### `veza_circuit_breaker_failures_total`
|
||
|
|
**Type**: Counter
|
||
|
|
**Labels**: `circuit_breaker_name`
|
||
|
|
|
||
|
|
**Exemple**:
|
||
|
|
```
|
||
|
|
veza_circuit_breaker_failures_total{circuit_breaker_name="stream-service"} 5
|
||
|
|
```
|
||
|
|
|
||
|
|
### `veza_circuit_breaker_consecutive_failures`
|
||
|
|
**Type**: Gauge
|
||
|
|
**Labels**: `circuit_breaker_name`
|
||
|
|
|
||
|
|
**Exemple**:
|
||
|
|
```
|
||
|
|
veza_circuit_breaker_consecutive_failures{circuit_breaker_name="stream-service"} 3
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Comportement sur Erreurs
|
||
|
|
|
||
|
|
### Codes HTTP 5xx
|
||
|
|
|
||
|
|
Les codes HTTP 5xx (500, 502, 503, etc.) sont considérés comme des **échecs** et comptent pour le circuit breaker:
|
||
|
|
|
||
|
|
```go
|
||
|
|
if resp.StatusCode >= 500 {
|
||
|
|
resp.Body.Close()
|
||
|
|
return nil, fmt.Errorf("server error: %d", resp.StatusCode)
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
|
### Circuit Ouvert
|
||
|
|
|
||
|
|
Quand le circuit est ouvert, les requêtes sont **rejetées immédiatement** sans appel HTTP:
|
||
|
|
|
||
|
|
```go
|
||
|
|
if err == gobreaker.ErrOpenState {
|
||
|
|
return nil, fmt.Errorf("circuit breaker is open: service unavailable")
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Tests
|
||
|
|
|
||
|
|
### Tests unitaires
|
||
|
|
|
||
|
|
```bash
|
||
|
|
go test ./internal/services -v -run TestCircuitBreaker
|
||
|
|
```
|
||
|
|
|
||
|
|
**Tests inclus**:
|
||
|
|
- Création du client
|
||
|
|
- Requêtes réussies
|
||
|
|
- Gestion des erreurs 5xx
|
||
|
|
- Ouverture du circuit après seuil
|
||
|
|
- Rejet de requêtes quand circuit ouvert
|
||
|
|
- Support du contexte (timeout/cancellation)
|
||
|
|
|
||
|
|
### Test d'intégration (mock server)
|
||
|
|
|
||
|
|
Un test simule un serveur qui retourne 5xx pour déclencher l'ouverture du circuit:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
go test ./internal/services -v -run TestCircuitBreakerHTTPClient_Do_ServerError
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Variables d'Environnement
|
||
|
|
|
||
|
|
Aucune variable d'environnement requise. La configuration est codée en dur dans le wrapper pour simplifier.
|
||
|
|
|
||
|
|
**Pour personnaliser** (si nécessaire):
|
||
|
|
- Modifier `internal/services/circuit_breaker.go`
|
||
|
|
- Ajuster `MaxRequests`, `Interval`, `Timeout`, `ReadyToTrip`
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Monitoring et Alertes
|
||
|
|
|
||
|
|
### Alertes recommandées
|
||
|
|
|
||
|
|
1. **Circuit ouvert trop souvent**:
|
||
|
|
```
|
||
|
|
veza_circuit_breaker_state{circuit_breaker_name="stream-service"} == 2
|
||
|
|
```
|
||
|
|
|
||
|
|
2. **Taux d'échec élevé**:
|
||
|
|
```
|
||
|
|
rate(veza_circuit_breaker_requests_total{result="failure"}[5m]) > 0.1
|
||
|
|
```
|
||
|
|
|
||
|
|
3. **Échecs consécutifs**:
|
||
|
|
```
|
||
|
|
veza_circuit_breaker_consecutive_failures > 3
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Dépannage
|
||
|
|
|
||
|
|
### Circuit reste ouvert
|
||
|
|
|
||
|
|
**Cause**: Service externe toujours en erreur
|
||
|
|
**Solution**: Vérifier la santé du service externe, attendre 30s (Timeout) pour half-open
|
||
|
|
|
||
|
|
### Trop de rejets
|
||
|
|
|
||
|
|
**Cause**: Seuil trop bas (5 échecs)
|
||
|
|
**Solution**: Augmenter `ReadyToTrip` dans `circuit_breaker.go`
|
||
|
|
|
||
|
|
### Métriques manquantes
|
||
|
|
|
||
|
|
**Cause**: Métriques non initialisées
|
||
|
|
**Solution**: Vérifier que `internal/metrics/circuit_breaker.go` est importé
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Références
|
||
|
|
|
||
|
|
- [gobreaker Documentation](https://github.com/sony/gobreaker)
|
||
|
|
- [Circuit Breaker Pattern](https://martinfowler.com/bliki/CircuitBreaker.html)
|
||
|
|
- [Prometheus Metrics](https://prometheus.io/docs/concepts/metric_types/)
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
**Dernière mise à jour**: 2025-01-27
|
||
|
|
**Maintenu par**: Veza Backend Team
|