code_public/ja4-platform
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(e2e): add distributed E2E test framework with parametric traffic generation

Browse Source 
Add run-e2e-test.sh with CLI parameters (--hits, --http-ratio, --dns, --tls,
--src-ips, --keep-analysis, --up) for configurable traffic generation. Traffic
runs from VM endpoints with multiple source IPs (alias IPs on eth0) to produce
distinct sessions for the ML pipeline. Fix curl TLS flags (--tlsv1.2 instead
of --tls-v1-2), skip redundant local verification in distributed mode, and
fix dashboard is_available() cache that never retried after ClickHouse recovery.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Jacquin Antoine
2026-04-15 00:09:32 +02:00
parent 7894d39f1c
commit f88b739992
40 changed files with 2154 additions and 337 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
docs/development.md
View File

@ -36,6 +36,15 @@ make build-dashboard # Image FastAPI + Jinja2
make test-all
```
### Tests E2E distribués (full stack : capture + ML + dashboard)
```bash
make e2e-up # Créer les 4 VMs (centos8/rocky9/rocky10/analysis)
make e2e-down # Détruire les VMs E2E
make test-e2e # Test E2E complet (capture + ML + dashboard)
make test-e2e-quick # Test E2E rapide (trafic réduit, 1 cycle bot-detector)
```
### Tests par service
| Service | Commande | Détails |
@ -122,7 +131,7 @@ uvicorn backend.main:app --reload --host 0.0.0.0 --port 8000
| Librairie | Usage |
|-----------|-------|
| `isotree` | Extended Isolation Forest (scoreur principal non supervisé) |
| `torch` | Autoencoder (PyTorch, architecture n->64->32->16->32->64->n) |
| `torch` | NFEnsemble / TrafficNormalizingFlow (PyTorch, Deep Ensemble M=5) |
| `xgboost` | Modèle supervisé (entraîné sur les labels SOC) |
| `hdbscan` | Clustering de campagnes d'attaque |
| `shap` | Explicabilité des scores d'anomalie |
@ -139,8 +148,8 @@ services/bot-detector/bot_detector/
├── log.py # Configuration du logging
├── infra.py # Connexion ClickHouse, health check
├── preprocessing.py # Feature engineering, filtrage, normalisation
├── models.py # EIF, Autoencoder, XGBoost (entraînement + scoring)
├── scoring.py # Ensemble triple voix, seuils adaptatifs
├── models.py # EIF, NFEnsemble/TrafficNormalizingFlow, XGBoost (entraînement + scoring)
├── scoring.py # Ensemble triple voix, seuils adaptatifs, ADWIN drift
├── browser.py # Détection de navigateur 5 axes multifactoriels
├── pipeline.py # Orchestration du cycle de détection
├── cycle.py # Boucle principale (cycle de 5 minutes)