code_public/ja4-platform
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(e2e): add distributed E2E test framework with parametric traffic generation

Browse Source 
Add run-e2e-test.sh with CLI parameters (--hits, --http-ratio, --dns, --tls,
--src-ips, --keep-analysis, --up) for configurable traffic generation. Traffic
runs from VM endpoints with multiple source IPs (alias IPs on eth0) to produce
distinct sessions for the ML pipeline. Fix curl TLS flags (--tlsv1.2 instead
of --tls-v1-2), skip redundant local verification in distributed mode, and
fix dashboard is_available() cache that never retried after ClickHouse recovery.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Jacquin Antoine
2026-04-15 00:09:32 +02:00
parent 7894d39f1c
commit f88b739992
40 changed files with 2154 additions and 337 deletions
Download Patch File Download Diff File Expand all files Collapse all files

34
tests/vm/Vagrantfile vendored
View File

@ -54,6 +54,9 @@ Vagrant.configure("2") do |config|
# ═══════════════════════════════════════════════════════════════════════════
config.vm.define "centos8", autostart: false do |node|
node.vm.box = "centos/8"
node.vm.network "private_network",
libvirt__network_name: "ja4-e2e",
type: "dhcp"
node.vm.provision "shell", path: "provision-el8.sh"
node.vm.post_up_message = "VM centos8 prête ! Tests : make test-vm-centos8"
end
@ -63,6 +66,9 @@ Vagrant.configure("2") do |config|
# ═══════════════════════════════════════════════════════════════════════════
config.vm.define "rocky9", primary: true do |node|
node.vm.box = "generic/rocky9"
node.vm.network "private_network",
libvirt__network_name: "ja4-e2e",
type: "dhcp"
node.vm.provision "shell", path: "provision.sh"
node.vm.post_up_message = <<~MSG
VM rocky9 prête !
@ -80,8 +86,36 @@ Vagrant.configure("2") do |config|
# ═══════════════════════════════════════════════════════════════════════════
config.vm.define "rocky10", autostart: false do |node|
node.vm.box = "almalinux/10"
node.vm.network "private_network",
libvirt__network_name: "ja4-e2e",
type: "dhcp"
node.vm.provision "shell", path: "provision.sh"
node.vm.post_up_message = "VM rocky10 prête ! Tests : make test-vm-rocky10"
end
# ═══════════════════════════════════════════════════════════════════════════
# VM 4 : Analysis Server (ClickHouse + bot-detector + dashboard)
#
# VM centralisée pour le test E2E distribué. Les endpoints EL8/9/10 envoient
# leurs logs ja4ebpf vers le ClickHouse de cette VM (192.168.42.10).
# ═══════════════════════════════════════════════════════════════════════════
config.vm.define "analysis", autostart: false do |node|
node.vm.box = "generic/rocky9"
node.vm.network "private_network", ip: "192.168.42.10",
libvirt__network_name: "ja4-e2e",
libvirt__netmask: "255.255.255.0"
node.vm.provider :libvirt do |v|
v.cpus = 4
v.memory = 12288 # 12 Go — torch + isotree build gourmand en RAM
end
node.vm.provision "shell", path: "provision-analysis.sh"
node.vm.post_up_message = <<~MSG
VM analysis prête !
Depuis la racine du projet :
make test-e2e # test E2E complet (capture + ML + dashboard)
make test-e2e-quick # test rapide avec trafic réduit
MSG
end
end