code_public/ja4-platform
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
145 Commits 1 Branch 0 Tags
master
Commit Graph

3 Commits

Author SHA1 Message Date
Jacquin Antoine
36b5065a0a feat(e2e): add multi-IP endpoint architecture with dedicated traffic VM 
Replace single-service-per-endpoint with all-ips mode running nginx, apache,
and hitch+varnish simultaneously on 3 dedicated IPs per VM (eth1 alias IPs).
Add a dedicated traffic VM with curl-impersonate for realistic TLS fingerprints,
parallelized traffic generation, and paired SNI_HOSTS/TARGET_IPS lists for
per-VM per-service hostname identification (e.g. rocky9-nginx-platform.test).

Key changes:
- run-tests-vm.sh: add setup_all_ips(), IP-specific Listen/bind directives
  with reset-before-apply pattern, graceful service availability checks
- run-e2e-test.sh: traffic VM architecture, all-ips mode, eth1 network,
  paired IP/SNI lists, updated cleanup for alias IPs
- generate-traffic.sh: parallel background jobs, curl-impersonate detection,
  auto source interface detection via ip route get, Host header in HTTP traffic
- Vagrantfile: add traffic VM with provision-traffic.sh
- provision-traffic.sh: install curl-impersonate and httpx for traffic gen
- test-rpm.sh: multi-interface TC check, updated ja4ebpf config
- clickhouse-init.sh: load CSV stubs for Anubis/bot-networks dictionaries
- Remove obsolete correlator/sentinel/mod-reqin-log docs
- Add h2_settings_ack column to http_logs schema
- Upgrade Go toolchain to 1.25.0

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-04-16 14:25:24 +02:00
Jacquin Antoine
d75825278e feat: multi-distro VM tests, ja4ebpf eBPF improvements, bot-detector scoring 
ja4ebpf:
- Refactor BPF TC capture with improved SYN offset handling and TCP option parsing
- Enhance TLS uprobe SSL hooking for better key extraction
- Add ClickHouse writer improvements for HTTP log materialized views
- Update RPM spec for Rocky Linux 8/9/10, fix systemd service
- Simplify loader with cleaner bpf2go integration

bot-detector:
- Add H2 SETTINGS per-parameter comparison in browser_matcher
- Enhance browser signatures and scoring pipeline
- Improve preprocessing and cycle detection

infra:
- Multi-distro Vagrantfile (centos8, rocky9, rocky10) with per-distro provisioning
- New Makefile targets: vm-up-all, test-vm-matrix, test-vm-centos8/rocky10
- Add debug helpers and run-test-from-host.sh for host-driven VM testing
- Update run-tests-vm.sh for cross-distro compatibility
- Remove accidental binary blob (\004)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-04-13 01:09:33 +02:00
toto
957918c565 fix(ja4ebpf): Rocky Linux RPM builder, remove correlated field, fix thesis 
- Dockerfile.package: migre go-builder de golang:bookworm (Debian) vers
  rockylinux:9, installe Go depuis le tarball officiel, remplace apt par
  dnf (clang llvm libbpf-devel bpftool)

- Suppression du champ 'correlated' de l'agent ja4ebpf : avec eBPF/XDP,
  la corrélation L3/L4↔L7 est toujours implicite par présence des champs.
  Supprimé de : session.go, manager.go, main.go (x5), clickhouse.go

- Thèse (6 corrections listées + cohérence correlated) :
  1. §3.5 + §3.9.1 : SSL_read retourne des octets bruts sans respecter les
     frontières H2 → buffer circulaire de réassemblage en Go userspace
  2. §3.1 : supprimé libpcap + CAP_NET_RAW, remplacé par définition uprobe
  3. §4 + §7 : compte exact 96 features en 8 familles (Famille 1–8),
     supprimé taxonomie F1–F11 obsolète, tous les totaux mis à jour
  4. §2.4 + §8 : remplacé 7 fausses URLs arXiv par [Référence à vérifier]
  5. §4 Famille 2 : ja4_drift_ratio → renvoi à Famille 8 (définition complète)
  6. §6.4 : ajouté limite 'Overhead de l'uprobe SSL_read'
  + §3.6 : supprimé correlated=0/1 du texte architectural

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-04-12 04:48:40 +02:00