feat(ja4ebpf): add multi-interface TC, LPM_TRIE ignore_src, unit tests, and fix bugs
- Add multi-interface TC attachment (default "any" = all UP interfaces) - Add BPF LPM_TRIE map ignored_src for kernel-side CIDR filtering - Add userspace ignore_src filtering for SSL/accept4 path via net.IPNet.Contains() - Add AcceptCache for fd→SessionKey correlation with TTL and Close() - Add 5 test files covering writer, procutil, dispatcher, accept_cache, and cmd - Fix formatTCPOptions infinite loop on EOL (case 0 break→return) - Fix pseudoOrderToShort panic on empty slice (negative cap) - Fix AcceptCache goroutine leak (add done channel + Close()) - Update config.yml.example with interfaces, listen_ports, ignore_src - Rewrite docs/services/ja4ebpf.md (was massively stale: XDP, RingBuffer, etc.) - Fix stale XDP/RingBuffer references in docs/architecture.md, thesis, tls.go Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@ -68,13 +68,13 @@ INSERT (Native TCP :9000)
|
||||
|
||||
1. **ja4ebpf TC ingress hook** capture les paquets réseau bruts. Pour chaque TCP SYN : `src_ip`, `src_port`, `ttl`, `window_size`, `mss`, `window_scale`, `df_bit`. Pour chaque TLS ClientHello : décodage du payload pour extraire `ciphers`, `extensions`, `elliptic_curves`, `alpn`, `sni` (calcul du hash JA4/JA3 en espace utilisateur Go).
|
||||
|
||||
2. **ja4ebpf uprobe SSL_read** accroche `SSL_read` dans la bibliothèque OpenSSL/BoringSSL du serveur web. Les données déchiffrées sont écrites dans un RingBuffer eBPF. Un kprobe sur `accept4` fournit la correspondance `fd → src_ip:src_port` pour annoter chaque buffer L7.
|
||||
2. **ja4ebpf uprobes SSL_read/SSL_write** s’attachent à `SSL_read` et `SSL_write` dans la bibliothèque OpenSSL/BoringSSL du serveur web. Les données déchiffrées sont écrites dans un PerfEventArray eBPF. Des tracepoints sur `accept4` fournissent la correspondance `fd → src_ip:src_port` pour annoter chaque buffer L7.
|
||||
|
||||
3. **ja4ebpf kprobe tcp_recvmsg** (HTTP port 80/8080) intercepte le payload TCP avant consommation par le serveur pour les connexions non chiffrées.
|
||||
3. **ja4ebpf TC ingress HTTP plain** (port 80/8080) capture les payloads TCP en clair directement depuis le hook TC ingress pour les connexions non chiffrées.
|
||||
|
||||
### Phase 2 — Corrélation en mémoire
|
||||
|
||||
4. **ja4ebpf 256-shard manager** (espace utilisateur Go) consomme les trois RingBuffers eBPF via des goroutines dédiées. Les événements L3/L4/L5 et L7 sont corrélés par `src_ip:src_port` dans une table de sessions shardée (256 shards, mutex par shard). Timeout orphelin : 500 ms (émission avec `correlated=0`). Détection Slowloris : émission partielle après 10 s. GC des sessions fantômes : toutes les 100 ms. Le dispatcher magic bytes route vers le parser HTTP/1.1 ou HTTP/2. Pour HTTP/2, la première frame SETTINGS + WINDOW_UPDATE est décodée pour le fingerprinting passif. L’objet corrélé est inséré dans **`ja4_logs.http_logs_raw`** par batch.
|
||||
4. **ja4ebpf 256-shard manager** (espace utilisateur Go) consomme les cinq PerfEventArray eBPF via des goroutines dédiées. Les événements L3/L4/L5 et L7 sont corrélés par `src_ip:src_port` dans une table de sessions shardée (256 shards, mutex par shard). Timeout orphelin : 500 ms (émission avec `correlated=0`). Détection Slowloris : émission partielle après 10 s. GC des sessions fantômes : toutes les 100 ms. Le dispatcher magic bytes route vers le parser HTTP/1.1 ou HTTP/2. Pour HTTP/2, la première frame SETTINGS + WINDOW_UPDATE est décodée pour le fingerprinting passif. L’objet corrélé est inséré dans **`ja4_logs.http_logs_raw`** par batch.
|
||||
|
||||
### Phase 3 — Enrichissement### Phase 3 — Enrichissement (ClickHouse)
|
||||
|
||||
@ -183,7 +183,7 @@ INSERT (Native TCP :9000)
|
||||
|
||||
## Algorithme de corrélation
|
||||
|
||||
**ja4ebpf** corrèle les événements L3/L4/L5 (RingBuffer TC ingress) avec les événements L7 (RingBuffer uprobe) via une table de sessions en mémoire :
|
||||
**ja4ebpf** corrèle les événements L3/L4/L5 (PerfEventArray TC ingress) avec les événements L7 (PerfEventArray uprobe) via une table de sessions en mémoire :
|
||||
|
||||
1. **Clé** : `src_ip + src_port` — l’IP source et le port éphémère identifient une connexion TCP de manière unique.
|
||||
2. **Sharding** : 256 shards (`src_port % 256`), chacun protégé par un `sync.Mutex`. Réduit la contention sous fort trafic.
|
||||
@ -274,7 +274,7 @@ Les deux empreintes sont générées par **ja4ebpf** (espace utilisateur Go) à
|
||||
| Frontend dashboard | htmx + Chart.js + ECharts + Tailwind CSS (CDN) |
|
||||
| Magasin de données | ClickHouse 24.8 (dual-database) |
|
||||
| Déploiement | systemd, Docker, RPM (Rocky 8/9/10) |
|
||||
| IPC | RingBuffers eBPF (kernel → userspace Go) |
|
||||
| IPC | PerfEventArray eBPF (kernel → userspace Go) |
|
||||
| Workspace Go | `go.work` (Go 1.24.6) |
|
||||
|
||||
## Fichiers de schéma SQL (13)
|
||||
|
||||
@ -4,36 +4,36 @@
|
||||
|
||||
`ja4ebpf` est l'agent de collecte de données de la plateforme ja4-platform. C'est un binaire Go unique qui utilise eBPF CO-RE (Compile Once — Run Everywhere) pour observer passivement le trafic réseau d'un serveur Linux, sans modifier ni interrompre le serveur web cible (Apache, Nginx, Varnish, HAProxy, ou tout processus utilisant OpenSSL/BoringSSL).
|
||||
|
||||
Il capture simultanément les métadonnées réseau L3/L4 (TCP SYN), les paramètres TLS L5 (ClientHello), et le contenu applicatif L7 (requêtes HTTP déchiffrées), corr le tout en mémoire par la clé `src_ip:src_port`, et envoie les données vers ClickHouse en batch.
|
||||
Il capture simultanément les métadonnées réseau L3/L4 (TCP SYN), les paramètres TLS L5 (ClientHello), et le contenu applicatif L7 (requêtes HTTP déchiffrées), corrèle le tout en mémoire par la clé `src_ip:src_port`, et envoie les données vers ClickHouse en batch.
|
||||
|
||||
## Architecture interne
|
||||
|
||||
```
|
||||
+-----------------+ TC ingress (XDP/TC) +--------------------+
|
||||
| réseau entrant |-------------------------->| bpf/tc_capture.c |
|
||||
+-----------------+ TC ingress +--------------------+
|
||||
| réseau entrant |---------------------------->| bpf/tc_capture.c |
|
||||
| | | |
|
||||
| SYN packet | --> rb_tcp_syn (16 MB) | Programme eBPF |
|
||||
| TLS ClientHello| --> rb_tls_hello (16 MB) | CO-RE |
|
||||
| HTTP port 80 | --> rb_http_plain (32 MB)| |
|
||||
| SYN packet | --> pb_tcp_syn (perf) | Programme eBPF |
|
||||
| TLS ClientHello| --> pb_tls_hello (perf) | CO-RE |
|
||||
| HTTP port 80 | --> pb_http_plain (perf) | |
|
||||
+-----------------+ +--------------------+
|
||||
|
|
||||
+-----------------+ uprobe SSL_read +--------------------+
|
||||
| serveur web |--------------------------> | bpf/uprobe_ssl.c |
|
||||
+-----------------+ uprobe SSL_read/SSL_write +--------------------+
|
||||
| serveur web |---------------------------> | bpf/uprobe_ssl.c |
|
||||
| (OpenSSL) | | |
|
||||
| flux déchiffré | --> rb_ssl_data (64 MB) | Programme eBPF |
|
||||
+-----------------+ | CO-RE |
|
||||
+--------------------+
|
||||
| flux déchiffré | --> pb_ssl_data (perf) | Programme eBPF |
|
||||
| accept4 events | --> pb_accept (perf) | CO-RE |
|
||||
+-----------------+ +--------------------+
|
||||
|
|
||||
+-----------v-----------+
|
||||
| Go userspace |
|
||||
| |
|
||||
| internal/loader/ |
|
||||
| - RingBuffer readers |
|
||||
| - PerfEvent readers |
|
||||
| - 5 goroutines |
|
||||
| |
|
||||
| internal/parser/ |
|
||||
| - JA4 calculator |
|
||||
| - H2 preface parser |
|
||||
| - JA4/JA3 calculator |
|
||||
| - H2ConnState (HPACK) |
|
||||
| - HTTP/1.1 parser |
|
||||
| |
|
||||
| internal/dispatcher/ |
|
||||
@ -41,6 +41,7 @@ Il capture simultanément les métadonnées réseau L3/L4 (TCP SYN), les paramè
|
||||
| |
|
||||
| internal/correlation/ |
|
||||
| - 256-shard manager |
|
||||
| - AcceptCache |
|
||||
| - GC 100ms |
|
||||
| - timeout 500ms |
|
||||
| |
|
||||
@ -61,32 +62,65 @@ Il capture simultanément les métadonnées réseau L3/L4 (TCP SYN), les paramè
|
||||
|
||||
### TC ingress — Couches L3/L4/L5
|
||||
|
||||
Le programme `bpf/tc_capture.c` est attaché à l'interface réseau via **TC (Traffic Control)** en ingress. Il s'exécute pour chaque paquet entrant :
|
||||
Le programme `bpf/tc_capture.c` est attaché aux interfaces réseau via **TC (Traffic Control)** en ingress (clsact qdisc). Par défaut, il s'attache sur **toutes les interfaces UP** (sauf loopback). Il s'exécute pour chaque paquet entrant :
|
||||
|
||||
**Filtrage kernel-side** :
|
||||
- **Ports autorisés** : la map `allowed_ports` (HASH) filtre par port destination/source. Seuls les ports configurés (défaut : 80, 443) sont traités.
|
||||
- **IP/CIDR ignorés** : la map `ignored_src` (LPM_TRIE) ignore le trafic provenant des CIDR configurés. Le lookup utilise le format `{prefixlen, data[4]}` en network byte order.
|
||||
|
||||
**Paquets TCP SYN** : extraction des options TCP et métadonnées IP depuis les headers du paquet.
|
||||
- `bpf_skb_load_bytes()` pour lire les options TCP depuis le skb
|
||||
- Envoyé dans le RingBuffer `rb_tcp_syn` (16 MB)
|
||||
- Émis via `pb_tcp_syn` (PerfEventArray)
|
||||
|
||||
**ClientHello TLS** : détection du type 0x16 (Handshake) et sous-type 0x01 (ClientHello).
|
||||
- `bpf_skb_load_bytes()` avec tailles en cascade (512 → 256 → 128) pour capturer SNI et extensions
|
||||
- La taille réellement copiée est stockée dans `payload_len`
|
||||
- Envoyé dans le RingBuffer `rb_tls_hello` (16 MB)
|
||||
- `bpf_skb_load_bytes()` avec tailles en cascade (1024 → 512 → 256) pour capturer SNI et extensions
|
||||
- Le struct `tls_hello_event` place le payload à l'offset 0 (compatible kernel 4.18)
|
||||
- Émis via `pb_tls_hello` (PerfEventArray)
|
||||
|
||||
**HTTP en clair (port 80/8080)** : pour les connexions non chiffrées.
|
||||
- SYN/FIN/RST exclus (uniquement les segments porteurs de données)
|
||||
- `bpf_skb_load_bytes()` avec tailles en cascade (256 → 128 → 64)
|
||||
- La taille réellement copiée est stockée dans `payload_len`
|
||||
- Envoyé dans le RingBuffer `rb_http_plain` (32 MB)
|
||||
- `bpf_skb_load_bytes()` avec tailles en cascade (512 → 256 → 128 → 64)
|
||||
- Le struct `http_plain_event` place le payload à l'offset 0
|
||||
- Émis via `pb_http_plain` (PerfEventArray)
|
||||
|
||||
### Uprobe SSL_read — Couche L7
|
||||
### Uprobes SSL_read/SSL_write — Couche L7
|
||||
|
||||
Les **uprobes** s'attachent dynamiquement à `SSL_read` (ou `SSL_read_ex`) dans le processus du serveur web. Elles s'exécutent *après* le déchiffrement TLS, capturant le flux HTTP en clair dans `rb_ssl_data` (64 MB).
|
||||
Les uprobes s'attachent dynamiquement aux fonctions OpenSSL dans `libssl.so` :
|
||||
|
||||
La clé de corrélation `src_ip:src_port` est extraite depuis la structure `SSL*` → file descriptor → socket noyau via `bpf_probe_read_kernel()`.
|
||||
| Hook | Type | Rôle |
|
||||
|------|------|------|
|
||||
| `SSL_set_fd` | uprobe | Associe `SSL*` → `fd`, peuple `ssl_conn_map` |
|
||||
| `SSL_read` | uprobe + uretprobe | Capture les requêtes du client (direction=0) |
|
||||
| `SSL_write` | uprobe + uretprobe | Capture les réponses du serveur (direction=1) |
|
||||
|
||||
### Kprobe accept4
|
||||
**Corrélation `fd → src_ip:src_port`** (3 niveaux de priorité) :
|
||||
1. `ssl_conn_map[ssl_ptr]` — si `SSL_set_fd` a été appelé et que `fd_conn_map[fd]` contient l'IP (via accept4)
|
||||
2. `accept_map[{pid_tgid, fd}]` — cache accept4 (tracepoint kernel)
|
||||
3. Fallback `/proc/<pid>/net/tcp` — lecture depuis l'espace utilisateur (moins fiable)
|
||||
|
||||
Un kprobe sur `accept4` peuple `rb_accept` (4 MB) avec le tuple `(src_ip, src_port, fd, pid_tgid)`, permettant d'associer chaque fd SSL à une connexion TCP.
|
||||
Les données sont émises via `pb_ssl_data` (PerfEventArray) avec un flag `direction` (0=client→serveur, 1=serveur→client).
|
||||
|
||||
### Tracepoints accept4
|
||||
|
||||
Les tracepoints `sys_enter_accept4` / `sys_exit_accept4` (plus stables que les kprobes) capturent le tuple `(src_ip, src_port, fd, pid_tgid)` pour chaque nouvelle connexion acceptée. Ils peuplent :
|
||||
- `accept_map[{pid_tgid, fd}]` — pour corrélation SSL côté BPF
|
||||
- `fd_conn_map[fd]` — pour SSL_set_fd
|
||||
- `pb_accept` (PerfEventArray) — pour corrélation SSL côté Go (AcceptCache)
|
||||
|
||||
### Maps eBPF résumé
|
||||
|
||||
| Map | Type | Rôle |
|
||||
|-----|------|------|
|
||||
| `allowed_ports` | HASH (key=u16, val=u8) | Ports TCP autorisés (peuplée depuis Go) |
|
||||
| `ignored_src` | LPM_TRIE (key={prefixlen, data[4]}, val=u8) | CIDR/IP sources à ignorer (peuplée depuis Go) |
|
||||
| `tc_stats` | PERCPU_ARRAY (7 compteurs) | Statistiques de debug BPF |
|
||||
| `ssl_conn_map` | HASH (key=ssl_ptr, val=ssl_conn_info) | Association SSL* → fd + IP |
|
||||
| `fd_conn_map` | HASH (key=fd, val=ssl_conn_info) | Association fd → IP (depuis accept4) |
|
||||
| `accept_map` | HASH (key={pid_tgid,fd}, val=accept_event) | Cache accept4 côté BPF |
|
||||
| `ssl_args_map` | HASH (key=pid_tgid, val=ssl_read_args) | Sauvegarde arguments SSL_read/Write entry |
|
||||
| `__tls_buf` | PERCPU_ARRAY (1 entrée) | Buffer temp > 512o (stack eBPF limit) |
|
||||
| `__http_buf` | PERCPU_ARRAY (1 entrée) | Buffer temp HTTP plain |
|
||||
| `__ssl_buf` | PERCPU_ARRAY (1 entrée) | Buffer temp SSL data |
|
||||
|
||||
## Corrélation in-memory
|
||||
|
||||
@ -94,11 +128,23 @@ Le gestionnaire `internal/correlation` maintient un état par connexion :
|
||||
|
||||
| Mécanisme | Valeur |
|
||||
|-----------|--------|
|
||||
| Sharding | 256 buckets (hash src_ip:src_port) |
|
||||
| Timeout orphelin | 500 ms (→ `correlated=0`) |
|
||||
| Sharding | 256 buckets (hash XOR src_ip ^ src_port) |
|
||||
| Timeout orphelin | 500 ms (→ session exportée vers ClickHouse) |
|
||||
| Détection Slowloris | 10 s (export partiel) |
|
||||
| GC interval | 100 ms |
|
||||
| Keep-Alive | max_keepalives incrémenté par requête |
|
||||
| Keep-Alive | `len(Requests)` incrémenté par requête |
|
||||
|
||||
### AcceptCache
|
||||
|
||||
Le `AcceptCache` map `{tgid, fd} → SessionKey + dstIP + dstPort` avec TTL 10s. Il est peuplé par les événements `accept4` et consulté par le consommateur SSL quand `ssl_conn_map` a `src_ip=0`. Purge automatique toutes les 30s.
|
||||
|
||||
### FDCache
|
||||
|
||||
Le `FDCache` résout `fd → IP:port` via `/proc/<pid>/net/tcp` (fallback quand accept4 n'est pas disponible). TTL 5s. Utilise `parseHexIPv4` / `parseHexIPv6` pour décoder le format little-endian du kernel.
|
||||
|
||||
### Filtrage ignore_src (userspace)
|
||||
|
||||
En plus du filtrage BPF (LPM_TRIE côté kernel), les 5 goroutines de consommation appliquent un filtrage userspace via `isIgnoredIP()` + `net.IPNet.Contains()`. Cela couvre le chemin SSL (SSL_read/SSL_write/accept4) où le BPF LPM_TRIE ne peut pas filtrer (l'IP vient de `ssl_conn_map` ou `/proc`, pas du paquet réseau).
|
||||
|
||||
### Routeur Magic Bytes (dispatcher)
|
||||
|
||||
@ -114,10 +160,25 @@ Buffer reçu (SSL data ou HTTP plain)
|
||||
+-- autre --> ProtoUnknown (ignoré)
|
||||
```
|
||||
|
||||
**Fingerprinting HTTP/2 passif** : après détection du preface `PRI * HTTP/2.0...`, le parser itère les frames :
|
||||
- Frame `SETTINGS` (type `0x04`) : extraction des 7 paramètres, valeur `-1` si absent
|
||||
- Frame `WINDOW_UPDATE` (type `0x08`, stream 0) : incrément de fenêtre connexion
|
||||
- Frame `HEADERS` (type `0x01`) : ordre des pseudo-headers (`m,a,s,p`)
|
||||
### Parsing HTTP/2 — H2ConnState
|
||||
|
||||
Le `H2ConnState` (package `internal/parser`) maintient un décodeur HPACK par-connexion avec table dynamique. Il traite les frames via `golang.org/x/net/http2.Framer` :
|
||||
|
||||
| Frame | Action |
|
||||
|-------|--------|
|
||||
| SETTINGS | Extraction des 7 paramètres (+ paramètre 0x7 JA4H2). Mise à jour de la taille de table HPACK. |
|
||||
| WINDOW_UPDATE (stream 0) | Capture de l'incrément de fenêtre connexion |
|
||||
| HEADERS + CONTINUATION | Assemblage des fragments, décodage HPACK, extraction pseudo-headers ordre |
|
||||
| DATA | Comptage des octets par stream |
|
||||
| PRIORITY | Capture des paramètres de priorité (StreamDep, Exclusive, Weight) |
|
||||
| RST_STREAM | Transition du stream vers "closed" |
|
||||
| GOAWAY | Capture LastStreamID + ErrCode |
|
||||
| PING | Détection ACK |
|
||||
|
||||
**Fingerprinting Akamai** :
|
||||
- `h2_fingerprint` : `SETTINGS[pairs]|WINDOW_UPDATE[value]|PRIORITY[0/1]|PSEUDO_ORDER[order]`
|
||||
- `h2_settings_fp` : liste brute des paramètres SETTINGS
|
||||
- `h2_pseudo_order` : notation abrégée (ex: `m,a,s,p` pour `:method, :authority, :scheme, :path`)
|
||||
|
||||
## Données collectées
|
||||
|
||||
@ -126,51 +187,53 @@ Buffer reçu (SSL data ou HTTP plain)
|
||||
| Champ | Description |
|
||||
|-------|-------------|
|
||||
| `src_ip`, `src_port` | Clé de corrélation |
|
||||
| `dst_ip`, `dst_port` | Destination IP et port (extrait du SYN) |
|
||||
| `dst_ip`, `dst_port` | Destination IP et port |
|
||||
| `ttl` | Time To Live initial |
|
||||
| `df_bit` | Don't Fragment bit |
|
||||
| `ip_id` | IP Identification (0 = Linux/VPN/spoofé) |
|
||||
| `ip_total_length` | Longueur totale IP (octets) |
|
||||
| `window_size` | Taille fenêtre TCP SYN |
|
||||
| `window_scale` | Option Window Scale (RFC 1323) |
|
||||
| `mss` | Maximum Segment Size |
|
||||
| `tcp_options` | Options TCP brutes (40 octets max) |
|
||||
| `tcp_jitter_variance` | Variance jitter inter-SYN |
|
||||
| `syn_timing_cv` | Délai SYN→ClientHello (ns) |
|
||||
| `window_scale` | Option Window Scale (RFC 1323), 0xFF = absent |
|
||||
| `mss` | Maximum Segment Size, 0 = absent |
|
||||
| `tcp_options` | Options TCP brutes (40 octets max), noms abrégés (MSS, WS, SACK, TS, NOP) |
|
||||
|
||||
### L5 (TLS ClientHello)
|
||||
|
||||
| Champ | Description |
|
||||
|-------|-------------|
|
||||
| `tls_version` | Version TLS la plus haute annoncée (extrait des SupportedVersions) |
|
||||
| `ciphers` | Liste suites cryptographiques |
|
||||
| `extensions` | Liste extensions TLS |
|
||||
| `elliptic_curves` | Courbes elliptiques supportées |
|
||||
| `point_formats` | Formats de points EC |
|
||||
| `cipher_suites` | Liste suites cryptographiques (hex séparées par tirets) |
|
||||
| `extensions` | Liste IDs extensions TLS (hex séparés par tirets) |
|
||||
| `supported_groups` | Groupes Diffie-Hellman supportés |
|
||||
| `ec_point_formats` | Formats de points elliptiques |
|
||||
| `alpn` | ALPN list (h2, http/1.1, ...) |
|
||||
| `sni` | Server Name Indication |
|
||||
| `ja4` | Empreinte JA4 calculée Go-side |
|
||||
| `ja4t` | Empreinte JA4T (TCP) calculée Go-side |
|
||||
| `ja4` | Empreinte JA4 (FoxIO) |
|
||||
| `ja3_raw` | Empreinte JA3 brute (version,ciphers,exts,groups,ecfmts) |
|
||||
| `ja3_hash` | JA3 hash MD5 |
|
||||
|
||||
### L7 HTTP/1.1
|
||||
|
||||
| Champ | Description |
|
||||
|-------|-------------|
|
||||
| `method` | Méthode HTTP |
|
||||
| `path` | Chemin |
|
||||
| `query_string` | Paramètres query |
|
||||
| `path` | Chemin (sans query string) |
|
||||
| `query_string` | Paramètres query (sans le '?') |
|
||||
| `host` | En-tête Host ou TLS SNI |
|
||||
| `http_version` | HTTP/1.0 ou HTTP/1.1 |
|
||||
| `headers_raw` | En-têtes dans leur ordre d'émission |
|
||||
| `header_order_signature` | Hash de l'ordre |
|
||||
| `status_code` | Code de statut |
|
||||
| `headers_raw` | Noms d'en-têtes joints par ";" |
|
||||
| `header_order_signature` | Signature de l'ordre |
|
||||
| `status_code` | Code de statut de la réponse |
|
||||
| `response_size` | Taille réponse (octets) |
|
||||
| `duration_ms` | Durée requête |
|
||||
| `timestamp_ns` | Horodatage ns absolu |
|
||||
| `client_headers` | JSON des en-têtes capturés |
|
||||
| `header_User-Agent`, etc. | 16 en-têtes individuels capturés |
|
||||
|
||||
### L7 HTTP/2 (preface client)
|
||||
|
||||
| Champ | Description |
|
||||
|-------|-------------|
|
||||
| `h2_header_table_size` | SETTINGS ID 1 (`nil` si absent du preface, omis dans le JSON) |
|
||||
| `h2_header_table_size` | SETTINGS ID 1 (-1 si absent du preface) |
|
||||
| `h2_enable_push` | SETTINGS ID 2 |
|
||||
| `h2_max_concurrent_streams` | SETTINGS ID 3 |
|
||||
| `h2_initial_window_size` | SETTINGS ID 4 |
|
||||
@ -178,63 +241,100 @@ Buffer reçu (SSL data ou HTTP plain)
|
||||
| `h2_max_header_list_size` | SETTINGS ID 6 |
|
||||
| `h2_enable_connect_protocol` | SETTINGS ID 8 (RFC 8441) |
|
||||
| `h2_window_update` | Incrément WINDOW_UPDATE connexion |
|
||||
| `h2_has_priority` | Flag PRIORITY dans HEADERS frame |
|
||||
| `h2_has_priority` | 1 si une frame PRIORITY a été reçue |
|
||||
| `h2_settings_ack` | 1 si SETTINGS ACK reçu |
|
||||
| `h2_pseudo_order` | Ordre pseudo-headers (ex: `m,a,s,p`) |
|
||||
| `h2_fingerprint` | Fingerprint composite Akamai |
|
||||
| `h2_settings_fp` | Chaîne brute des SETTINGS |
|
||||
|
||||
## eBPF CO-RE
|
||||
|
||||
| Aspect | Détail |
|
||||
|--------|--------|
|
||||
| Compilateur | `clang` + `llvm` |
|
||||
| Target | `bpf` (architecture BPF 64 bits) |
|
||||
| Compilateur | `clang` + `llvm` via `bpf2go` |
|
||||
| Target | `bpf` (amd64, kernel 4.18+) |
|
||||
| BTF source | `/sys/kernel/btf/vmlinux` (disponible RHEL 8+) |
|
||||
| Relocations | `cilium/ebpf` résout automatiquement les offsets struct |
|
||||
| Embed | `go:generate` génère `bpf_bpfel.go` avec bytecode embarqué |
|
||||
| Embed | `go:generate` génère `ja4tc_x86_bpfel.go` + `ja4ssl_x86_bpfel.go` |
|
||||
| Compatibilité | Rocky/RHEL Linux 8, 9, 10 (kernel 4.18+) |
|
||||
| IPC | `BPF_MAP_TYPE_PERF_EVENT_ARRAY` (kernel 4.4+) |
|
||||
| Lecture paquets | `bpf_skb_load_bytes()` (kernel 4.5+) avec tailles constantes en cascade |
|
||||
|
||||
**Contrainte kernel 4.18** : le vérificateur BPF rejette les tailles variables vers map values. Toutes les copies de payload utilisent des tailles constantes (1024 → 512 → 256 pour TLS, 512 → 256 → 128 → 64 pour HTTP). Les structs > 512o utilisent un `PERCPU_ARRAY` temporaire (limite stack eBPF).
|
||||
|
||||
## Configuration
|
||||
|
||||
```yaml
|
||||
# /etc/ja4ebpf/config.yml
|
||||
interface: eth0
|
||||
target_binary: /usr/sbin/httpd
|
||||
|
||||
# Interfaces réseau à surveiller (TC ingress).
|
||||
# "any" = toutes les interfaces UP (sauf loopback).
|
||||
# Ou liste explicite : ["eth0", "eth1"]
|
||||
interfaces:
|
||||
- any
|
||||
|
||||
# Chemin vers libssl pour les uprobes SSL_read/SSL_write/SSL_set_fd
|
||||
ssl_lib_path: "/usr/lib64/libssl.so.3"
|
||||
|
||||
# Ports TCP à surveiller (filtrage BPF côté kernel)
|
||||
listen_ports:
|
||||
- 80
|
||||
- 443
|
||||
|
||||
# CIDR/IP sources à ignorer (filtrage BPF LPM_TRIE + filtrage userspace SSL)
|
||||
# ignore_src:
|
||||
# - 10.0.0.0/8
|
||||
# - 172.16.0.0/12
|
||||
# - 192.168.0.0/16
|
||||
# - 127.0.0.1
|
||||
|
||||
# Mode debug
|
||||
debug: false
|
||||
|
||||
clickhouse:
|
||||
dsn: clickhouse://data_writer:pwd@localhost:9000/ja4_logs
|
||||
table: http_logs_raw
|
||||
dsn: "clickhouse://default:@127.0.0.1:9000/ja4_logs?async_insert=0"
|
||||
batch_size: 500
|
||||
flush_interval_ms: 200
|
||||
flush_secs: 1
|
||||
|
||||
correlation:
|
||||
session_timeout_ms: 500
|
||||
slowloris_threshold_s: 10
|
||||
gc_interval_ms: 100
|
||||
timeout_ms: 500
|
||||
slowloris_ms: 10000
|
||||
|
||||
log:
|
||||
level: "info"
|
||||
format: "json"
|
||||
```
|
||||
|
||||
### Variables d'environnement
|
||||
|
||||
| Variable | Défaut | Description |
|
||||
|----------|--------|-------------|
|
||||
| `JA4EBPF_INTERFACE` | `eth0` | Interface réseau |
|
||||
| `JA4EBPF_TARGET_BINARY` | `/usr/sbin/httpd` | Binaire à hooker (uprobe SSL_read) |
|
||||
| `JA4EBPF_CLICKHOUSE_DSN` | — | DSN ClickHouse |
|
||||
| `JA4EBPF_BATCH_SIZE` | `500` | Taille des batchs d'insertion |
|
||||
| `JA4EBPF_FLUSH_INTERVAL_MS` | `200` | Intervalle de flush (ms) |
|
||||
| `JA4EBPF_SESSION_TIMEOUT_MS` | `500` | Timeout session orpheline |
|
||||
| `JA4EBPF_SLOWLORIS_THRESHOLD_S` | `10` | Seuil détection Slowloris (s) |
|
||||
| `JA4EBPF_CONFIG` | `/etc/ja4ebpf/config.yml` | Chemin du fichier config |
|
||||
| `JA4EBPF_INTERFACES` | `any` | Interfaces (séparées par virgule) |
|
||||
| `JA4EBPF_INTERFACE` | — | Rétrocompatibilité : une seule interface |
|
||||
| `JA4EBPF_SSL_LIB_PATH` | `/usr/lib64/libssl.so.3` | Chemin libssl |
|
||||
| `JA4EBPF_CLICKHOUSE_DSN` | voir config | DSN ClickHouse |
|
||||
| `JA4EBPF_DEBUG` | `false` | Mode debug (`true`, `1`, `yes`) |
|
||||
| `JA4EBPF_LISTEN_PORTS` | `80,443` | Ports surveillés (séparés par virgule) |
|
||||
| `JA4EBPF_IGNORE_SRC` | — | CIDR ignorés (séparés par virgule) |
|
||||
|
||||
## Build
|
||||
|
||||
```bash
|
||||
# Compilation eBPF → Go (nécessite clang sur la machine cible)
|
||||
go generate ./internal/loader/
|
||||
|
||||
# Build du binaire
|
||||
go build ./cmd/ja4ebpf/
|
||||
|
||||
# Build complet (bytecode eBPF + binaire Go) — Docker Rocky Linux
|
||||
make build-ja4ebpf
|
||||
make build
|
||||
|
||||
# Tests unitaires (nécessite NET_RAW/NET_ADMIN/BPF capabilities)
|
||||
make test-ja4ebpf
|
||||
# Tests unitaires
|
||||
go test ./...
|
||||
|
||||
# Build RPMs el8/el9/el10
|
||||
# Build RPMs
|
||||
make rpm-ja4ebpf
|
||||
# → services/ja4ebpf/dist/rpm/el{8,9,10}/
|
||||
```
|
||||
|
||||
## Structure du code
|
||||
@ -242,31 +342,48 @@ make rpm-ja4ebpf
|
||||
```
|
||||
services/ja4ebpf/
|
||||
├── bpf/
|
||||
│ ├── bpf_types.h # Structs C partagées + déclarations maps eBPF
|
||||
│ ├── bpf_types.h # Structs C partagées + déclarations maps PerfEventArray
|
||||
│ ├── headers/vmlinux.h # Types kernel BTF (auto-généré)
|
||||
│ ├── tc_capture.c # Programme TC ingress (L3/L4/L5 + HTTP plain)
|
||||
│ └── uprobe_ssl.c # Programme uprobe SSL_read (L7 déchiffré)
|
||||
│ └── uprobe_ssl.c # Programme uprobes SSL + tracepoints accept4
|
||||
├── cmd/ja4ebpf/
|
||||
│ └── main.go # Point d'entrée : 5 goroutines consumer
|
||||
│ ├── main.go # Point d'entrée : 5 goroutines consumer + config
|
||||
│ └── main_test.go # Tests parseCIDRs, parseIgnoreNets, isIgnoredIP, parseTCPOptions
|
||||
├── internal/
|
||||
│ ├── loader/
|
||||
│ │ └── loader.go # Chargement eBPF + RingBuffer readers + désérialisation
|
||||
│ │ ├── loader.go # Chargement eBPF + PerfEvent readers + attachement TC/uprobes
|
||||
│ │ ├── ja4tc_x86_bpfel.go # Bytecode TC embarqué (généré par bpf2go)
|
||||
│ │ └── ja4ssl_x86_bpfel.go# Bytecode SSL embarqué (généré par bpf2go)
|
||||
│ ├── parser/
|
||||
│ │ ├── ja4.go # Calcul empreintes JA4 / JA4T
|
||||
│ │ ├── http2.go # Parser HTTP/2 preface (SETTINGS, WINDOW_UPDATE, HEADERS)
|
||||
│ │ └── http1.go # Parser HTTP/1.1
|
||||
│ │ ├── tls.go # ParseClientHello + ComputeJA4 + ComputeJA3
|
||||
│ │ ├── http1.go # Parser HTTP/1.1 (requêtes + réponses)
|
||||
│ │ ├── http2.go # Constantes HTTP/2 + filtres en-têtes capturés
|
||||
│ │ ├── h2conn.go # H2ConnState : framer + HPACK + fingerprinting
|
||||
│ │ ├── tls_test.go # Tests JA4, JA3, SNI, extensions
|
||||
│ │ ├── http1_test.go # Tests HTTP/1.1
|
||||
│ │ └── http2_test.go # Tests H2ConnState complet
|
||||
│ ├── dispatcher/
|
||||
│ │ └── dispatcher.go # Routeur Magic Bytes (ProtoHTTP1/2/Unknown)
|
||||
│ │ ├── dispatcher.go # Routeur Magic Bytes (ProtoHTTP1/2/Unknown)
|
||||
│ │ └── dispatcher_test.go # Tests Classify
|
||||
│ ├── correlation/
|
||||
│ │ ├── manager.go # Gestionnaire sessions 256-shard
|
||||
│ │ └── session.go # Structs L3L4, TLSInfo, SessionState
|
||||
│ │ ├── session.go # Structs L3L4, TLSInfo, HTTPRequest, SessionState
|
||||
│ │ ├── manager.go # Gestionnaire sessions 256-shard + GC
|
||||
│ │ ├── accept_cache.go # AcceptCache {tgid,fd} → SessionKey (TTL 10s)
|
||||
│ │ ├── correlation_test.go# Tests Manager + SessionState
|
||||
│ │ └── accept_cache_test.go# Tests AcceptCache
|
||||
│ ├── procutil/
|
||||
│ │ ├── proc_lookup.go # FDCache : résolution fd → IP via /proc/net/tcp
|
||||
│ │ └── proc_lookup_test.go# Tests parseHexIPv4/IPv6, isIPv4MappedIPv6
|
||||
│ └── writer/
|
||||
│ └── writer.go # Writer ClickHouse (batch + retry)
|
||||
│ ├── clickhouse.go # Writer ClickHouse (batch + JSON → http_logs_raw)
|
||||
│ └── clickhouse_test.go # Tests formatTCPOptions, H2 fingerprints, etc.
|
||||
├── packaging/
|
||||
│ ├── rpm/ja4ebpf.spec # Spec RPM (el8/el9/el10)
|
||||
│ └── systemd/ja4ebpf.service # Unit systemd
|
||||
├── config.yml.example # Exemple de configuration
|
||||
├── Dockerfile # Image de production
|
||||
├── Dockerfile.tests # Image de tests
|
||||
├── Dockerfile.package # Build RPM multi-distro (5 stages)
|
||||
├── Dockerfile.package # Build RPM multi-distro
|
||||
└── Makefile
|
||||
```
|
||||
|
||||
@ -279,18 +396,9 @@ L'agent tourne sous l'utilisateur `ja4ebpf` (UID/GID 490 fixe). Les capabilities
|
||||
| `CAP_BPF` | Chargement des programmes eBPF (kernel 5.8+) |
|
||||
| `CAP_SYS_ADMIN` | Uprobes + RHEL 8 (kernel 4.18, pré-CAP_BPF) |
|
||||
| `CAP_NET_ADMIN` | Attachement hooks TC ingress |
|
||||
| `CAP_NET_RAW` | Accès raw socket (fallback) |
|
||||
| `CAP_PERFMON` | Accès perf events pour les uprobes |
|
||||
| `CAP_SYS_PTRACE` | Résolution des offsets de fonctions pour les uprobes |
|
||||
| `CAP_DAC_READ_SEARCH` | Lecture `/proc/<pid>/maps` pour localiser libssl.so |
|
||||
|
||||
`LimitMEMLOCK=infinity` est requis pour le `mlock()` des maps eBPF.
|
||||
|
||||
## Tests d'intégration
|
||||
|
||||
Stacks Docker Compose testant l'agent contre différents serveurs web :
|
||||
|
||||
```bash
|
||||
make test-integration # Apache httpd (référence)
|
||||
make test-nginx # Nginx
|
||||
make test-nginx-varnish # Nginx + Varnish (reverse proxy)
|
||||
make test-hitch-varnish # Hitch (TLS) + Varnish
|
||||
make test-all-stacks # Les 4 stacks en séquence
|
||||
```
|
||||
|
||||
@ -10,11 +10,11 @@
|
||||
┌──────────────────────────────────────────────────────────────────┐
|
||||
│ SOURCES DE DONNÉES │
|
||||
├───────────────────────────┬──────────────────────────────────────┤
|
||||
│ TC ingress (XDP/TC) │ uprobe SSL_read │
|
||||
│ TC ingress │ uprobe SSL_read/SSL_write │
|
||||
│ Couches L3/L4/L5 │ Couche L7 HTTP déchiffré │
|
||||
│ │ │
|
||||
│ │ │
|
||||
│ réseau XDP/TC → │ Go Magic Bytes dispatcher → │
|
||||
│ réseau TC → │ Go Magic Bytes dispatcher → │
|
||||
│ - SYN : TTL, IP-ID, DF, │ HTTP/1.1 : method, path, query, │
|
||||
│ MSS, Window, Scale │ headers (bruts + ordre), │
|
||||
│ - TLS ClientHello : │ status, taille, durée_ms, │
|
||||
|
||||
@ -47,6 +47,28 @@ struct {
|
||||
__type(value, __u64);
|
||||
} tc_stats SEC(".maps");
|
||||
|
||||
/* Map de ports autorisés — peuplée depuis Go au démarrage.
|
||||
* key = port (uint16), value = 1 (autorisé).
|
||||
* Ports non présents dans la map sont ignorés. */
|
||||
struct {
|
||||
__uint(type, BPF_MAP_TYPE_HASH);
|
||||
__uint(max_entries, 64);
|
||||
__type(key, __u16);
|
||||
__type(value, __u8);
|
||||
} allowed_ports SEC(".maps");
|
||||
|
||||
/* Map LPM_TRIE des CIDR/IP sources à ignorer — peuplée depuis Go.
|
||||
* key = {prefixlen, ip[4]} (8 octets), value = 1 (ignorer).
|
||||
* Un lookup réussi = IP source à ignorer → return TC_ACT_OK.
|
||||
* data est en network byte order (big-endian) pour correspondre à iph.saddr. */
|
||||
struct {
|
||||
__uint(type, BPF_MAP_TYPE_LPM_TRIE);
|
||||
__uint(max_entries, 256);
|
||||
__type(key, struct { __u32 prefixlen; __u8 data[4]; });
|
||||
__type(value, __u8);
|
||||
__uint(map_flags, BPF_F_NO_PREALLOC);
|
||||
} ignored_src SEC(".maps");
|
||||
|
||||
#define STAT_TOTAL 0
|
||||
#define STAT_IPV4 1
|
||||
#define STAT_TCP 2
|
||||
@ -137,6 +159,36 @@ int capture_tc(struct __sk_buff *ctx)
|
||||
|
||||
__u32 payload_off = ETH_HLEN + ip_hlen + tcp_hlen;
|
||||
|
||||
__u32 avail = 0;
|
||||
__u32 zero = 0;
|
||||
|
||||
/* Vérification globale : port autorisé ? (SYN, TLS, HTTP)
|
||||
* On autorise si dst_port OU src_port est dans allowed_ports.
|
||||
* En ingress TC, les réponses ont src_port=80/443 (serveur distant)
|
||||
* et dst_port=ephemeral (client local). */
|
||||
__u8 *port_allowed = bpf_map_lookup_elem(&allowed_ports, &dst_port);
|
||||
if (!port_allowed) {
|
||||
port_allowed = bpf_map_lookup_elem(&allowed_ports, &src_port);
|
||||
if (!port_allowed)
|
||||
return TC_ACT_OK;
|
||||
}
|
||||
|
||||
/* Vérification : IP source ignorée ? (LPM_TRIE lookup /32) */
|
||||
struct { __u32 prefixlen; __u8 data[4]; } lpm_key = {};
|
||||
lpm_key.prefixlen = 32;
|
||||
/* Copier src_ip (network byte order) dans data[4] byte par byte.
|
||||
* src_ip est en network byte order (big-endian) depuis iph.saddr.
|
||||
* Sur x86 little-endian, il faut extraire du MSB vers le LSB
|
||||
* pour que data[] soit en network byte order comme les clés Go. */
|
||||
__u32 src_ip_h = bpf_ntohl(src_ip);
|
||||
lpm_key.data[0] = (__u8)((src_ip_h >> 24) & 0xFF);
|
||||
lpm_key.data[1] = (__u8)((src_ip_h >> 16) & 0xFF);
|
||||
lpm_key.data[2] = (__u8)((src_ip_h >> 8) & 0xFF);
|
||||
lpm_key.data[3] = (__u8)(src_ip_h & 0xFF);
|
||||
__u8 *src_ignored = bpf_map_lookup_elem(&ignored_src, &lpm_key);
|
||||
if (src_ignored)
|
||||
return TC_ACT_OK;
|
||||
|
||||
/* ===================================================================
|
||||
* TCP SYN
|
||||
* ===================================================================*/
|
||||
@ -160,15 +212,23 @@ int capture_tc(struct __sk_buff *ctx)
|
||||
evt.timestamp_ns = bpf_ktime_get_ns();
|
||||
evt.tcp_options_len = 0;
|
||||
|
||||
/* Copie des options TCP via bpf_skb_load_bytes avec taille constante.
|
||||
* On lit MAX_TCP_OPTIONS=40 octets depuis le début des options.
|
||||
* Si le paquet est trop court, l'appel échoue → options absentes. */
|
||||
/* Copie des options TCP via bpf_skb_load_bytes avec cascade de tailles.
|
||||
* Le vérificateur BPF exige une taille constante pour bpf_skb_load_bytes.
|
||||
* On essaie 40, puis 20, puis 10 octets — le premier appel qui réussit
|
||||
* donne les options disponibles (même partielles). */
|
||||
__u32 opts_off = tcp_off + 20;
|
||||
__u32 opts_len = tcp_hlen - 20;
|
||||
if (opts_len > 0 && opts_len <= MAX_TCP_OPTIONS &&
|
||||
opts_off + MAX_TCP_OPTIONS <= pkt_len) {
|
||||
bpf_skb_load_bytes(ctx, opts_off, evt.tcp_options_raw, MAX_TCP_OPTIONS);
|
||||
if (opts_len > 0 && opts_len <= MAX_TCP_OPTIONS) {
|
||||
if (opts_off + 40 <= pkt_len) {
|
||||
bpf_skb_load_bytes(ctx, opts_off, evt.tcp_options_raw, 40);
|
||||
evt.tcp_options_len = (__u8)opts_len;
|
||||
} else if (opts_off + 20 <= pkt_len) {
|
||||
bpf_skb_load_bytes(ctx, opts_off, evt.tcp_options_raw, 20);
|
||||
evt.tcp_options_len = (__u8)(opts_len > 20 ? 20 : opts_len);
|
||||
} else if (opts_off + 10 <= pkt_len) {
|
||||
bpf_skb_load_bytes(ctx, opts_off, evt.tcp_options_raw, 10);
|
||||
evt.tcp_options_len = (__u8)(opts_len > 10 ? 10 : opts_len);
|
||||
}
|
||||
}
|
||||
|
||||
bpf_perf_event_output(ctx, &pb_tcp_syn, BPF_F_CURRENT_CPU,
|
||||
@ -180,21 +240,20 @@ int capture_tc(struct __sk_buff *ctx)
|
||||
}
|
||||
|
||||
/* ===================================================================
|
||||
* TLS ClientHello (port 443)
|
||||
* TLS ClientHello
|
||||
* ===================================================================*/
|
||||
if (dst_port == HTTPS_PORT) {
|
||||
/* Lire les 6 premiers octets du payload pour vérifier le type TLS */
|
||||
if (payload_off + 6 > pkt_len)
|
||||
return TC_ACT_OK;
|
||||
goto try_http;
|
||||
|
||||
__u8 tls_hdr[6];
|
||||
bpf_skb_load_bytes(ctx, payload_off, tls_hdr, 6);
|
||||
|
||||
if (tls_hdr[0] != TLS_CONTENT_HANDSHAKE || tls_hdr[5] != TLS_MSG_CLIENT_HELLO)
|
||||
return TC_ACT_OK;
|
||||
goto try_http;
|
||||
|
||||
/* Avail via pkt_len (scalaire pur) */
|
||||
__u32 avail = 0;
|
||||
avail = 0;
|
||||
if (pkt_len > payload_off) {
|
||||
avail = pkt_len - payload_off;
|
||||
if (avail > MAX_TLS_PAYLOAD)
|
||||
@ -203,7 +262,6 @@ int capture_tc(struct __sk_buff *ctx)
|
||||
if (avail == 0)
|
||||
return TC_ACT_OK;
|
||||
|
||||
__u32 zero = 0;
|
||||
struct tls_hello_event *tls_evt = bpf_map_lookup_elem(&__tls_buf, &zero);
|
||||
if (!tls_evt)
|
||||
return TC_ACT_OK;
|
||||
@ -246,19 +304,19 @@ int capture_tc(struct __sk_buff *ctx)
|
||||
if (cnt) (*cnt)++;
|
||||
|
||||
return TC_ACT_OK;
|
||||
}
|
||||
|
||||
try_http:
|
||||
|
||||
/* ===================================================================
|
||||
* HTTP en clair (port 80 / 8080)
|
||||
* HTTP en clair (ports autorisés, non-TLS)
|
||||
* ===================================================================*/
|
||||
if (dst_port == HTTP_PORT || dst_port == HTTP_ALT_PORT) {
|
||||
if (tcp_flags & (TH_SYN | TH_FIN | TH_RST))
|
||||
return TC_ACT_OK;
|
||||
if (payload_off >= pkt_len)
|
||||
return TC_ACT_OK;
|
||||
|
||||
/* Avail via pkt_len (scalaire pur) */
|
||||
__u32 avail = 0;
|
||||
avail = 0;
|
||||
if (pkt_len > payload_off) {
|
||||
avail = pkt_len - payload_off;
|
||||
if (avail > MAX_HTTP_PAYLOAD)
|
||||
@ -267,7 +325,6 @@ int capture_tc(struct __sk_buff *ctx)
|
||||
if (avail == 0)
|
||||
return TC_ACT_OK;
|
||||
|
||||
__u32 zero = 0;
|
||||
struct http_plain_event *h_evt = bpf_map_lookup_elem(&__http_buf, &zero);
|
||||
if (!h_evt)
|
||||
return TC_ACT_OK;
|
||||
@ -286,7 +343,7 @@ int capture_tc(struct __sk_buff *ctx)
|
||||
h_evt->timestamp_ns = bpf_ktime_get_ns();
|
||||
|
||||
/* Copie via bpf_skb_load_bytes avec tailles constantes en cascade.
|
||||
* Les requêtes HTTP sont souvent < 512 octets, on descend à 256 puis 128. */
|
||||
* Les requêtes HTTP sont souvent < 512 octets, on descend à 256, 128, 64. */
|
||||
if (payload_off + 512 <= pkt_len) {
|
||||
bpf_skb_load_bytes(ctx, payload_off, h_evt, 512);
|
||||
h_evt->payload_len = 512;
|
||||
@ -296,6 +353,9 @@ int capture_tc(struct __sk_buff *ctx)
|
||||
} else if (payload_off + 128 <= pkt_len) {
|
||||
bpf_skb_load_bytes(ctx, payload_off, h_evt, 128);
|
||||
h_evt->payload_len = 128;
|
||||
} else if (payload_off + 64 <= pkt_len) {
|
||||
bpf_skb_load_bytes(ctx, payload_off, h_evt, 64);
|
||||
h_evt->payload_len = 64;
|
||||
} else {
|
||||
return TC_ACT_OK;
|
||||
}
|
||||
@ -306,7 +366,6 @@ int capture_tc(struct __sk_buff *ctx)
|
||||
key = STAT_HTTP_SUBMIT;
|
||||
cnt = bpf_map_lookup_elem(&tc_stats, &key);
|
||||
if (cnt) (*cnt)++;
|
||||
}
|
||||
|
||||
return TC_ACT_OK;
|
||||
}
|
||||
|
||||
@ -7,9 +7,11 @@ import (
|
||||
"context"
|
||||
"encoding/binary"
|
||||
"fmt"
|
||||
"net"
|
||||
"log"
|
||||
"os"
|
||||
"os/signal"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
"syscall"
|
||||
@ -32,12 +34,17 @@ var fdCache = procutil.NewFDCache(5 * time.Second)
|
||||
// Prioritaire sur fdCache car source de vérité (tracepoint kernel).
|
||||
var acceptCache = correlation.NewAcceptCache(10 * time.Second)
|
||||
|
||||
// ignoreNets contient les CIDR sources à ignorer (peuplé depuis cfg.IgnoreSrc).
|
||||
var ignoreNets []*net.IPNet
|
||||
|
||||
// Config décrit la configuration complète du démon ja4ebpf.
|
||||
// Chargée depuis un fichier YAML et enrichie par les variables d'environnement
|
||||
// avec le préfixe JA4EBPF_.
|
||||
type Config struct {
|
||||
Interface string `yaml:"interface"` // interface réseau à surveiller (ex: "eth0")
|
||||
Interfaces []string `yaml:"interfaces"` // interfaces à surveiller (défaut: ["any"])
|
||||
SSLLibPath string `yaml:"ssl_lib_path"` // chemin vers libssl (ex: "/usr/lib64/libssl.so.3")
|
||||
ListenPorts []uint16 `yaml:"listen_ports"` // ports à surveiller (défaut: [80, 443])
|
||||
IgnoreSrc []string `yaml:"ignore_src"` // CIDR/IP sources à ignorer (ex: ["10.0.0.0/8"])
|
||||
Debug bool `yaml:"debug"` // mode debug : dump compteurs BPF, log verbeux, ClickHouse optionnel
|
||||
|
||||
ClickHouse struct {
|
||||
@ -63,9 +70,10 @@ func loadConfig(path string) (*Config, error) {
|
||||
cfg := &Config{}
|
||||
|
||||
// Valeurs par défaut
|
||||
cfg.Interface = "eth0"
|
||||
cfg.Interfaces = []string{"any"}
|
||||
cfg.SSLLibPath = "/usr/lib64/libssl.so.3"
|
||||
cfg.ClickHouse.DSN = "clickhouse://default:@localhost:9000/ja4_logs"
|
||||
cfg.ListenPorts = []uint16{80, 443}
|
||||
cfg.ClickHouse.DSN = "clickhouse://default:@localhost:9000/ja4_logs?async_insert=0"
|
||||
cfg.ClickHouse.BatchSize = 500
|
||||
cfg.ClickHouse.FlushSecs = 1
|
||||
cfg.Correlation.TimeoutMS = 5000
|
||||
@ -85,8 +93,12 @@ func loadConfig(path string) (*Config, error) {
|
||||
}
|
||||
|
||||
// Surcharges via variables d'environnement
|
||||
if v := os.Getenv("JA4EBPF_INTERFACES"); v != "" {
|
||||
cfg.Interfaces = strings.Split(v, ",")
|
||||
}
|
||||
// Rétrocompatibilité : JA4EBPF_INTERFACE écrase la liste
|
||||
if v := os.Getenv("JA4EBPF_INTERFACE"); v != "" {
|
||||
cfg.Interface = v
|
||||
cfg.Interfaces = []string{v}
|
||||
}
|
||||
if v := os.Getenv("JA4EBPF_SSL_LIB_PATH"); v != "" {
|
||||
cfg.SSLLibPath = v
|
||||
@ -97,11 +109,83 @@ func loadConfig(path string) (*Config, error) {
|
||||
if v := os.Getenv("JA4EBPF_DEBUG"); v != "" {
|
||||
cfg.Debug = strings.EqualFold(v, "true") || v == "1" || v == "yes"
|
||||
}
|
||||
if v := os.Getenv("JA4EBPF_LISTEN_PORTS"); v != "" {
|
||||
cfg.ListenPorts = nil
|
||||
for _, s := range strings.Split(v, ",") {
|
||||
p, err := strconv.ParseUint(strings.TrimSpace(s), 10, 16)
|
||||
if err != nil {
|
||||
log.Printf("[ja4ebpf] port invalide dans JA4EBPF_LISTEN_PORTS: %q", s)
|
||||
continue
|
||||
}
|
||||
cfg.ListenPorts = append(cfg.ListenPorts, uint16(p))
|
||||
}
|
||||
}
|
||||
if v := os.Getenv("JA4EBPF_IGNORE_SRC"); v != "" {
|
||||
cfg.IgnoreSrc = strings.Split(v, ",")
|
||||
}
|
||||
|
||||
return cfg, nil
|
||||
}
|
||||
|
||||
// main est le point d'entrée du programme.
|
||||
// parseCIDRs convertit une liste de CIDR/IP en clés LPM_TRIE (big-endian).
|
||||
func parseCIDRs(cidrs []string) ([]loader.LPMKey, error) {
|
||||
var keys []loader.LPMKey
|
||||
for _, cidr := range cidrs {
|
||||
cidr = strings.TrimSpace(cidr)
|
||||
if !strings.Contains(cidr, "/") {
|
||||
cidr += "/32"
|
||||
}
|
||||
_, ipNet, err := net.ParseCIDR(cidr)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("CIDR invalide %q: %w", cidr, err)
|
||||
}
|
||||
ip4 := ipNet.IP.To4()
|
||||
if ip4 == nil {
|
||||
continue
|
||||
}
|
||||
prefixLen, _ := ipNet.Mask.Size()
|
||||
var data [4]byte
|
||||
copy(data[:], ip4)
|
||||
keys = append(keys, loader.LPMKey{
|
||||
Prefixlen: uint32(prefixLen),
|
||||
Data: data,
|
||||
})
|
||||
}
|
||||
return keys, nil
|
||||
}
|
||||
|
||||
// isIgnoredIP vérifie si une adresse IPv4 (4 octets) match un des CIDR ignore_src.
|
||||
func isIgnoredIP(ip [4]byte) bool {
|
||||
ip4 := net.IPv4(ip[0], ip[1], ip[2], ip[3])
|
||||
for _, cidr := range ignoreNets {
|
||||
if cidr.Contains(ip4) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// parseIgnoreNets convertit la liste de CIDR ignore_src en []*net.IPNet.
|
||||
func parseIgnoreNets(cidrs []string) []*net.IPNet {
|
||||
var nets []*net.IPNet
|
||||
for _, cidr := range cidrs {
|
||||
cidr = strings.TrimSpace(cidr)
|
||||
if !strings.Contains(cidr, "/") {
|
||||
cidr += "/32"
|
||||
}
|
||||
_, ipNet, err := net.ParseCIDR(cidr)
|
||||
if err != nil {
|
||||
log.Printf("[ja4ebpf] CIDR ignore_src invalide %q: %v", cidr, err)
|
||||
continue
|
||||
}
|
||||
if ipNet.IP.To4() != nil {
|
||||
nets = append(nets, ipNet)
|
||||
}
|
||||
}
|
||||
return nets
|
||||
}
|
||||
|
||||
func main() {
|
||||
// Déterminer le chemin du fichier de configuration
|
||||
configPath := os.Getenv("JA4EBPF_CONFIG")
|
||||
@ -117,7 +201,10 @@ func main() {
|
||||
if cfg.Debug {
|
||||
log.Printf("[ja4ebpf] MODE DEBUG ACTIVÉ")
|
||||
}
|
||||
log.Printf("[ja4ebpf] démarrage — interface=%s ssl=%s debug=%v", cfg.Interface, cfg.SSLLibPath, cfg.Debug)
|
||||
log.Printf("[ja4ebpf] démarrage — interfaces=%v ssl=%s debug=%v", cfg.Interfaces, cfg.SSLLibPath, cfg.Debug)
|
||||
|
||||
// Peupler ignoreNets pour filtrage userspace (SSL_read/SSL_write/accept4)
|
||||
ignoreNets = parseIgnoreNets(cfg.IgnoreSrc)
|
||||
|
||||
// Contexte principal avec annulation sur signal système
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
@ -134,12 +221,44 @@ func main() {
|
||||
}
|
||||
defer ldr.Close()
|
||||
|
||||
// --- 2. Attachement TC ingress ---
|
||||
log.Printf("[ja4ebpf] attachement TC ingress sur %s...", cfg.Interface)
|
||||
if err := ldr.AttachTC(cfg.Interface); err != nil {
|
||||
log.Fatalf("erreur attachement TC sur %s: %v", cfg.Interface, err)
|
||||
// --- 1b. Peuplement de la map allowed_ports ---
|
||||
if err := ldr.PopulatePorts(cfg.ListenPorts); err != nil {
|
||||
log.Fatalf("[ja4ebpf] erreur peuplement allowed_ports: %v", err)
|
||||
}
|
||||
for _, p := range cfg.ListenPorts {
|
||||
log.Printf("[ja4ebpf] port %d surveillé", p)
|
||||
}
|
||||
|
||||
// --- 1c. Peuplement de la map ignored_src (LPM_TRIE) ---
|
||||
if len(cfg.IgnoreSrc) > 0 {
|
||||
lpmKeys, err := parseCIDRs(cfg.IgnoreSrc)
|
||||
if err != nil {
|
||||
log.Fatalf("[ja4ebpf] erreur parsing ignore_src: %v", err)
|
||||
}
|
||||
if err := ldr.PopulateIgnoredSrc(lpmKeys); err != nil {
|
||||
log.Fatalf("[ja4ebpf] erreur peuplement ignored_src: %v", err)
|
||||
}
|
||||
for _, c := range cfg.IgnoreSrc {
|
||||
log.Printf("[ja4ebpf] ignore src: %s", c)
|
||||
}
|
||||
}
|
||||
|
||||
// --- 2. Attachement TC ingress ---
|
||||
if len(cfg.Interfaces) == 1 && cfg.Interfaces[0] == "any" {
|
||||
ifaces, err := ldr.AttachTCAll()
|
||||
if err != nil {
|
||||
log.Fatalf("[ja4ebpf] erreur attachement TC: %v", err)
|
||||
}
|
||||
log.Printf("[ja4ebpf] TC ingress attaché sur: %v", ifaces)
|
||||
} else {
|
||||
for _, iface := range cfg.Interfaces {
|
||||
log.Printf("[ja4ebpf] attachement TC ingress sur %s...", iface)
|
||||
if err := ldr.AttachTC(iface); err != nil {
|
||||
log.Fatalf("[ja4ebpf] erreur attachement TC %s: %v", iface, err)
|
||||
}
|
||||
log.Printf("[ja4ebpf] TC ingress attaché sur %s", iface)
|
||||
}
|
||||
}
|
||||
log.Printf("[ja4ebpf] TC ingress attaché sur %s", cfg.Interface)
|
||||
|
||||
// --- 3. Attachement uprobes SSL ---
|
||||
if err := ldr.AttachUprobes(cfg.SSLLibPath); err != nil {
|
||||
@ -349,6 +468,11 @@ func consumeSynEvents(ctx context.Context, rd *perf.Reader, mgr *correlation.Man
|
||||
tcpOpts := make([]byte, optLen)
|
||||
copy(tcpOpts, data[23:23+optLen])
|
||||
|
||||
// Filtrer les IPs sources ignorées (ignore_src)
|
||||
if isIgnoredIP(key.SrcIP) {
|
||||
continue
|
||||
}
|
||||
|
||||
// Analyser les options TCP brutes pour extraire MSS et Window Scale
|
||||
mss, windowScale := parseTCPOptions(tcpOpts)
|
||||
|
||||
@ -426,6 +550,11 @@ func consumeTLSEvents(ctx context.Context, rd *perf.Reader, mgr *correlation.Man
|
||||
tlsDstIP[2] = byte(dstIPRaw >> 8)
|
||||
tlsDstIP[3] = byte(dstIPRaw)
|
||||
|
||||
// Filtrer les IPs sources ignorées (ignore_src)
|
||||
if isIgnoredIP(key.SrcIP) {
|
||||
continue
|
||||
}
|
||||
|
||||
// Parser le ClientHello et calculer JA4
|
||||
ch, err := parser.ParseClientHello(payload)
|
||||
if err != nil {
|
||||
@ -567,6 +696,12 @@ func consumeSSLEvents(ctx context.Context, rd *perf.Reader, mgr *correlation.Man
|
||||
continue
|
||||
}
|
||||
|
||||
// Filtrer les IPs sources ignorées (ignore_src)
|
||||
if key.SrcIP != [4]byte{} && isIgnoredIP(key.SrcIP) {
|
||||
log.Printf("[debug-ssl] FILTERED srcIP=%d.%d.%d.%d", key.SrcIP[0], key.SrcIP[1], key.SrcIP[2], key.SrcIP[3])
|
||||
continue
|
||||
}
|
||||
|
||||
counter.Add(1)
|
||||
|
||||
// === Routeur par direction ===
|
||||
@ -592,121 +727,87 @@ func consumeSSLEvents(ctx context.Context, rd *perf.Reader, mgr *correlation.Man
|
||||
})
|
||||
}
|
||||
|
||||
// HTTP/2 server HEADERS frame (contient :status)
|
||||
if parser.IsH2FrameHeader(sslData) {
|
||||
h2kv := parser.ExtractH2HeaderKV(sslData)
|
||||
if statusCode, ok := h2kv[":status"]; ok {
|
||||
// HTTP/2 : traiter via H2ConnState si la connexion est H2
|
||||
mgr.Update(key, func(s *correlation.SessionState) {
|
||||
if len(s.Requests) > 0 {
|
||||
if s.H2Conn == nil {
|
||||
return
|
||||
}
|
||||
result, err := s.H2Conn.ProcessFrames(sslData, 1)
|
||||
if err != nil || result == nil {
|
||||
return
|
||||
}
|
||||
// Extraire le code de statut des réponses serveur
|
||||
if result.StatusCode > 0 && len(s.Requests) > 0 {
|
||||
last := &s.Requests[len(s.Requests)-1]
|
||||
if last.StatusCode == 0 {
|
||||
// Conversion du code de statut H2 (ex: "200" → 200)
|
||||
code := 0
|
||||
for _, c := range statusCode {
|
||||
if c >= '0' && c <= '9' {
|
||||
code = code*10 + int(c-'0')
|
||||
}
|
||||
}
|
||||
if code >= 100 && code <= 599 {
|
||||
last.StatusCode = code
|
||||
last.StatusCode = result.StatusCode
|
||||
}
|
||||
}
|
||||
// Mettre à jour les paramètres SETTINGS serveur
|
||||
if result.ServerSettings != nil {
|
||||
s.H2Conn.ServerSettings = result.ServerSettings
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
// === Client → Serveur : requêtes HTTP (direction=0) ===
|
||||
|
||||
if parser.DetectH2Preface(sslData) {
|
||||
// HTTP/2 : extraire les paramètres SETTINGS et en-têtes depuis la préface
|
||||
// HTTP/2 : préface détectée, créer H2ConnState et traiter les frames
|
||||
afterPreface := sslData
|
||||
if len(afterPreface) > parser.H2MagicPrefaceLen() {
|
||||
afterPreface = sslData[parser.H2MagicPrefaceLen():]
|
||||
}
|
||||
h2settings, err := parser.ParseH2ClientPreface(afterPreface)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
|
||||
mgr.Update(key, func(s *correlation.SessionState) {
|
||||
req := correlation.HTTPRequest{
|
||||
Timestamp: time.Now(),
|
||||
// Créer le H2ConnState s'il n'existe pas
|
||||
if s.H2Conn == nil {
|
||||
s.H2Conn = parser.NewH2ConnState()
|
||||
}
|
||||
if h2settings != nil {
|
||||
req.HTTP2Settings = &correlation.HTTP2Settings{
|
||||
HeaderTableSize: h2settings.HeaderTableSize,
|
||||
EnablePush: h2settings.EnablePush,
|
||||
MaxConcurrentStreams: h2settings.MaxConcurrentStreams,
|
||||
InitialWindowSize: h2settings.InitialWindowSize,
|
||||
MaxFrameSize: h2settings.MaxFrameSize,
|
||||
MaxHeaderListSize: h2settings.MaxHeaderListSize,
|
||||
UnknownSettings: h2settings.UnknownSettings,
|
||||
EnableConnectProtocol: h2settings.EnableConnectProtocol,
|
||||
WindowUpdateIncrement: h2settings.WindowUpdateIncrement,
|
||||
PseudoHeaderOrder: h2settings.PseudoHeaderOrder,
|
||||
}
|
||||
// Extraire les en-têtes H2 (User-Agent, Accept, etc.)
|
||||
if len(h2settings.HeaderKV) > 0 {
|
||||
req.HeaderKV = h2settings.HeaderKV
|
||||
req.HeaderOrder = h2settings.HeaderOrder
|
||||
req.HeaderOrderSig = strings.Join(h2settings.HeaderOrder, ";")
|
||||
if h2settings.HeaderKV[":method"] != "" {
|
||||
req.Method = h2settings.HeaderKV[":method"]
|
||||
}
|
||||
if h2settings.HeaderKV[":path"] != "" {
|
||||
p := h2settings.HeaderKV[":path"]
|
||||
if idx := strings.Index(p, "?"); idx >= 0 {
|
||||
req.Path = p[:idx]
|
||||
req.QueryString = p[idx+1:]
|
||||
} else {
|
||||
req.Path = p
|
||||
}
|
||||
}
|
||||
if h2settings.HeaderKV[":authority"] != "" {
|
||||
req.Host = h2settings.HeaderKV[":authority"]
|
||||
}
|
||||
}
|
||||
}
|
||||
if len(s.Requests) == 0 {
|
||||
req.HTTPVersion = "HTTP/2"
|
||||
s.Requests = append(s.Requests, req)
|
||||
}
|
||||
// Si la session n'a pas de L3L4 (pas de SYN capturé),
|
||||
// peupler dst_ip/dst_port depuis le cache accept4
|
||||
if s.L3L4 == nil && (dstIPFromAccept != [4]byte{} || dstPortFromAccept != 0) {
|
||||
s.L3L4 = &correlation.L3L4{
|
||||
DstIP: dstIPFromAccept,
|
||||
DstPort: dstPortFromAccept,
|
||||
}
|
||||
}
|
||||
_ = s.TLS // corrélation implicite
|
||||
|
||||
result, _ := s.H2Conn.ProcessFrames(afterPreface, 0)
|
||||
applyH2Result(s, result, dstIPFromAccept, dstPortFromAccept)
|
||||
})
|
||||
continue
|
||||
}
|
||||
|
||||
// HTTP/2 frames seules (sans préface — SSL_read ultérieurs)
|
||||
if parser.IsH2FrameHeader(sslData) {
|
||||
h2kv := parser.ExtractH2HeaderKV(sslData)
|
||||
if len(h2kv) > 0 {
|
||||
// Utiliser H2ConnState si disponible
|
||||
var h2connExists bool
|
||||
mgr.Update(key, func(s *correlation.SessionState) {
|
||||
if len(s.Requests) > 0 {
|
||||
h2connExists = s.H2Conn != nil
|
||||
})
|
||||
|
||||
if h2connExists {
|
||||
mgr.Update(key, func(s *correlation.SessionState) {
|
||||
result, _ := s.H2Conn.ProcessFrames(sslData, 0)
|
||||
if result == nil {
|
||||
return
|
||||
}
|
||||
|
||||
// En-têtes décodés
|
||||
if len(result.Headers) > 0 && len(s.Requests) > 0 {
|
||||
last := &s.Requests[len(s.Requests)-1]
|
||||
if last.HeaderKV == nil {
|
||||
last.HeaderKV = make(map[string]string)
|
||||
}
|
||||
for k, v := range h2kv {
|
||||
if _, exists := last.HeaderKV[k]; !exists {
|
||||
last.HeaderKV[k] = v
|
||||
for _, h := range result.Headers {
|
||||
nameLower := strings.ToLower(h.Name)
|
||||
if parser.HpackCapturedHeaders[nameLower] && h.Value != "" {
|
||||
if _, exists := last.HeaderKV[nameLower]; !exists {
|
||||
last.HeaderKV[nameLower] = h.Value
|
||||
last.HeaderOrder = append(last.HeaderOrder, nameLower)
|
||||
}
|
||||
}
|
||||
// Mettre à jour method/path/host si pas encore remplis
|
||||
if last.Method == "" && h2kv[":method"] != "" {
|
||||
last.Method = h2kv[":method"]
|
||||
switch nameLower {
|
||||
case ":method":
|
||||
if last.Method == "" {
|
||||
last.Method = h.Value
|
||||
}
|
||||
if last.Path == "" && h2kv[":path"] != "" {
|
||||
p := h2kv[":path"]
|
||||
case ":path":
|
||||
if last.Path == "" {
|
||||
p := h.Value
|
||||
if idx := strings.Index(p, "?"); idx >= 0 {
|
||||
last.Path = p[:idx]
|
||||
last.QueryString = p[idx+1:]
|
||||
@ -714,15 +815,37 @@ func consumeSSLEvents(ctx context.Context, rd *perf.Reader, mgr *correlation.Man
|
||||
last.Path = p
|
||||
}
|
||||
}
|
||||
if last.Host == "" && h2kv[":authority"] != "" {
|
||||
last.Host = h2kv[":authority"]
|
||||
case ":authority":
|
||||
if last.Host == "" {
|
||||
last.Host = h.Value
|
||||
}
|
||||
}
|
||||
}
|
||||
if len(last.HeaderOrder) > 0 && last.HeaderOrderSig == "" {
|
||||
last.HeaderOrderSig = strings.Join(last.HeaderOrder, ";")
|
||||
}
|
||||
}
|
||||
|
||||
// Mettre à jour SETTINGS client si présents
|
||||
if result.ClientSettings != nil && len(s.Requests) > 0 {
|
||||
last := &s.Requests[len(s.Requests)-1]
|
||||
updateH2Settings(last, result.ClientSettings)
|
||||
}
|
||||
})
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
// Première frame H2 sans préface — créer H2ConnState
|
||||
if parser.IsH2FrameHeader(sslData) {
|
||||
mgr.Update(key, func(s *correlation.SessionState) {
|
||||
s.H2Conn = parser.NewH2ConnState()
|
||||
result, _ := s.H2Conn.ProcessFrames(sslData, 0)
|
||||
applyH2Result(s, result, dstIPFromAccept, dstPortFromAccept)
|
||||
})
|
||||
continue
|
||||
}
|
||||
|
||||
|
||||
if parser.IsHTTP1Request(sslData) {
|
||||
// HTTP/1.x : parser la requête
|
||||
req := parser.ParseHTTP1Request(sslData)
|
||||
@ -799,6 +922,11 @@ func consumeAcceptEvents(ctx context.Context, rd *perf.Reader, mgr *correlation.
|
||||
continue
|
||||
}
|
||||
|
||||
// Filtrer les IPs sources ignorées (ignore_src)
|
||||
if isIgnoredIP(key.SrcIP) {
|
||||
continue
|
||||
}
|
||||
|
||||
// Peupler le cache accept4 pour corrélation SSL
|
||||
tgid := uint32(pidTgid >> 32)
|
||||
acceptCache.Store(tgid, fd, key, [4]byte{}, 0)
|
||||
@ -857,6 +985,11 @@ func consumeHTTPPlainEvents(ctx context.Context, rd *perf.Reader, mgr *correlati
|
||||
httpDstIP[2] = byte(dstIPRaw >> 8)
|
||||
httpDstIP[3] = byte(dstIPRaw)
|
||||
|
||||
// Filtrer les IPs sources ignorées (ignore_src)
|
||||
if isIgnoredIP(key.SrcIP) {
|
||||
continue
|
||||
}
|
||||
|
||||
// Extraire le payload HTTP
|
||||
if len(data) < 4110 {
|
||||
continue
|
||||
@ -903,3 +1036,122 @@ func consumeHTTPPlainEvents(ctx context.Context, rd *perf.Reader, mgr *correlati
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// applyH2Result applique le résultat du parsing H2 à la session.
|
||||
// Crée ou met à jour la requête HTTP avec les paramètres SETTINGS et en-têtes.
|
||||
func applyH2Result(s *correlation.SessionState, result *parser.H2FrameResult, dstIPFromAccept [4]byte, dstPortFromAccept uint16) {
|
||||
if result == nil {
|
||||
if len(s.Requests) == 0 {
|
||||
s.Requests = append(s.Requests, correlation.HTTPRequest{
|
||||
Timestamp: time.Now(),
|
||||
HTTPVersion: "HTTP/2",
|
||||
})
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
req := correlation.HTTPRequest{
|
||||
Timestamp: time.Now(),
|
||||
HTTPVersion: "HTTP/2",
|
||||
}
|
||||
|
||||
// Paramètres SETTINGS client
|
||||
if result.ClientSettings != nil {
|
||||
req.HTTP2Settings = &correlation.HTTP2Settings{
|
||||
HeaderTableSize: result.ClientSettings.HeaderTableSize,
|
||||
EnablePush: result.ClientSettings.EnablePush,
|
||||
MaxConcurrentStreams: result.ClientSettings.MaxConcurrentStreams,
|
||||
InitialWindowSize: result.ClientSettings.InitialWindowSize,
|
||||
MaxFrameSize: result.ClientSettings.MaxFrameSize,
|
||||
MaxHeaderListSize: result.ClientSettings.MaxHeaderListSize,
|
||||
UnknownSettings: result.ClientSettings.UnknownSettings,
|
||||
EnableConnectProtocol: result.ClientSettings.EnableConnectProtocol,
|
||||
WindowUpdateIncrement: result.ClientSettings.WindowUpdateIncrement,
|
||||
PseudoHeaderOrder: result.ClientSettings.PseudoHeaderOrder,
|
||||
}
|
||||
}
|
||||
|
||||
// En-têtes décodés
|
||||
if len(result.Headers) > 0 {
|
||||
req.HeaderKV = make(map[string]string)
|
||||
for _, h := range result.Headers {
|
||||
nameLower := strings.ToLower(h.Name)
|
||||
if parser.HpackCapturedHeaders[nameLower] && h.Value != "" {
|
||||
req.HeaderKV[nameLower] = h.Value
|
||||
req.HeaderOrder = append(req.HeaderOrder, nameLower)
|
||||
}
|
||||
switch nameLower {
|
||||
case ":method":
|
||||
req.Method = h.Value
|
||||
case ":path":
|
||||
if idx := strings.Index(h.Value, "?"); idx >= 0 {
|
||||
req.Path = h.Value[:idx]
|
||||
req.QueryString = h.Value[idx+1:]
|
||||
} else {
|
||||
req.Path = h.Value
|
||||
}
|
||||
case ":authority":
|
||||
req.Host = h.Value
|
||||
}
|
||||
}
|
||||
if len(req.HeaderOrder) > 0 {
|
||||
req.HeaderOrderSig = strings.Join(req.HeaderOrder, ";")
|
||||
}
|
||||
}
|
||||
|
||||
// Pseudo-headers order (toujours disponible via result, même sans ClientSettings)
|
||||
if len(result.PseudoHeaderOrder) > 0 {
|
||||
if req.HTTP2Settings == nil {
|
||||
req.HTTP2Settings = &correlation.HTTP2Settings{}
|
||||
}
|
||||
req.HTTP2Settings.PseudoHeaderOrder = result.PseudoHeaderOrder
|
||||
}
|
||||
|
||||
if len(s.Requests) == 0 {
|
||||
s.Requests = append(s.Requests, req)
|
||||
}
|
||||
if s.L3L4 == nil && (dstIPFromAccept != [4]byte{} || dstPortFromAccept != 0) {
|
||||
s.L3L4 = &correlation.L3L4{
|
||||
DstIP: dstIPFromAccept,
|
||||
DstPort: dstPortFromAccept,
|
||||
}
|
||||
}
|
||||
_ = s.TLS
|
||||
}
|
||||
|
||||
// updateH2Settings met à jour les paramètres HTTP/2 d'une requête existante.
|
||||
func updateH2Settings(last *correlation.HTTPRequest, settings *parser.HTTP2Settings) {
|
||||
if last.HTTP2Settings == nil {
|
||||
last.HTTP2Settings = &correlation.HTTP2Settings{
|
||||
WindowUpdateIncrement: settings.WindowUpdateIncrement,
|
||||
PseudoHeaderOrder: settings.PseudoHeaderOrder,
|
||||
}
|
||||
}
|
||||
if settings.HeaderTableSize >= 0 {
|
||||
last.HTTP2Settings.HeaderTableSize = settings.HeaderTableSize
|
||||
}
|
||||
if settings.EnablePush >= 0 {
|
||||
last.HTTP2Settings.EnablePush = settings.EnablePush
|
||||
}
|
||||
if settings.MaxConcurrentStreams >= 0 {
|
||||
last.HTTP2Settings.MaxConcurrentStreams = settings.MaxConcurrentStreams
|
||||
}
|
||||
if settings.InitialWindowSize >= 0 {
|
||||
last.HTTP2Settings.InitialWindowSize = settings.InitialWindowSize
|
||||
}
|
||||
if settings.MaxFrameSize >= 0 {
|
||||
last.HTTP2Settings.MaxFrameSize = settings.MaxFrameSize
|
||||
}
|
||||
if settings.MaxHeaderListSize >= 0 {
|
||||
last.HTTP2Settings.MaxHeaderListSize = settings.MaxHeaderListSize
|
||||
}
|
||||
if settings.UnknownSettings >= 0 {
|
||||
last.HTTP2Settings.UnknownSettings = settings.UnknownSettings
|
||||
}
|
||||
if settings.EnableConnectProtocol >= 0 {
|
||||
last.HTTP2Settings.EnableConnectProtocol = settings.EnableConnectProtocol
|
||||
}
|
||||
if len(settings.PseudoHeaderOrder) > 0 {
|
||||
last.HTTP2Settings.PseudoHeaderOrder = settings.PseudoHeaderOrder
|
||||
}
|
||||
}
|
||||
|
||||
271
services/ja4ebpf/cmd/ja4ebpf/main_test.go
Normal file
271
services/ja4ebpf/cmd/ja4ebpf/main_test.go
Normal file
@ -0,0 +1,271 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/binary"
|
||||
"net"
|
||||
"testing"
|
||||
|
||||
"github.com/antitbone/ja4/ja4ebpf/internal/loader"
|
||||
)
|
||||
|
||||
func TestParseCIDRs(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
input []string
|
||||
wantLen int
|
||||
wantErr bool
|
||||
check func(keys []loader.LPMKey) bool
|
||||
}{
|
||||
{
|
||||
"single IP",
|
||||
[]string{"192.168.1.1"},
|
||||
1, false,
|
||||
func(keys []loader.LPMKey) bool {
|
||||
return keys[0].Prefixlen == 32 && keys[0].Data == [4]byte{192, 168, 1, 1}
|
||||
},
|
||||
},
|
||||
{
|
||||
"single CIDR /24",
|
||||
[]string{"10.0.0.0/24"},
|
||||
1, false,
|
||||
func(keys []loader.LPMKey) bool {
|
||||
return keys[0].Prefixlen == 24 && keys[0].Data == [4]byte{10, 0, 0, 0}
|
||||
},
|
||||
},
|
||||
{
|
||||
"multiple CIDRs",
|
||||
[]string{"10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16"},
|
||||
3, false,
|
||||
func(keys []loader.LPMKey) bool {
|
||||
return keys[0].Prefixlen == 8 && keys[1].Prefixlen == 12 && keys[2].Prefixlen == 16
|
||||
},
|
||||
},
|
||||
{
|
||||
"IP without slash gets /32",
|
||||
[]string{"127.0.0.1"},
|
||||
1, false,
|
||||
func(keys []loader.LPMKey) bool {
|
||||
return keys[0].Prefixlen == 32
|
||||
},
|
||||
},
|
||||
{
|
||||
"whitespace trimmed",
|
||||
[]string{" 10.0.0.0/8 "},
|
||||
1, false,
|
||||
func(keys []loader.LPMKey) bool {
|
||||
return keys[0].Prefixlen == 8
|
||||
},
|
||||
},
|
||||
{
|
||||
"invalid CIDR",
|
||||
[]string{"not-a-cidr/8"},
|
||||
0, true, nil,
|
||||
},
|
||||
{
|
||||
"IPv6 ignored",
|
||||
[]string{"::1/128"},
|
||||
0, false, nil,
|
||||
},
|
||||
{
|
||||
"empty list",
|
||||
[]string{},
|
||||
0, false, nil,
|
||||
},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
keys, err := parseCIDRs(tt.input)
|
||||
if tt.wantErr {
|
||||
if err == nil {
|
||||
t.Error("expected error, got nil")
|
||||
}
|
||||
return
|
||||
}
|
||||
if err != nil {
|
||||
t.Errorf("unexpected error: %v", err)
|
||||
return
|
||||
}
|
||||
if len(keys) != tt.wantLen {
|
||||
t.Errorf("got %d keys, want %d", len(keys), tt.wantLen)
|
||||
return
|
||||
}
|
||||
if tt.check != nil && !tt.check(keys) {
|
||||
t.Errorf("check failed for keys=%v", keys)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseCIDRs_ByteOrder(t *testing.T) {
|
||||
// Verify that LPMKey.Data stores IP in network byte order (big-endian)
|
||||
keys, err := parseCIDRs([]string{"10.1.2.3/32"})
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if len(keys) != 1 {
|
||||
t.Fatalf("expected 1 key, got %d", len(keys))
|
||||
}
|
||||
|
||||
// 10.1.2.3 → data should be [10, 1, 2, 3] (network byte order)
|
||||
want := [4]byte{10, 1, 2, 3}
|
||||
if keys[0].Data != want {
|
||||
t.Errorf("LPMKey.Data = %v, want %v (network byte order)", keys[0].Data, want)
|
||||
}
|
||||
|
||||
// Verify prefixlen
|
||||
if keys[0].Prefixlen != 32 {
|
||||
t.Errorf("Prefixlen = %d, want 32", keys[0].Prefixlen)
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseCIDRs_RFC1918(t *testing.T) {
|
||||
cidrs := []string{"10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16", "127.0.0.1"}
|
||||
keys, err := parseCIDRs(cidrs)
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if len(keys) != 4 {
|
||||
t.Fatalf("expected 4 keys, got %d", len(keys))
|
||||
}
|
||||
|
||||
expected := []struct {
|
||||
prefix uint32
|
||||
data [4]byte
|
||||
}{
|
||||
{8, [4]byte{10, 0, 0, 0}},
|
||||
{12, [4]byte{172, 16, 0, 0}},
|
||||
{16, [4]byte{192, 168, 0, 0}},
|
||||
{32, [4]byte{127, 0, 0, 1}},
|
||||
}
|
||||
for i, exp := range expected {
|
||||
if keys[i].Prefixlen != exp.prefix {
|
||||
t.Errorf("keys[%d].Prefixlen = %d, want %d", i, keys[i].Prefixlen, exp.prefix)
|
||||
}
|
||||
if keys[i].Data != exp.data {
|
||||
t.Errorf("keys[%d].Data = %v, want %v", i, keys[i].Data, exp.data)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseIgnoreNets(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
input []string
|
||||
want int
|
||||
}{
|
||||
{"empty", nil, 0},
|
||||
{"single", []string{"10.0.0.0/8"}, 1},
|
||||
{"multiple", []string{"10.0.0.0/8", "192.168.0.0/16"}, 2},
|
||||
{"IP auto /32", []string{"127.0.0.1"}, 1},
|
||||
{"invalid logged", []string{"not-valid/8", "10.0.0.0/8"}, 1},
|
||||
{"IPv6 skipped", []string{"::1/128"}, 0},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
nets := parseIgnoreNets(tt.input)
|
||||
if len(nets) != tt.want {
|
||||
t.Errorf("parseIgnoreNets() = %d nets, want %d", len(nets), tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseIgnoreNets_MaskCorrect(t *testing.T) {
|
||||
nets := parseIgnoreNets([]string{"10.0.0.0/8"})
|
||||
if len(nets) != 1 {
|
||||
t.Fatalf("expected 1 net, got %d", len(nets))
|
||||
}
|
||||
ones, bits := nets[0].Mask.Size()
|
||||
if ones != 8 || bits != 32 {
|
||||
t.Errorf("mask = /%d of %d, want /8 of 32", ones, bits)
|
||||
}
|
||||
}
|
||||
|
||||
func TestIsIgnoredIP(t *testing.T) {
|
||||
// Set up global ignoreNets for testing
|
||||
origNets := ignoreNets
|
||||
defer func() { ignoreNets = origNets }()
|
||||
|
||||
ignoreNets = parseIgnoreNets([]string{"10.0.0.0/8", "192.168.0.0/16", "127.0.0.1"})
|
||||
|
||||
tests := []struct {
|
||||
ip [4]byte
|
||||
want bool
|
||||
}{
|
||||
{[4]byte{10, 0, 0, 1}, true}, // 10.x → ignored
|
||||
{[4]byte{10, 255, 255, 255}, true}, // 10.x → ignored
|
||||
{[4]byte{192, 168, 1, 1}, true}, // 192.168.x → ignored
|
||||
{[4]byte{127, 0, 0, 1}, true}, // 127.0.0.1 → ignored
|
||||
{[4]byte{8, 8, 8, 8}, false}, // public → not ignored
|
||||
{[4]byte{172, 16, 0, 1}, false}, // 172.16 not in our list → not ignored
|
||||
{[4]byte{1, 2, 3, 4}, false}, // public → not ignored
|
||||
}
|
||||
for _, tt := range tests {
|
||||
got := isIgnoredIP(tt.ip)
|
||||
if got != tt.want {
|
||||
ip := net.IPv4(tt.ip[0], tt.ip[1], tt.ip[2], tt.ip[3])
|
||||
t.Errorf("isIgnoredIP(%s) = %v, want %v", ip, got, tt.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestIsIgnoredIP_EmptyNets(t *testing.T) {
|
||||
origNets := ignoreNets
|
||||
defer func() { ignoreNets = origNets }()
|
||||
|
||||
ignoreNets = nil
|
||||
if isIgnoredIP([4]byte{10, 0, 0, 1}) {
|
||||
t.Error("isIgnoredIP should return false with empty ignoreNets")
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseTCPOptions(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
opts []byte
|
||||
wantMSS uint16
|
||||
wantWS uint8
|
||||
}{
|
||||
{"empty", nil, 0, 0xFF},
|
||||
{"MSS only", []byte{2, 4, 0x05, 0xB4}, 1460, 0xFF},
|
||||
{"WS only", []byte{3, 3, 6}, 0, 6},
|
||||
{"MSS+WS", []byte{2, 4, 0x05, 0xB4, 3, 3, 6}, 1460, 6},
|
||||
{"NOP+MSS+WS+SACK+TS", []byte{
|
||||
1, // NOP
|
||||
2, 4, 0x05, 0xB4, // MSS=1460
|
||||
3, 3, 7, // WS=7
|
||||
4, 2, // SACK
|
||||
1, // NOP
|
||||
8, 10, 0, 0, 0, 1, 0, 0, 0, 0, // TS
|
||||
}, 1460, 7},
|
||||
{"EOL", []byte{0}, 0, 0xFF},
|
||||
{"MSS only first byte", []byte{2}, 0, 0xFF}, // malformed: length byte missing
|
||||
{"truncated MSS value", []byte{2, 4, 0x05}, 0, 0xFF}, // length says 4 but only 1 byte of value
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
mss, ws := parseTCPOptions(tt.opts)
|
||||
if mss != tt.wantMSS {
|
||||
t.Errorf("mss = %d, want %d", mss, tt.wantMSS)
|
||||
}
|
||||
if ws != tt.wantWS {
|
||||
t.Errorf("windowScale = %d, want %d", ws, tt.wantWS)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseTCPOptions_MSSByteOrder(t *testing.T) {
|
||||
// MSS value 1460 = 0x05B4, big-endian in TCP options
|
||||
opts := []byte{2, 4, 0x05, 0xB4}
|
||||
mss, _ := parseTCPOptions(opts)
|
||||
if mss != 1460 {
|
||||
t.Errorf("MSS = %d, want 1460 (big-endian 0x05B4)", mss)
|
||||
}
|
||||
|
||||
// Verify it matches binary.BigEndian.Uint16
|
||||
expected := binary.BigEndian.Uint16([]byte{0x05, 0xB4})
|
||||
if mss != expected {
|
||||
t.Errorf("MSS = %d, expected from BigEndian = %d", mss, expected)
|
||||
}
|
||||
}
|
||||
@ -1,19 +1,35 @@
|
||||
# Configuration de l'agent ja4ebpf
|
||||
# Copiez ce fichier en config.yml et adaptez les valeurs.
|
||||
|
||||
# Interface réseau à surveiller (XDP ingress)
|
||||
interface: eth0
|
||||
# Interfaces réseau à surveiller (TC ingress).
|
||||
# "any" = toutes les interfaces UP (sauf loopback).
|
||||
# Ou liste explicite : ["eth0", "eth1"]
|
||||
interfaces:
|
||||
- any
|
||||
|
||||
# Chemin vers libssl pour les uprobes SSL_read/SSL_write
|
||||
# Chemin vers libssl pour les uprobes SSL_read/SSL_write/SSL_set_fd
|
||||
ssl_lib_path: "/usr/lib64/libssl.so.3"
|
||||
|
||||
# Ports TCP à surveiller (filtrage BPF côté kernel)
|
||||
listen_ports:
|
||||
- 80
|
||||
- 443
|
||||
|
||||
# CIDR/IP sources à ignorer (filtrage BPF LPM_TRIE + filtrage userspace SSL)
|
||||
# Le trafic provenant de ces réseaux est ignoré à toutes les couches.
|
||||
# ignore_src:
|
||||
# - 10.0.0.0/8
|
||||
# - 172.16.0.0/12
|
||||
# - 192.168.0.0/16
|
||||
# - 127.0.0.1
|
||||
|
||||
# Mode debug : dump compteurs BPF + événements consommés toutes les 5s
|
||||
# ClickHouse optionnel en mode debug
|
||||
debug: false
|
||||
|
||||
# Paramètres de connexion ClickHouse
|
||||
clickhouse:
|
||||
dsn: "clickhouse://default:@127.0.0.1:9000/ja4_logs"
|
||||
dsn: "clickhouse://default:@127.0.0.1:9000/ja4_logs?async_insert=0"
|
||||
batch_size: 500
|
||||
flush_secs: 1
|
||||
|
||||
|
||||
@ -27,6 +27,7 @@ type AcceptCache struct {
|
||||
mu sync.RWMutex
|
||||
cache map[acceptCacheKey]*acceptCacheEntry
|
||||
ttl time.Duration
|
||||
done chan struct{}
|
||||
}
|
||||
|
||||
// NewAcceptCache crée un cache avec la durée de vie spécifiée.
|
||||
@ -34,11 +35,17 @@ func NewAcceptCache(ttl time.Duration) *AcceptCache {
|
||||
c := &AcceptCache{
|
||||
cache: make(map[acceptCacheKey]*acceptCacheEntry),
|
||||
ttl: ttl,
|
||||
done: make(chan struct{}),
|
||||
}
|
||||
go c.purgeLoop()
|
||||
return c
|
||||
}
|
||||
|
||||
// Close arrête la goroutine de purge.
|
||||
func (c *AcceptCache) Close() {
|
||||
close(c.done)
|
||||
}
|
||||
|
||||
// Store enregistre l'association {tgid, fd} → SessionKey.
|
||||
func (c *AcceptCache) Store(tgid, fd uint32, key SessionKey, dstIP [4]byte, dstPort uint16) {
|
||||
c.mu.Lock()
|
||||
@ -65,7 +72,11 @@ func (c *AcceptCache) Lookup(tgid, fd uint32) (SessionKey, [4]byte, uint16, bool
|
||||
func (c *AcceptCache) purgeLoop() {
|
||||
ticker := time.NewTicker(30 * time.Second)
|
||||
defer ticker.Stop()
|
||||
for range ticker.C {
|
||||
for {
|
||||
select {
|
||||
case <-c.done:
|
||||
return
|
||||
case <-ticker.C:
|
||||
c.mu.Lock()
|
||||
now := time.Now()
|
||||
for k, e := range c.cache {
|
||||
@ -76,3 +87,4 @@ func (c *AcceptCache) purgeLoop() {
|
||||
c.mu.Unlock()
|
||||
}
|
||||
}
|
||||
}
|
||||
117
services/ja4ebpf/internal/correlation/accept_cache_test.go
Normal file
117
services/ja4ebpf/internal/correlation/accept_cache_test.go
Normal file
@ -0,0 +1,117 @@
|
||||
package correlation
|
||||
|
||||
import (
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestAcceptCache_StoreAndLookup(t *testing.T) {
|
||||
cache := NewAcceptCache(10 * time.Second)
|
||||
defer cache.Close()
|
||||
|
||||
key := SessionKey{SrcIP: [4]byte{10, 0, 0, 1}, SrcPort: 12345}
|
||||
dstIP := [4]byte{192, 168, 1, 1}
|
||||
dstPort := uint16(443)
|
||||
|
||||
cache.Store(100, 5, key, dstIP, dstPort)
|
||||
|
||||
gotKey, gotDstIP, gotDstPort, ok := cache.Lookup(100, 5)
|
||||
if !ok {
|
||||
t.Fatal("AcceptCache.Lookup() not found")
|
||||
}
|
||||
if gotKey != key {
|
||||
t.Errorf("Lookup key = %v, want %v", gotKey, key)
|
||||
}
|
||||
if gotDstIP != dstIP {
|
||||
t.Errorf("Lookup dstIP = %v, want %v", gotDstIP, dstIP)
|
||||
}
|
||||
if gotDstPort != dstPort {
|
||||
t.Errorf("Lookup dstPort = %d, want %d", gotDstPort, dstPort)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAcceptCache_NotFound(t *testing.T) {
|
||||
cache := NewAcceptCache(10 * time.Second)
|
||||
defer cache.Close()
|
||||
|
||||
_, _, _, ok := cache.Lookup(999, 999)
|
||||
if ok {
|
||||
t.Error("AcceptCache.Lookup() should return false for missing key")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAcceptCache_Overwrite(t *testing.T) {
|
||||
cache := NewAcceptCache(10 * time.Second)
|
||||
defer cache.Close()
|
||||
|
||||
key1 := SessionKey{SrcIP: [4]byte{10, 0, 0, 1}, SrcPort: 12345}
|
||||
key2 := SessionKey{SrcIP: [4]byte{10, 0, 0, 2}, SrcPort: 54321}
|
||||
|
||||
cache.Store(100, 5, key1, [4]byte{}, 0)
|
||||
cache.Store(100, 5, key2, [4]byte{1, 2, 3, 4}, 8080)
|
||||
|
||||
gotKey, gotDstIP, gotDstPort, ok := cache.Lookup(100, 5)
|
||||
if !ok {
|
||||
t.Fatal("AcceptCache.Lookup() not found after overwrite")
|
||||
}
|
||||
if gotKey != key2 {
|
||||
t.Errorf("Lookup key = %v, want %v (overwritten)", gotKey, key2)
|
||||
}
|
||||
if gotDstIP != [4]byte{1, 2, 3, 4} {
|
||||
t.Errorf("Lookup dstIP = %v, want [1 2 3 4]", gotDstIP)
|
||||
}
|
||||
if gotDstPort != 8080 {
|
||||
t.Errorf("Lookup dstPort = %d, want 8080", gotDstPort)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAcceptCache_Expiration(t *testing.T) {
|
||||
// Very short TTL to test expiration
|
||||
cache := NewAcceptCache(50 * time.Millisecond)
|
||||
defer cache.Close()
|
||||
|
||||
key := SessionKey{SrcIP: [4]byte{10, 0, 0, 1}, SrcPort: 12345}
|
||||
cache.Store(100, 5, key, [4]byte{}, 0)
|
||||
|
||||
// Should exist immediately
|
||||
_, _, _, ok := cache.Lookup(100, 5)
|
||||
if !ok {
|
||||
t.Fatal("AcceptCache.Lookup() should find entry before TTL")
|
||||
}
|
||||
|
||||
// Wait for expiration
|
||||
time.Sleep(80 * time.Millisecond)
|
||||
|
||||
// Should be expired now
|
||||
_, _, _, ok = cache.Lookup(100, 5)
|
||||
if ok {
|
||||
t.Error("AcceptCache.Lookup() should return false after TTL expiration")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAcceptCache_DifferentKeys(t *testing.T) {
|
||||
cache := NewAcceptCache(10 * time.Second)
|
||||
defer cache.Close()
|
||||
|
||||
key1 := SessionKey{SrcIP: [4]byte{10, 0, 0, 1}, SrcPort: 12345}
|
||||
key2 := SessionKey{SrcIP: [4]byte{10, 0, 0, 2}, SrcPort: 54321}
|
||||
|
||||
cache.Store(100, 5, key1, [4]byte{}, 0)
|
||||
cache.Store(200, 10, key2, [4]byte{1, 1, 1, 1}, 443)
|
||||
|
||||
gotKey1, _, _, ok1 := cache.Lookup(100, 5)
|
||||
if !ok1 || gotKey1 != key1 {
|
||||
t.Errorf("Lookup(100,5) = %v, %v; want %v, true", gotKey1, ok1, key1)
|
||||
}
|
||||
|
||||
gotKey2, _, _, ok2 := cache.Lookup(200, 10)
|
||||
if !ok2 || gotKey2 != key2 {
|
||||
t.Errorf("Lookup(200,10) = %v, %v; want %v, true", gotKey2, ok2, key2)
|
||||
}
|
||||
|
||||
// Cross lookup should not find
|
||||
_, _, _, ok3 := cache.Lookup(100, 10)
|
||||
if ok3 {
|
||||
t.Error("Lookup(100,10) should not find entry")
|
||||
}
|
||||
}
|
||||
@ -5,6 +5,8 @@ package correlation
|
||||
import (
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/antitbone/ja4/ja4ebpf/internal/parser"
|
||||
)
|
||||
|
||||
// SessionKey identifie une connexion TCP de façon unique.
|
||||
@ -81,6 +83,7 @@ type SessionState struct {
|
||||
TLS *TLSInfo // données TLS (peut être nil si HTTP plain)
|
||||
Requests []HTTPRequest // requêtes HTTP observées
|
||||
MaxKeepAlives int // nombre maximum de requêtes keep-alive
|
||||
H2Conn *parser.H2ConnState // état HTTP/2 par-connexion (nil pour HTTP/1.x)
|
||||
|
||||
FirstSeen time.Time // horodatage de création de la session
|
||||
LastActivity time.Time // horodatage de la dernière activité
|
||||
|
||||
64
services/ja4ebpf/internal/dispatcher/dispatcher_test.go
Normal file
64
services/ja4ebpf/internal/dispatcher/dispatcher_test.go
Normal file
@ -0,0 +1,64 @@
|
||||
package dispatcher
|
||||
|
||||
import (
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestClassify(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
data []byte
|
||||
want Protocol
|
||||
}{
|
||||
{"empty", nil, ProtoUnknown},
|
||||
{"empty slice", []byte{}, ProtoUnknown},
|
||||
{"GET request", []byte("GET / HTTP/1.1\r\n"), ProtoHTTP1},
|
||||
{"POST request", []byte("POST /api HTTP/1.1\r\n"), ProtoHTTP1},
|
||||
{"PUT request", []byte("PUT /resource HTTP/1.1\r\n"), ProtoHTTP1},
|
||||
{"DELETE request", []byte("DELETE /item HTTP/1.1\r\n"), ProtoHTTP1},
|
||||
{"HEAD request", []byte("HEAD / HTTP/1.1\r\n"), ProtoHTTP1},
|
||||
{"OPTIONS request", []byte("OPTIONS * HTTP/1.1\r\n"), ProtoHTTP1},
|
||||
{"PATCH request", []byte("PATCH /data HTTP/1.1\r\n"), ProtoHTTP1},
|
||||
{"CONNECT request", []byte("CONNECT host:443 HTTP/1.1\r\n"), ProtoHTTP1},
|
||||
{"TRACE request", []byte("TRACE / HTTP/1.1\r\n"), ProtoHTTP1},
|
||||
{"HTTP/2 preface", []byte("PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n"), ProtoHTTP2},
|
||||
{"HTTP/2 with frames", append([]byte("PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n"), []byte{0, 0, 0, 4, 0, 0, 0, 0, 0}...), ProtoHTTP2},
|
||||
{"partial H2 preface", []byte("PRI * HTTP"), ProtoHTTP2},
|
||||
{"garbage", []byte{0x15, 0x03, 0x01, 0x00}, ProtoUnknown},
|
||||
{"TLS record", []byte{0x16, 0x03, 0x01, 0x00, 0x80}, ProtoUnknown},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
got := Classify(tt.data)
|
||||
if got != tt.want {
|
||||
t.Errorf("Classify(%q) = %v, want %v", tt.data[:min(len(tt.data), 30)], got, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestClassify_PartialH2Preface(t *testing.T) {
|
||||
// A fragment that is a prefix of the H2 magic but shorter
|
||||
fragment := []byte("PRI * HTTP/2.0\r\n\r\nS")
|
||||
got := Classify(fragment)
|
||||
if got != ProtoHTTP2 {
|
||||
t.Errorf("Classify(partial H2 preface) = %v, want ProtoHTTP2", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMinInt(t *testing.T) {
|
||||
tests := []struct {
|
||||
a, b, want int
|
||||
}{
|
||||
{1, 2, 1},
|
||||
{5, 3, 3},
|
||||
{0, 0, 0},
|
||||
{-1, 1, -1},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
got := minInt(tt.a, tt.b)
|
||||
if got != tt.want {
|
||||
t.Errorf("minInt(%d, %d) = %d, want %d", tt.a, tt.b, got, tt.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -120,7 +120,9 @@ type Ja4TcMapSpecs struct {
|
||||
SslBuf *ebpf.MapSpec `ebpf:"__ssl_buf"`
|
||||
TlsBuf *ebpf.MapSpec `ebpf:"__tls_buf"`
|
||||
AcceptMap *ebpf.MapSpec `ebpf:"accept_map"`
|
||||
AllowedPorts *ebpf.MapSpec `ebpf:"allowed_ports"`
|
||||
FdConnMap *ebpf.MapSpec `ebpf:"fd_conn_map"`
|
||||
IgnoredSrc *ebpf.MapSpec `ebpf:"ignored_src"`
|
||||
PbAccept *ebpf.MapSpec `ebpf:"pb_accept"`
|
||||
PbHttpPlain *ebpf.MapSpec `ebpf:"pb_http_plain"`
|
||||
PbSslData *ebpf.MapSpec `ebpf:"pb_ssl_data"`
|
||||
@ -154,7 +156,9 @@ type Ja4TcMaps struct {
|
||||
SslBuf *ebpf.Map `ebpf:"__ssl_buf"`
|
||||
TlsBuf *ebpf.Map `ebpf:"__tls_buf"`
|
||||
AcceptMap *ebpf.Map `ebpf:"accept_map"`
|
||||
AllowedPorts *ebpf.Map `ebpf:"allowed_ports"`
|
||||
FdConnMap *ebpf.Map `ebpf:"fd_conn_map"`
|
||||
IgnoredSrc *ebpf.Map `ebpf:"ignored_src"`
|
||||
PbAccept *ebpf.Map `ebpf:"pb_accept"`
|
||||
PbHttpPlain *ebpf.Map `ebpf:"pb_http_plain"`
|
||||
PbSslData *ebpf.Map `ebpf:"pb_ssl_data"`
|
||||
@ -171,7 +175,9 @@ func (m *Ja4TcMaps) Close() error {
|
||||
m.SslBuf,
|
||||
m.TlsBuf,
|
||||
m.AcceptMap,
|
||||
m.AllowedPorts,
|
||||
m.FdConnMap,
|
||||
m.IgnoredSrc,
|
||||
m.PbAccept,
|
||||
m.PbHttpPlain,
|
||||
m.PbSslData,
|
||||
|
||||
@ -6,7 +6,7 @@ package loader
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"net"
|
||||
"log"
|
||||
"os"
|
||||
|
||||
"github.com/cilium/ebpf"
|
||||
@ -28,9 +28,11 @@ const perCPUBufferSize = 256 * 1024
|
||||
type Loader struct {
|
||||
tcObjs *Ja4TcObjects // généré par bpf2go (tc_capture.c)
|
||||
sslObjs *Ja4SslObjects // généré par bpf2go (uprobe_ssl.c)
|
||||
tcNlLink netlink.Link // interface netlink pour cleanup TC
|
||||
tcLinks []netlink.Link // interfaces netlink pour cleanup TC
|
||||
uprobeLinks []link.Link
|
||||
statsMap *ebpf.Map // map tc_stats pour lecture des compteurs BPF (mode debug)
|
||||
allowedPorts *ebpf.Map // map allowed_ports pour filtrage par port
|
||||
ignoredSrc *ebpf.Map // map ignored_src (LPM_TRIE) pour filtrage IP/CIDR
|
||||
|
||||
// SynReader lit les événements TCP SYN depuis pb_tcp_syn.
|
||||
SynReader *perf.Reader
|
||||
@ -78,6 +80,45 @@ func (l *Loader) ReadStats() (map[uint32]uint64, error) {
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// PopulatePorts remplit la map BPF allowed_ports avec les ports spécifiés.
|
||||
// Doit être appelé avant AttachTC. Chaque port autorisé reçoit la valeur 1.
|
||||
func (l *Loader) PopulatePorts(ports []uint16) error {
|
||||
if l.allowedPorts == nil {
|
||||
return fmt.Errorf("map allowed_ports non disponible")
|
||||
}
|
||||
for _, port := range ports {
|
||||
var key uint16 = port
|
||||
var val uint8 = 1
|
||||
if err := l.allowedPorts.Put(key, val); err != nil {
|
||||
return fmt.Errorf("ajout port %d dans allowed_ports: %w", port, err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// LPMKey est la clé pour BPF_MAP_TYPE_LPM_TRIE (IPv4).
|
||||
// Data est stocké en network byte order (big-endian) en mémoire
|
||||
// pour correspondre à iph.saddr dans le programme BPF.
|
||||
type LPMKey struct {
|
||||
Prefixlen uint32
|
||||
Data [4]byte // IP en network byte order
|
||||
}
|
||||
|
||||
// PopulateIgnoredSrc remplit la map BPF ignored_src (LPM_TRIE) avec les CIDR/IP à ignorer.
|
||||
// Les IPs doivent être en network byte order (big-endian) pour le LPM_TRIE.
|
||||
func (l *Loader) PopulateIgnoredSrc(cidrs []LPMKey) error {
|
||||
if l.ignoredSrc == nil {
|
||||
return fmt.Errorf("map ignored_src non disponible")
|
||||
}
|
||||
for _, key := range cidrs {
|
||||
var val uint8 = 1
|
||||
if err := l.ignoredSrc.Put(key, val); err != nil {
|
||||
return fmt.Errorf("ajout CIDR dans ignored_src: %w", err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// New charge le bytecode eBPF embarqué, supprime la limite mémoire
|
||||
// RLIMIT_MEMLOCK (requise pour les maps eBPF),
|
||||
// et retourne un Loader prêt à être attaché aux hooks.
|
||||
@ -99,28 +140,6 @@ func New() (*Loader, error) {
|
||||
return nil, fmt.Errorf("chargement objets TC eBPF: %w", err)
|
||||
}
|
||||
|
||||
// Trouver la map tc_stats par iteration des maps kernel
|
||||
var statsMap *ebpf.Map
|
||||
var mapID ebpf.MapID = 0
|
||||
for {
|
||||
nextID, err := ebpf.MapGetNextID(mapID)
|
||||
if err != nil {
|
||||
break
|
||||
}
|
||||
m, err := ebpf.NewMapFromID(nextID)
|
||||
if err != nil {
|
||||
mapID = nextID
|
||||
continue
|
||||
}
|
||||
info, err := m.Info()
|
||||
if err == nil && info.Name == "tc_stats" {
|
||||
statsMap = m
|
||||
break
|
||||
}
|
||||
m.Close()
|
||||
mapID = nextID
|
||||
}
|
||||
|
||||
// Charger les objets SSL/uprobe (uprobe_ssl.c)
|
||||
sslObjs := &Ja4SslObjects{}
|
||||
if err := LoadJa4SslObjects(sslObjs, nil); err != nil {
|
||||
@ -177,7 +196,9 @@ func New() (*Loader, error) {
|
||||
return &Loader{
|
||||
tcObjs: tcObjs,
|
||||
sslObjs: sslObjs,
|
||||
statsMap: statsMap,
|
||||
statsMap: tcObjs.TcStats,
|
||||
allowedPorts: tcObjs.AllowedPorts,
|
||||
ignoredSrc: tcObjs.IgnoredSrc,
|
||||
SynReader: synReader,
|
||||
TLSReader: tlsReader,
|
||||
SSLReader: sslReader,
|
||||
@ -190,17 +211,49 @@ func New() (*Loader, error) {
|
||||
// réseau spécifiée. Crée le qdisc clsact (idempotent) et attache le filtre BPF
|
||||
// en mode direct-action. Compatible kernel 4.1+.
|
||||
func (l *Loader) AttachTC(iface string) error {
|
||||
// Trouver l'interface par nom (standard Go net package)
|
||||
netIface, err := net.InterfaceByName(iface)
|
||||
nlLink, err := netlink.LinkByName(iface)
|
||||
if err != nil {
|
||||
return fmt.Errorf("interface réseau %q introuvable: %w", iface, err)
|
||||
}
|
||||
|
||||
// Obtenir le link netlink par index (plus fiable que par nom)
|
||||
nlLink, err := netlink.LinkByIndex(netIface.Index)
|
||||
if err != nil {
|
||||
return fmt.Errorf("netlink link index %d introuvable: %w", netIface.Index, err)
|
||||
if err := l.attachTCOnLink(nlLink); err != nil {
|
||||
return err
|
||||
}
|
||||
l.tcLinks = append(l.tcLinks, nlLink)
|
||||
return nil
|
||||
}
|
||||
|
||||
// AttachTCAll attache le programme TC ingress sur toutes les interfaces
|
||||
// réseau non-loopback et opérationnelles (OperUp).
|
||||
// Retourne la liste des noms d'interfaces attachées.
|
||||
func (l *Loader) AttachTCAll() ([]string, error) {
|
||||
links, err := netlink.LinkList()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("énumération interfaces: %w", err)
|
||||
}
|
||||
var attached []string
|
||||
for _, nlLink := range links {
|
||||
if nlLink.Type() == "loopback" {
|
||||
continue
|
||||
}
|
||||
if nlLink.Attrs().OperState != netlink.OperUp {
|
||||
continue
|
||||
}
|
||||
if err := l.attachTCOnLink(nlLink); err != nil {
|
||||
log.Printf("[loader] TC %s: %v (ignoré)", nlLink.Attrs().Name, err)
|
||||
continue
|
||||
}
|
||||
attached = append(attached, nlLink.Attrs().Name)
|
||||
l.tcLinks = append(l.tcLinks, nlLink)
|
||||
}
|
||||
if len(attached) == 0 {
|
||||
return nil, fmt.Errorf("aucune interface TC attachée")
|
||||
}
|
||||
return attached, nil
|
||||
}
|
||||
|
||||
// attachTCOnLink attache le programme TC ingress sur un link netlink donné.
|
||||
func (l *Loader) attachTCOnLink(nlLink netlink.Link) error {
|
||||
iface := nlLink.Attrs().Name
|
||||
|
||||
// Créer le qdisc clsact (idempotent via QdiscReplace)
|
||||
qdisc := &netlink.Clsact{
|
||||
@ -230,8 +283,6 @@ func (l *Loader) AttachTC(iface string) error {
|
||||
if err := netlink.FilterReplace(filter); err != nil {
|
||||
return fmt.Errorf("TC filter ingress sur %q: %w", iface, err)
|
||||
}
|
||||
|
||||
l.tcNlLink = nlLink
|
||||
return nil
|
||||
}
|
||||
|
||||
@ -328,11 +379,11 @@ func (l *Loader) Close() error {
|
||||
l.SynReader.Close()
|
||||
}
|
||||
|
||||
// Détacher le filtre TC ingress
|
||||
if l.tcNlLink != nil {
|
||||
// Détacher les filtres TC ingress sur toutes les interfaces
|
||||
for _, nlLink := range l.tcLinks {
|
||||
filter := &netlink.BpfFilter{
|
||||
FilterAttrs: netlink.FilterAttrs{
|
||||
LinkIndex: l.tcNlLink.Attrs().Index,
|
||||
LinkIndex: nlLink.Attrs().Index,
|
||||
Parent: netlink.HANDLE_MIN_INGRESS,
|
||||
Handle: 1,
|
||||
Priority: 1,
|
||||
|
||||
646
services/ja4ebpf/internal/parser/h2conn.go
Normal file
646
services/ja4ebpf/internal/parser/h2conn.go
Normal file
@ -0,0 +1,646 @@
|
||||
// Package parser fournit le parseur HTTP/2 basé sur golang.org/x/net/http2
|
||||
// et le décodeur HPACK pour l'extraction des empreintes de fingerprinting réseau.
|
||||
package parser
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"fmt"
|
||||
"io"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"golang.org/x/net/http2"
|
||||
"golang.org/x/net/http2/hpack"
|
||||
)
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Constantes HTTP/2 (RFC 9113)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// H2Magic est la préface HTTP/2 client (RFC 7540 §3.5), exportée pour usage
|
||||
// par le routeur Magic Bytes (package dispatcher) et les consommateurs RingBuffer.
|
||||
const H2Magic = "PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n"
|
||||
|
||||
// h2MagicPrefaceLen est la longueur du préambule HTTP/2 client.
|
||||
const h2MagicPrefaceLen = 24
|
||||
|
||||
// h2MagicPreface est le préambule ("PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n") envoyé
|
||||
// par tout client HTTP/2 avant la première frame SETTINGS.
|
||||
var h2MagicPreface = []byte(H2Magic)
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Types exportés
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// H2FrameRecord est un enregistrement par frame dans le résultat de ProcessFrames.
|
||||
// Fournit une chronologie fine des frames HTTP/2 avec offset logique, direction et métadonnées.
|
||||
type H2FrameRecord struct {
|
||||
Index uint32 // offset logique (incrémenté par frame dans H2ConnState)
|
||||
Direction uint8 // 0=client→serveur, 1=serveur→client
|
||||
Type http2.FrameType // type de frame (DATA, HEADERS, SETTINGS, etc.)
|
||||
Flags http2.Flags // drapeaux de la frame
|
||||
StreamID uint32 // ID du stream (0 pour les frames de connexion)
|
||||
Length uint32 // longueur du payload en octets
|
||||
}
|
||||
|
||||
// H2Priority contient les paramètres de priorité d'un stream HTTP/2 (RFC 9113 §5.3).
|
||||
type H2Priority struct {
|
||||
StreamDep uint32 // stream dépendant
|
||||
Exclusive bool // priorité exclusive
|
||||
Weight uint8 // poids (1-256)
|
||||
}
|
||||
|
||||
// HTTP2Settings contient les paramètres SETTINGS et WINDOW_UPDATE du client HTTP/2.
|
||||
type HTTP2Settings struct {
|
||||
HeaderTableSize int32 // SETTINGS_HEADER_TABLE_SIZE (-1 si absent)
|
||||
EnablePush int32 // SETTINGS_ENABLE_PUSH
|
||||
MaxConcurrentStreams int32 // SETTINGS_MAX_CONCURRENT_STREAMS
|
||||
InitialWindowSize int32 // SETTINGS_INITIAL_WINDOW_SIZE
|
||||
MaxFrameSize int32 // SETTINGS_MAX_FRAME_SIZE
|
||||
MaxHeaderListSize int32 // SETTINGS_MAX_HEADER_LIST_SIZE
|
||||
UnknownSettings int32 // paramètre 0x7 (JA4H2)
|
||||
EnableConnectProtocol int32 // SETTINGS_ENABLE_CONNECT_PROTOCOL (0x8, RFC 8441)
|
||||
WindowUpdateIncrement uint32 // valeur WINDOW_UPDATE sur stream 0
|
||||
PseudoHeaderOrder []string
|
||||
HeaderKV map[string]string // en-têtes extraits
|
||||
HeaderOrder []string // noms des en-têtes dans l'ordre d'arrivée
|
||||
}
|
||||
|
||||
// CapturedHeader est un en-tête HTTP/2 capturé avec son nom et sa valeur.
|
||||
type CapturedHeader struct {
|
||||
Name string
|
||||
Value string
|
||||
}
|
||||
|
||||
// H2FrameResult contient les données extraites d'un appel à ProcessFrames.
|
||||
type H2FrameResult struct {
|
||||
// En-têtes décodés (HEADERS + CONTINUATION assemblés)
|
||||
Headers []CapturedHeader
|
||||
HeaderStreamID uint32
|
||||
|
||||
// Paramètres SETTINGS
|
||||
ClientSettings *HTTP2Settings // non-nil si frame SETTINGS client vue
|
||||
ServerSettings *HTTP2Settings // non-nil si frame SETTINGS serveur vue
|
||||
|
||||
// WINDOW_UPDATE sur stream 0 (connexion)
|
||||
ConnWindowUpdate uint32
|
||||
|
||||
// Code de statut HTTP (:status extrait des en-têtes serveur)
|
||||
StatusCode int
|
||||
|
||||
// Streams fermés (END_STREAM ou RST_STREAM)
|
||||
StreamClosed []uint32
|
||||
|
||||
// GOAWAY
|
||||
GoAwayLastStream uint32
|
||||
GoAwayErrCode http2.ErrCode
|
||||
|
||||
// Compteurs de frames par type
|
||||
FrameCounts map[http2.FrameType]int
|
||||
|
||||
// Préface détectée
|
||||
PrefaceDetected bool
|
||||
|
||||
// Pseudo-headers extraits (ordre)
|
||||
PseudoHeaderOrder []string
|
||||
|
||||
// NOUVEAU Phase 2 : chronologie des frames de cet appel
|
||||
Frames []H2FrameRecord
|
||||
|
||||
// NOUVEAU Phase 2 : compteurs fine-grained
|
||||
SettingsAckSeen bool // SETTINGS ACK reçu dans ce batch
|
||||
PingAckSeen bool // PING ACK reçu dans ce batch
|
||||
}
|
||||
|
||||
// H2StreamState suit l'état d'un stream HTTP/2.
|
||||
type H2StreamState struct {
|
||||
ID uint32
|
||||
Initiator uint8 // 0=client (impair), 1=serveur (pair)
|
||||
State string // "idle", "open", "half-closed-local", "half-closed-remote", "closed"
|
||||
EndStream bool
|
||||
DataBytes int64
|
||||
RSTCode uint32
|
||||
Priority *H2Priority // non-nil si frame PRIORITY reçue
|
||||
WindowIncr uint32 // WINDOW_UPDATE incrément cumulé sur ce stream
|
||||
FrameTypes []http2.FrameType // historique condensé (types uniquement, pas payload)
|
||||
}
|
||||
|
||||
// H2ConnState maintient l'état par-connexion HTTP/2, incluant le décodeur HPACK.
|
||||
// Stocké dans correlation.SessionState et persisté entre les événements SSL_read.
|
||||
type H2ConnState struct {
|
||||
hdec *hpack.Decoder // décodeur HPACK par-connexion (table dynamique)
|
||||
headerBuf bytes.Buffer // fragments HEADERS+CONTINUATION en attente
|
||||
headerFragStream uint32 // stream ID des fragments en attente
|
||||
|
||||
// État de connexion
|
||||
ClientSettings *HTTP2Settings
|
||||
ServerSettings *HTTP2Settings
|
||||
FrameCounts map[http2.FrameType]int
|
||||
PrefaceSeen bool
|
||||
|
||||
// Suivi des streams
|
||||
Streams map[uint32]*H2StreamState
|
||||
|
||||
// GOAWAY
|
||||
LastStreamID uint32
|
||||
GoAwayErr http2.ErrCode
|
||||
|
||||
// NOUVEAU Phase 2
|
||||
frameIndex uint32 // compteur logique de frames (persisté entre appels)
|
||||
SettingsAck bool // SETTINGS ACK reçu (client→serveur)
|
||||
ServerAck bool // SETTINGS ACK reçu (serveur→client)
|
||||
}
|
||||
|
||||
// NewH2ConnState crée un nouvel état de connexion HTTP/2 avec un décodeur HPACK frais.
|
||||
func NewH2ConnState() *H2ConnState {
|
||||
return &H2ConnState{
|
||||
hdec: hpack.NewDecoder(4096, nil),
|
||||
FrameCounts: make(map[http2.FrameType]int),
|
||||
Streams: make(map[uint32]*H2StreamState),
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Détection du préambule HTTP/2 (fonctions utilitaires exportées)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// DetectH2Preface vérifie si le buffer commence par le préambule HTTP/2.
|
||||
func DetectH2Preface(data []byte) bool {
|
||||
if len(data) < h2MagicPrefaceLen {
|
||||
return false
|
||||
}
|
||||
for i := 0; i < h2MagicPrefaceLen; i++ {
|
||||
if data[i] != h2MagicPreface[i] {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// H2MagicPrefaceLen retourne la longueur du préambule HTTP/2.
|
||||
func H2MagicPrefaceLen() int {
|
||||
return h2MagicPrefaceLen
|
||||
}
|
||||
|
||||
// IsH2FrameHeader vérifie si les données commencent par un en-tête de frame HTTP/2 valide.
|
||||
// Utilisé comme détection rapide avant de créer un H2ConnState.
|
||||
func IsH2FrameHeader(data []byte) bool {
|
||||
if len(data) < 9 {
|
||||
return false
|
||||
}
|
||||
r := bytes.NewReader(data)
|
||||
fr := http2.NewFramer(io.Discard, r)
|
||||
fr.AllowIllegalReads = true
|
||||
_, err := fr.ReadFrame()
|
||||
return err == nil
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Traitement des frames HTTP/2
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// ProcessFrames parse les frames HTTP/2 depuis sslData via http2.Framer,
|
||||
// met à jour l'état de connexion, et retourne les en-têtes décodés.
|
||||
// direction: 0 = client→serveur, 1 = serveur→client.
|
||||
func (c *H2ConnState) ProcessFrames(data []byte, direction uint8) (*H2FrameResult, error) {
|
||||
r := bytes.NewReader(data)
|
||||
fr := http2.NewFramer(io.Discard, r)
|
||||
fr.AllowIllegalReads = true
|
||||
// NE PAS positionner ReadMetaHeaders — on gère HPACK nous-mêmes
|
||||
// pour maintenir la table dynamique par-connexion.
|
||||
|
||||
result := &H2FrameResult{
|
||||
FrameCounts: make(map[http2.FrameType]int),
|
||||
}
|
||||
|
||||
for {
|
||||
frame, err := fr.ReadFrame()
|
||||
if err != nil {
|
||||
// io.EOF ou données insuffisantes → fin du traitement
|
||||
break
|
||||
}
|
||||
|
||||
c.frameIndex++
|
||||
|
||||
// Enregistrer la frame dans le résultat
|
||||
rec := H2FrameRecord{
|
||||
Index: c.frameIndex,
|
||||
Direction: direction,
|
||||
Type: frame.Header().Type,
|
||||
Flags: http2.Flags(frame.Header().Flags),
|
||||
StreamID: frame.Header().StreamID,
|
||||
Length: frame.Header().Length,
|
||||
}
|
||||
result.Frames = append(result.Frames, rec)
|
||||
|
||||
c.FrameCounts[frame.Header().Type]++
|
||||
result.FrameCounts[frame.Header().Type]++
|
||||
|
||||
switch f := frame.(type) {
|
||||
case *http2.SettingsFrame:
|
||||
if f.IsAck() {
|
||||
if direction == 0 {
|
||||
c.SettingsAck = true
|
||||
result.SettingsAckSeen = true
|
||||
} else {
|
||||
c.ServerAck = true
|
||||
}
|
||||
} else {
|
||||
c.processSettings(f, direction, result)
|
||||
}
|
||||
case *http2.HeadersFrame:
|
||||
c.processHeaders(f, direction, result)
|
||||
case *http2.ContinuationFrame:
|
||||
c.processContinuation(f, result)
|
||||
case *http2.WindowUpdateFrame:
|
||||
c.processWindowUpdate(f, direction, result)
|
||||
case *http2.DataFrame:
|
||||
c.processData(f, direction, result)
|
||||
case *http2.PingFrame:
|
||||
if f.IsAck() {
|
||||
result.PingAckSeen = true
|
||||
}
|
||||
case *http2.GoAwayFrame:
|
||||
c.processGoAway(f, result)
|
||||
case *http2.RSTStreamFrame:
|
||||
c.processRSTStream(f, result)
|
||||
case *http2.PriorityFrame:
|
||||
c.processPriority(f, result)
|
||||
}
|
||||
}
|
||||
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Traitement des frames individuelles
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
func (c *H2ConnState) processSettings(f *http2.SettingsFrame, direction uint8, result *H2FrameResult) {
|
||||
settings := &HTTP2Settings{
|
||||
HeaderTableSize: -1,
|
||||
EnablePush: -1,
|
||||
MaxConcurrentStreams: -1,
|
||||
InitialWindowSize: -1,
|
||||
MaxFrameSize: -1,
|
||||
MaxHeaderListSize: -1,
|
||||
EnableConnectProtocol: -1,
|
||||
}
|
||||
|
||||
f.ForeachSetting(func(s http2.Setting) error {
|
||||
switch s.ID {
|
||||
case http2.SettingHeaderTableSize:
|
||||
settings.HeaderTableSize = int32(s.Val)
|
||||
// Mettre à jour la taille de la table dynamique HPACK côté client
|
||||
if direction == 0 {
|
||||
c.hdec.SetMaxDynamicTableSize(s.Val)
|
||||
}
|
||||
case http2.SettingEnablePush:
|
||||
settings.EnablePush = int32(s.Val)
|
||||
case http2.SettingMaxConcurrentStreams:
|
||||
settings.MaxConcurrentStreams = int32(s.Val)
|
||||
case http2.SettingInitialWindowSize:
|
||||
settings.InitialWindowSize = int32(s.Val)
|
||||
case http2.SettingMaxFrameSize:
|
||||
settings.MaxFrameSize = int32(s.Val)
|
||||
case http2.SettingMaxHeaderListSize:
|
||||
settings.MaxHeaderListSize = int32(s.Val)
|
||||
case http2.SettingEnableConnectProtocol:
|
||||
settings.EnableConnectProtocol = int32(s.Val)
|
||||
case 7: // paramètre non standard (JA4H2)
|
||||
settings.UnknownSettings = int32(s.Val)
|
||||
}
|
||||
return nil
|
||||
})
|
||||
|
||||
if direction == 0 {
|
||||
// Client SETTINGS → merger avec l'existant
|
||||
c.ClientSettings = mergeSettings(c.ClientSettings, settings)
|
||||
result.ClientSettings = c.ClientSettings
|
||||
} else {
|
||||
c.ServerSettings = mergeSettings(c.ServerSettings, settings)
|
||||
result.ServerSettings = c.ServerSettings
|
||||
}
|
||||
}
|
||||
|
||||
// mergeSettings fusionne les nouveaux paramètres dans les existants.
|
||||
// Les paramètres non-présents dans le nouveau gardent leur valeur existante.
|
||||
func mergeSettings(base, new *HTTP2Settings) *HTTP2Settings {
|
||||
if base == nil {
|
||||
return new
|
||||
}
|
||||
if new == nil {
|
||||
return base
|
||||
}
|
||||
// Le nouveau remplace les champs présents (valeur >= 0)
|
||||
if new.HeaderTableSize >= 0 {
|
||||
base.HeaderTableSize = new.HeaderTableSize
|
||||
}
|
||||
if new.EnablePush >= 0 {
|
||||
base.EnablePush = new.EnablePush
|
||||
}
|
||||
if new.MaxConcurrentStreams >= 0 {
|
||||
base.MaxConcurrentStreams = new.MaxConcurrentStreams
|
||||
}
|
||||
if new.InitialWindowSize >= 0 {
|
||||
base.InitialWindowSize = new.InitialWindowSize
|
||||
}
|
||||
if new.MaxFrameSize >= 0 {
|
||||
base.MaxFrameSize = new.MaxFrameSize
|
||||
}
|
||||
if new.MaxHeaderListSize >= 0 {
|
||||
base.MaxHeaderListSize = new.MaxHeaderListSize
|
||||
}
|
||||
if new.EnableConnectProtocol >= 0 {
|
||||
base.EnableConnectProtocol = new.EnableConnectProtocol
|
||||
}
|
||||
if new.UnknownSettings >= 0 {
|
||||
base.UnknownSettings = new.UnknownSettings
|
||||
}
|
||||
// Conserver les données d'en-têtes existantes si le nouveau n'en a pas
|
||||
if len(new.PseudoHeaderOrder) > 0 {
|
||||
base.PseudoHeaderOrder = new.PseudoHeaderOrder
|
||||
}
|
||||
if len(new.HeaderKV) > 0 {
|
||||
base.HeaderKV = new.HeaderKV
|
||||
}
|
||||
if len(new.HeaderOrder) > 0 {
|
||||
base.HeaderOrder = new.HeaderOrder
|
||||
}
|
||||
return base
|
||||
}
|
||||
|
||||
func (c *H2ConnState) processHeaders(f *http2.HeadersFrame, direction uint8, result *H2FrameResult) {
|
||||
streamID := f.StreamID
|
||||
|
||||
// Créer le stream si nécessaire
|
||||
if streamID > 0 {
|
||||
if _, ok := c.Streams[streamID]; !ok {
|
||||
initiator := uint8(0) // client (impair)
|
||||
if streamID%2 == 0 {
|
||||
initiator = 1 // serveur (pair)
|
||||
}
|
||||
c.Streams[streamID] = &H2StreamState{ID: streamID, Initiator: initiator, State: "open"}
|
||||
}
|
||||
stream := c.Streams[streamID]
|
||||
stream.FrameTypes = append(stream.FrameTypes, http2.FrameHeaders)
|
||||
}
|
||||
|
||||
// Buffer le fragment de bloc d'en-têtes
|
||||
c.headerBuf.Write(f.HeaderBlockFragment())
|
||||
c.headerFragStream = streamID
|
||||
|
||||
// END_STREAM sur la frame HEADERS → transition d'état
|
||||
if f.StreamEnded() {
|
||||
c.transitionStream(streamID, direction, result)
|
||||
}
|
||||
|
||||
// Si END_HEADERS est positionné, décoder immédiatement
|
||||
if f.Flags&http2.FlagHeadersEndHeaders != 0 {
|
||||
c.decodeHeaders(result)
|
||||
}
|
||||
// Sinon, attendre les frames CONTINUATION
|
||||
}
|
||||
|
||||
func (c *H2ConnState) processContinuation(f *http2.ContinuationFrame, result *H2FrameResult) {
|
||||
c.headerBuf.Write(f.HeaderBlockFragment())
|
||||
|
||||
// Si END_HEADERS est positionné, décoder le bloc complet
|
||||
if f.Flags&http2.FlagContinuationEndHeaders != 0 {
|
||||
c.decodeHeaders(result)
|
||||
}
|
||||
}
|
||||
|
||||
// decodeHeaders décode le bloc HPACK accumulé en utilisant le décodeur par-connexion.
|
||||
// La table dynamique HPACK est maintenue entre les appels.
|
||||
func (c *H2ConnState) decodeHeaders(result *H2FrameResult) {
|
||||
var headers []CapturedHeader
|
||||
|
||||
c.hdec.SetEmitFunc(func(hf hpack.HeaderField) {
|
||||
headers = append(headers, CapturedHeader{
|
||||
Name: hf.Name,
|
||||
Value: hf.Value,
|
||||
})
|
||||
})
|
||||
|
||||
c.hdec.Write(c.headerBuf.Bytes())
|
||||
c.hdec.Close() // Finalise le bloc HPACK, préserve la table dynamique
|
||||
|
||||
streamID := c.headerFragStream
|
||||
c.headerBuf.Reset()
|
||||
c.headerFragStream = 0
|
||||
|
||||
result.Headers = headers
|
||||
result.HeaderStreamID = streamID
|
||||
|
||||
// Extraire les pseudo-headers et en-têtes capturés
|
||||
var pseudoOrder []string
|
||||
kv := make(map[string]string)
|
||||
var order []string
|
||||
|
||||
for _, h := range headers {
|
||||
nameLower := strings.ToLower(h.Name)
|
||||
if strings.HasPrefix(nameLower, ":") {
|
||||
pseudoOrder = append(pseudoOrder, nameLower)
|
||||
}
|
||||
if HpackCapturedHeaders[nameLower] && h.Value != "" {
|
||||
kv[nameLower] = h.Value
|
||||
order = append(order, nameLower)
|
||||
}
|
||||
}
|
||||
|
||||
result.PseudoHeaderOrder = pseudoOrder
|
||||
|
||||
// Extraire :status de tout stream (serveur ou client)
|
||||
for _, h := range headers {
|
||||
if strings.ToLower(h.Name) == ":status" {
|
||||
if code, err := strconv.Atoi(h.Value); err == nil && code >= 100 && code <= 599 {
|
||||
result.StatusCode = code
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Si on a des en-têtes, mettre à jour les ClientSettings
|
||||
// avec les données d'en-têtes (pour la session HTTP/2)
|
||||
if streamID > 0 && len(kv) > 0 {
|
||||
isClientStream := streamID%2 == 1
|
||||
|
||||
if isClientStream {
|
||||
// Mettre à jour les en-têtes du client
|
||||
if c.ClientSettings != nil {
|
||||
if len(pseudoOrder) > 0 {
|
||||
c.ClientSettings.PseudoHeaderOrder = pseudoOrder
|
||||
}
|
||||
if c.ClientSettings.HeaderKV == nil {
|
||||
c.ClientSettings.HeaderKV = make(map[string]string)
|
||||
}
|
||||
for k, v := range kv {
|
||||
c.ClientSettings.HeaderKV[k] = v
|
||||
}
|
||||
if len(order) > 0 {
|
||||
c.ClientSettings.HeaderOrder = order
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (c *H2ConnState) processWindowUpdate(f *http2.WindowUpdateFrame, direction uint8, result *H2FrameResult) {
|
||||
if f.StreamID == 0 {
|
||||
// WINDOW_UPDATE sur le flux de connexion
|
||||
if direction == 0 {
|
||||
// Client
|
||||
if c.ClientSettings != nil {
|
||||
c.ClientSettings.WindowUpdateIncrement = f.Increment
|
||||
}
|
||||
}
|
||||
result.ConnWindowUpdate = f.Increment
|
||||
} else {
|
||||
// WINDOW_UPDATE per-stream
|
||||
if stream, ok := c.Streams[f.StreamID]; ok {
|
||||
stream.WindowIncr += f.Increment
|
||||
stream.FrameTypes = append(stream.FrameTypes, http2.FrameWindowUpdate)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (c *H2ConnState) processData(f *http2.DataFrame, direction uint8, result *H2FrameResult) {
|
||||
streamID := f.Header().StreamID
|
||||
if stream, ok := c.Streams[streamID]; ok {
|
||||
stream.DataBytes += int64(len(f.Data()))
|
||||
stream.FrameTypes = append(stream.FrameTypes, http2.FrameData)
|
||||
}
|
||||
if f.StreamEnded() {
|
||||
c.transitionStream(streamID, direction, result)
|
||||
}
|
||||
}
|
||||
|
||||
func (c *H2ConnState) processGoAway(f *http2.GoAwayFrame, result *H2FrameResult) {
|
||||
c.LastStreamID = f.LastStreamID
|
||||
c.GoAwayErr = f.ErrCode
|
||||
result.GoAwayLastStream = f.LastStreamID
|
||||
result.GoAwayErrCode = f.ErrCode
|
||||
}
|
||||
|
||||
func (c *H2ConnState) processRSTStream(f *http2.RSTStreamFrame, result *H2FrameResult) {
|
||||
streamID := f.Header().StreamID
|
||||
if stream, ok := c.Streams[streamID]; ok {
|
||||
stream.RSTCode = uint32(f.ErrCode)
|
||||
stream.State = "closed"
|
||||
stream.FrameTypes = append(stream.FrameTypes, http2.FrameRSTStream)
|
||||
}
|
||||
result.StreamClosed = append(result.StreamClosed, streamID)
|
||||
}
|
||||
|
||||
// processPriority traite les frames PRIORITY (RFC 9113 §5.3).
|
||||
func (c *H2ConnState) processPriority(f *http2.PriorityFrame, result *H2FrameResult) {
|
||||
streamID := f.Header().StreamID
|
||||
if stream, ok := c.Streams[streamID]; ok {
|
||||
stream.Priority = &H2Priority{
|
||||
StreamDep: f.PriorityParam.StreamDep,
|
||||
Exclusive: f.PriorityParam.Exclusive,
|
||||
Weight: f.PriorityParam.Weight,
|
||||
}
|
||||
stream.FrameTypes = append(stream.FrameTypes, http2.FramePriority)
|
||||
}
|
||||
}
|
||||
|
||||
// transitionStream gère les transitions d'état du stream HTTP/2 (RFC 9113 §5.1).
|
||||
func (c *H2ConnState) transitionStream(streamID uint32, direction uint8, result *H2FrameResult) {
|
||||
stream, ok := c.Streams[streamID]
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
|
||||
if stream.State == "closed" {
|
||||
return
|
||||
}
|
||||
|
||||
switch {
|
||||
case stream.State == "open" && direction == 0:
|
||||
stream.State = "half-closed-remote"
|
||||
case stream.State == "open" && direction == 1:
|
||||
stream.State = "half-closed-local"
|
||||
case stream.State == "half-closed-local" || stream.State == "half-closed-remote":
|
||||
stream.State = "closed"
|
||||
default:
|
||||
stream.State = "closed"
|
||||
}
|
||||
|
||||
stream.EndStream = true
|
||||
result.StreamClosed = append(result.StreamClosed, streamID)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// En-têtes capturés — réutilise hpackCapturedHeaders de http2.go
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Utilitaires de formatage
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// PseudoOrderToShort convertit la liste de pseudo-headers en notation abrégée.
|
||||
// Ex: [":method", ":authority", ":scheme", ":path"] → "m,a,s,p"
|
||||
func PseudoOrderToShort(headers []string) string {
|
||||
short := make([]byte, 0, len(headers)*2-1)
|
||||
for i, h := range headers {
|
||||
if i > 0 {
|
||||
short = append(short, ',')
|
||||
}
|
||||
switch {
|
||||
case h == ":method":
|
||||
short = append(short, 'm')
|
||||
case h == ":authority":
|
||||
short = append(short, 'a')
|
||||
case h == ":scheme":
|
||||
short = append(short, 's')
|
||||
case h == ":path":
|
||||
short = append(short, 'p')
|
||||
case h == ":status":
|
||||
short = append(short, 't')
|
||||
default:
|
||||
short = append(short, '?')
|
||||
}
|
||||
}
|
||||
return string(short)
|
||||
}
|
||||
|
||||
// FrameTypeString retourne le nom lisible d'un type de frame HTTP/2.
|
||||
func FrameTypeString(t http2.FrameType) string {
|
||||
switch t {
|
||||
case http2.FrameData:
|
||||
return "DATA"
|
||||
case http2.FrameHeaders:
|
||||
return "HEADERS"
|
||||
case http2.FramePriority:
|
||||
return "PRIORITY"
|
||||
case http2.FrameRSTStream:
|
||||
return "RST_STREAM"
|
||||
case http2.FrameSettings:
|
||||
return "SETTINGS"
|
||||
case http2.FramePushPromise:
|
||||
return "PUSH_PROMISE"
|
||||
case http2.FramePing:
|
||||
return "PING"
|
||||
case http2.FrameGoAway:
|
||||
return "GOAWAY"
|
||||
case http2.FrameWindowUpdate:
|
||||
return "WINDOW_UPDATE"
|
||||
case http2.FrameContinuation:
|
||||
return "CONTINUATION"
|
||||
default:
|
||||
return fmt.Sprintf("UNKNOWN(%d)", t)
|
||||
}
|
||||
}
|
||||
|
||||
// FrameCountsToString sérialise les compteurs de frames en chaîne lisible.
|
||||
func FrameCountsToString(counts map[http2.FrameType]int) string {
|
||||
if len(counts) == 0 {
|
||||
return ""
|
||||
}
|
||||
parts := make([]string, 0, len(counts))
|
||||
for t, n := range counts {
|
||||
parts = append(parts, fmt.Sprintf("%s:%d", FrameTypeString(t), n))
|
||||
}
|
||||
return strings.Join(parts, ",")
|
||||
}
|
||||
@ -1,401 +1,16 @@
|
||||
// Package parser fournit les parseurs pour les protocoles HTTP/1.x, HTTP/2 et TLS.
|
||||
//
|
||||
// Le parsing HTTP/2 est désormais assuré par internal/parser/h2conn.go qui utilise
|
||||
// golang.org/x/net/http2.Framer et golang.org/x/net/http2/hpack.Decoder pour une
|
||||
// conformité RFC complète, incluant la table dynamique HPACK et l'assemblage
|
||||
// HEADERS+CONTINUATION.
|
||||
//
|
||||
// Ce fichier ne conserve que les constantes et le filtre d'en-têtes partagés.
|
||||
package parser
|
||||
|
||||
import (
|
||||
"encoding/binary"
|
||||
"fmt"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// H2Magic est la préface HTTP/2 client (RFC 7540 §3.5), exportée pour usage
|
||||
// par le routeur Magic Bytes (package dispatcher) et les consommateurs RingBuffer.
|
||||
const H2Magic = "PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n"
|
||||
|
||||
// h2MagicPrefaceLen est la longueur du préambule HTTP/2 client.
|
||||
const h2MagicPrefaceLen = 24
|
||||
|
||||
// h2MagicPreface est le préambule ("PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n") envoyé
|
||||
// par tout client HTTP/2 avant la première frame SETTINGS.
|
||||
var h2MagicPreface = []byte(H2Magic)
|
||||
|
||||
// Identifiants de types de frames HTTP/2 (RFC 7540, §11.2).
|
||||
const (
|
||||
h2FrameData = 0
|
||||
h2FrameHeaders = 1
|
||||
h2FramePriority = 2
|
||||
h2FrameRSTStream = 3
|
||||
h2FrameSettings = 4
|
||||
h2FramePushPromise = 5
|
||||
h2FramePing = 6
|
||||
h2FrameGoAway = 7
|
||||
h2FrameWindowUpdate = 8
|
||||
h2FrameContinuation = 9
|
||||
)
|
||||
|
||||
// Identifiants des paramètres SETTINGS (RFC 7540, §11.3 + RFC 8441).
|
||||
const (
|
||||
h2SettingHeaderTableSize = 1
|
||||
h2SettingEnablePush = 2
|
||||
h2SettingMaxConcurrentStreams = 3
|
||||
h2SettingInitialWindowSize = 4
|
||||
h2SettingMaxFrameSize = 5
|
||||
h2SettingMaxHeaderListSize = 6
|
||||
h2SettingEnableConnectProtocol = 8
|
||||
)
|
||||
|
||||
// h2FrameHeader représente l'en-tête fixe de 9 octets d'une frame HTTP/2.
|
||||
type h2FrameHeader struct {
|
||||
Length uint32 // longueur du payload (3 octets)
|
||||
Type uint8 // type de frame
|
||||
Flags uint8 // flags
|
||||
StreamID uint32 // identifiant de stream (masque 0x7FFFFFFF)
|
||||
}
|
||||
|
||||
// parseH2FrameHeader décode l'en-tête de 9 octets d'une frame HTTP/2.
|
||||
func parseH2FrameHeader(data []byte) (h2FrameHeader, error) {
|
||||
if len(data) < 9 {
|
||||
return h2FrameHeader{}, fmt.Errorf("données insuffisantes pour l'en-tête frame HTTP/2: %d octets", len(data))
|
||||
}
|
||||
// Longueur sur 3 octets big-endian
|
||||
length := uint32(data[0])<<16 | uint32(data[1])<<8 | uint32(data[2])
|
||||
return h2FrameHeader{
|
||||
Length: length,
|
||||
Type: data[3],
|
||||
Flags: data[4],
|
||||
StreamID: binary.BigEndian.Uint32(data[5:9]) & 0x7FFFFFFF,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// DetectH2Preface vérifie si le buffer commence par le préambule HTTP/2.
|
||||
func DetectH2Preface(data []byte) bool {
|
||||
if len(data) < h2MagicPrefaceLen {
|
||||
return false
|
||||
}
|
||||
for i := 0; i < h2MagicPrefaceLen; i++ {
|
||||
if data[i] != h2MagicPreface[i] {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// H2MagicPrefaceLen retourne la longueur du préambule HTTP/2.
|
||||
func H2MagicPrefaceLen() int {
|
||||
return h2MagicPrefaceLen
|
||||
}
|
||||
|
||||
// HTTP2Settings contient les paramètres SETTINGS et WINDOW_UPDATE du client HTTP/2.
|
||||
type HTTP2Settings struct {
|
||||
HeaderTableSize int32 // SETTINGS_HEADER_TABLE_SIZE (-1 si absent)
|
||||
EnablePush int32 // SETTINGS_ENABLE_PUSH
|
||||
MaxConcurrentStreams int32 // SETTINGS_MAX_CONCURRENT_STREAMS
|
||||
InitialWindowSize int32 // SETTINGS_INITIAL_WINDOW_SIZE
|
||||
MaxFrameSize int32 // SETTINGS_MAX_FRAME_SIZE
|
||||
MaxHeaderListSize int32 // SETTINGS_MAX_HEADER_LIST_SIZE
|
||||
UnknownSettings int32 // paramètre 0x7 (JA4H2)
|
||||
EnableConnectProtocol int32 // SETTINGS_ENABLE_CONNECT_PROTOCOL (0x8, RFC 8441)
|
||||
WindowUpdateIncrement uint32 // valeur WINDOW_UPDATE sur stream 0
|
||||
PseudoHeaderOrder []string // ordre des pseudo-headers [:method, :authority, ...]
|
||||
HeaderKV map[string]string // en-têtes extraits du premier HEADERS frame
|
||||
HeaderOrder []string // noms des en-têtes dans l'ordre d'arrivée
|
||||
}
|
||||
|
||||
// ParseH2ClientPreface extrait les paramètres SETTINGS et le WINDOW_UPDATE
|
||||
// depuis le flux HTTP/2 déchiffré du client.
|
||||
// data doit commencer APRÈS le magic preface (offset 24).
|
||||
func ParseH2ClientPreface(data []byte) (*HTTP2Settings, error) {
|
||||
settings := &HTTP2Settings{
|
||||
HeaderTableSize: -1,
|
||||
EnablePush: -1,
|
||||
MaxConcurrentStreams: -1,
|
||||
InitialWindowSize: -1,
|
||||
MaxFrameSize: -1,
|
||||
MaxHeaderListSize: -1,
|
||||
UnknownSettings: -1,
|
||||
EnableConnectProtocol: -1,
|
||||
}
|
||||
|
||||
offset := 0
|
||||
// Parser au maximum 10 frames pour éviter une boucle infinie
|
||||
for frameIdx := 0; frameIdx < 10 && offset < len(data); frameIdx++ {
|
||||
if offset+9 > len(data) {
|
||||
break
|
||||
}
|
||||
|
||||
hdr, err := parseH2FrameHeader(data[offset:])
|
||||
if err != nil {
|
||||
break
|
||||
}
|
||||
offset += 9
|
||||
|
||||
payloadEnd := offset + int(hdr.Length)
|
||||
if payloadEnd > len(data) {
|
||||
break
|
||||
}
|
||||
payload := data[offset:payloadEnd]
|
||||
offset = payloadEnd
|
||||
|
||||
switch hdr.Type {
|
||||
case h2FrameSettings:
|
||||
// Parser uniquement les SETTINGS du client (stream 0)
|
||||
if hdr.StreamID == 0 {
|
||||
pairs, err := parseH2SettingsFrame(payload)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
for id, val := range pairs {
|
||||
switch id {
|
||||
case h2SettingHeaderTableSize:
|
||||
settings.HeaderTableSize = int32(val)
|
||||
case h2SettingEnablePush:
|
||||
settings.EnablePush = int32(val)
|
||||
case h2SettingMaxConcurrentStreams:
|
||||
settings.MaxConcurrentStreams = int32(val)
|
||||
case h2SettingInitialWindowSize:
|
||||
settings.InitialWindowSize = int32(val)
|
||||
case h2SettingMaxFrameSize:
|
||||
settings.MaxFrameSize = int32(val)
|
||||
case h2SettingMaxHeaderListSize:
|
||||
settings.MaxHeaderListSize = int32(val)
|
||||
case 7: // paramètre non standard (JA4H2)
|
||||
settings.UnknownSettings = int32(val)
|
||||
case h2SettingEnableConnectProtocol:
|
||||
settings.EnableConnectProtocol = int32(val)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
case h2FrameWindowUpdate:
|
||||
// WINDOW_UPDATE sur stream 0 = flux de connexion
|
||||
if hdr.StreamID == 0 && len(payload) >= 4 {
|
||||
settings.WindowUpdateIncrement = binary.BigEndian.Uint32(payload[0:4]) & 0x7FFFFFFF
|
||||
}
|
||||
|
||||
case h2FrameHeaders:
|
||||
// Extraire l'ordre des pseudo-headers et les en-têtes réguliers
|
||||
if hdr.StreamID > 0 && len(settings.PseudoHeaderOrder) == 0 {
|
||||
settings.PseudoHeaderOrder = ParseH2PseudoHeaders(payload)
|
||||
// Extraire aussi les en-têtes réguliers (User-Agent, Accept, etc.)
|
||||
kv, order := DecodeH2HeadersBlock(payload)
|
||||
if len(kv) > 0 {
|
||||
settings.HeaderKV = kv
|
||||
settings.HeaderOrder = order
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return settings, nil
|
||||
}
|
||||
|
||||
// parseH2SettingsFrame extrait les paires (identifiant, valeur) d'une frame SETTINGS.
|
||||
// Chaque paire fait 6 octets : identifiant(2) + valeur(4).
|
||||
func parseH2SettingsFrame(payload []byte) (map[uint16]uint32, error) {
|
||||
if len(payload)%6 != 0 {
|
||||
return nil, fmt.Errorf("longueur de frame SETTINGS invalide: %d (doit être multiple de 6)", len(payload))
|
||||
}
|
||||
result := make(map[uint16]uint32)
|
||||
for i := 0; i+6 <= len(payload); i += 6 {
|
||||
id := binary.BigEndian.Uint16(payload[i : i+2])
|
||||
val := binary.BigEndian.Uint32(payload[i+2 : i+6])
|
||||
result[id] = val
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// ParseH2PseudoHeaders extrait l'ordre des pseudo-headers depuis un bloc HPACK.
|
||||
//
|
||||
// Implémentation simplifiée : détecte les pseudo-headers via les index HPACK statiques.
|
||||
// Table statique HPACK (RFC 7541, Annexe A) — index pertinents :
|
||||
// 1 :authority
|
||||
// 2 :method = GET
|
||||
// 3 :method = POST
|
||||
// 4 :path = /
|
||||
// 5 :path = /index.html
|
||||
// 6 :scheme = http
|
||||
// 7 :scheme = https
|
||||
func ParseH2PseudoHeaders(headersBlock []byte) []string {
|
||||
// Index HPACK statique → pseudo-header
|
||||
hpackStaticPseudo := map[int]string{
|
||||
1: ":authority",
|
||||
2: ":method",
|
||||
3: ":method",
|
||||
4: ":path",
|
||||
5: ":path",
|
||||
6: ":scheme",
|
||||
7: ":scheme",
|
||||
}
|
||||
|
||||
seen := make(map[string]bool)
|
||||
var order []string
|
||||
offset := 0
|
||||
|
||||
for offset < len(headersBlock) {
|
||||
b := headersBlock[offset]
|
||||
|
||||
// Représentation indexée (bit 7 = 1) : RFC 7541 §6.1
|
||||
if b&0x80 != 0 {
|
||||
idx := int(b & 0x7F)
|
||||
if name, ok := hpackStaticPseudo[idx]; ok {
|
||||
if !seen[name] {
|
||||
seen[name] = true
|
||||
order = append(order, name)
|
||||
}
|
||||
} else if idx == 0 {
|
||||
// Fin de la liste d'index ou encodage multi-octets
|
||||
offset++
|
||||
continue
|
||||
} else {
|
||||
// Index dynamique ou non-pseudo-header : arrêter le scan
|
||||
break
|
||||
}
|
||||
offset++
|
||||
continue
|
||||
}
|
||||
|
||||
// Représentation littérale avec index incrémental (bits 7-6 = 01) : RFC 7541 §6.2.1
|
||||
if b&0xC0 == 0x40 {
|
||||
idx := int(b & 0x3F)
|
||||
if name, ok := hpackStaticPseudo[idx]; ok {
|
||||
if !seen[name] {
|
||||
seen[name] = true
|
||||
order = append(order, name)
|
||||
}
|
||||
}
|
||||
offset++
|
||||
// Sauter la valeur (longueur + contenu)
|
||||
if offset >= len(headersBlock) {
|
||||
break
|
||||
}
|
||||
valueLen := int(headersBlock[offset] & 0x7F) // ignorer le bit Huffman
|
||||
offset += 1 + valueLen
|
||||
continue
|
||||
}
|
||||
|
||||
// Tout autre encodage : arrêter (ce n'est probablement plus un pseudo-header)
|
||||
break
|
||||
}
|
||||
|
||||
return order
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// HPACK static table (RFC 7541, Appendix A) — index → header name
|
||||
// Seuls les noms sont listés (les valeurs par défaut sont ignorées car
|
||||
// les en-têtes d'intérêt comme User-Agent sont toujours envoyés en littéral).
|
||||
// ---------------------------------------------------------------------------
|
||||
// hpackStaticEntry est une entrée de la table statique HPACK (RFC 7541 Appendix A).
|
||||
type hpackStaticEntry struct {
|
||||
Name string
|
||||
Value string
|
||||
}
|
||||
|
||||
// hpackStaticTable est la table statique HPACK (RFC 7541 Appendix A).
|
||||
// Index 1-61 : RFC 7541 original. Index 62-100 : extensions RFC 9204 + navigateurs.
|
||||
var hpackStaticTable = map[int]hpackStaticEntry{
|
||||
1: {":authority", ""},
|
||||
2: {":method", "GET"},
|
||||
3: {":method", "POST"},
|
||||
4: {":path", "/"},
|
||||
5: {":path", "/index.html"},
|
||||
6: {":scheme", "http"},
|
||||
7: {":scheme", "https"},
|
||||
8: {":status", "200"},
|
||||
9: {":status", "204"},
|
||||
10: {":status", "206"},
|
||||
11: {":status", "304"},
|
||||
12: {":status", "400"},
|
||||
13: {":status", "404"},
|
||||
14: {":status", "500"},
|
||||
15: {"accept-charset", ""},
|
||||
16: {"accept-encoding", "gzip, deflate"},
|
||||
17: {"accept-language", ""},
|
||||
18: {"accept", ""},
|
||||
19: {"accept", "*/*"},
|
||||
20: {"access-control-allow-origin", ""},
|
||||
21: {"accept-encoding", ""},
|
||||
22: {"accept-encoding", "gzip, deflate"},
|
||||
23: {"accept-language", ""},
|
||||
24: {"accept-language", ""},
|
||||
25: {"access-control-allow-credentials", ""},
|
||||
26: {"access-control-allow-headers", ""},
|
||||
27: {"access-control-allow-methods", ""},
|
||||
28: {"access-control-allow-origin", ""},
|
||||
29: {"access-control-request-headers", ""},
|
||||
30: {"access-control-request-method", ""},
|
||||
31: {"age", ""},
|
||||
32: {"authorization", ""},
|
||||
33: {"cache-control", ""},
|
||||
34: {"cache-control", "max-age=0"},
|
||||
35: {"cookie", ""},
|
||||
36: {"cookie", ""},
|
||||
37: {"date", ""},
|
||||
38: {"etag", ""},
|
||||
39: {"expect", ""},
|
||||
40: {"from", ""},
|
||||
41: {"host", ""},
|
||||
42: {"if-match", ""},
|
||||
43: {"if-modified-since", ""},
|
||||
44: {"if-none-match", ""},
|
||||
45: {"if-range", ""},
|
||||
46: {"if-unmodified-since", ""},
|
||||
47: {"last-modified", ""},
|
||||
48: {"link", ""},
|
||||
49: {"location", ""},
|
||||
50: {"max-forwards", ""},
|
||||
51: {"proxy-authenticate", ""},
|
||||
52: {"proxy-authorization", ""},
|
||||
53: {"range", ""},
|
||||
54: {"referer", ""},
|
||||
55: {"refresh", ""},
|
||||
56: {"retry-after", ""},
|
||||
57: {"server", ""},
|
||||
58: {"set-cookie", ""},
|
||||
59: {"strict-transport-security", ""},
|
||||
60: {"transfer-encoding", ""},
|
||||
61: {"user-agent", ""},
|
||||
62: {"vary", ""},
|
||||
63: {"vary", "Accept-Encoding"},
|
||||
64: {"via", ""},
|
||||
65: {"www-authenticate", ""},
|
||||
66: {"x-forwarded-for", ""},
|
||||
67: {"x-forwarded-proto", ""},
|
||||
68: {"x-requested-with", ""},
|
||||
69: {"sec-websocket-key", ""},
|
||||
70: {"sec-websocket-version", ""},
|
||||
71: {"te", ""},
|
||||
72: {"upgrade", ""},
|
||||
73: {"sec-ch-ua", ""},
|
||||
74: {"sec-ch-ua-mobile", "?0"},
|
||||
75: {"sec-ch-ua-platform", ""},
|
||||
76: {"sec-fetch-dest", ""},
|
||||
77: {"sec-fetch-mode", ""},
|
||||
78: {"sec-fetch-site", ""},
|
||||
79: {"sec-fetch-user", "?1"},
|
||||
80: {"priority", ""},
|
||||
81: {"accept", ""},
|
||||
82: {"accept", "application/dns-message"},
|
||||
83: {"accept-language", ""},
|
||||
84: {":method", "CONNECT"},
|
||||
85: {":method", "DELETE"},
|
||||
86: {":method", "HEAD"},
|
||||
87: {":method", "OPTIONS"},
|
||||
88: {":method", "PATCH"},
|
||||
89: {":method", "PUT"},
|
||||
90: {":method", "TRACE"},
|
||||
91: {":path", "/"},
|
||||
92: {":path", "/0"},
|
||||
93: {":path", "/1"},
|
||||
94: {":path", "/2"},
|
||||
95: {":path", "/3"},
|
||||
96: {":path", "/4"},
|
||||
97: {":path", "/5"},
|
||||
98: {":path", "/6"},
|
||||
99: {":path", "/7"},
|
||||
100: {":path", "/8"},
|
||||
}
|
||||
|
||||
// hpackCapturedHeaders est la liste des en-têtes H2 dont on capture la valeur.
|
||||
var hpackCapturedHeaders = map[string]bool{
|
||||
// hpackCapturedHeaders est la liste des en-têtes HTTP/2 dont on capture la valeur.
|
||||
// Utilisé par h2conn.go pour filtrer les en-têtes décodés.
|
||||
var HpackCapturedHeaders = map[string]bool{
|
||||
"user-agent": true,
|
||||
"accept": true,
|
||||
"accept-encoding": true,
|
||||
@ -414,229 +29,8 @@ var hpackCapturedHeaders = map[string]bool{
|
||||
":path": true,
|
||||
":authority": true,
|
||||
":scheme": true,
|
||||
":status": true,
|
||||
"cookie": true,
|
||||
"referer": true,
|
||||
"host": true,
|
||||
}
|
||||
|
||||
// hpackInteger décode un entier HPACK avec le préfixe spécifié (RFC 7541 §5.1).
|
||||
// Retourne la valeur décodée et le nombre d'octets consommés.
|
||||
func hpackInteger(data []byte, prefixBits int) (int, int) {
|
||||
if len(data) == 0 {
|
||||
return 0, 0
|
||||
}
|
||||
mask := (1 << prefixBits) - 1
|
||||
value := int(data[0] & byte(mask))
|
||||
offset := 1
|
||||
|
||||
if value < mask {
|
||||
return value, offset
|
||||
}
|
||||
|
||||
// Extension multi-octets
|
||||
m := 0
|
||||
for offset < len(data) && offset < 6 { // limite de sécurité
|
||||
b := int(data[offset])
|
||||
value += (b & 0x7F) << m
|
||||
m += 7
|
||||
offset++
|
||||
if b&0x80 == 0 {
|
||||
break
|
||||
}
|
||||
}
|
||||
return value, offset
|
||||
}
|
||||
|
||||
// hpackString décode une chaîne HPACK (RFC 7541 §5.2).
|
||||
// Retourne la chaîne décodée et le nombre d'octets consommés.
|
||||
// Le décodage Huffman n'est pas implémenté — les chaînes Huffman sont ignorées.
|
||||
func hpackString(data []byte) (string, int) {
|
||||
if len(data) == 0 {
|
||||
return "", 0
|
||||
}
|
||||
isHuffman := data[0]&0x80 != 0
|
||||
length, offset := hpackInteger(data, 7)
|
||||
|
||||
if isHuffman {
|
||||
// Huffman non implémenté — on ne peut pas décoder la valeur
|
||||
return "", offset + length
|
||||
}
|
||||
|
||||
if offset+length > len(data) {
|
||||
// Données tronquées — retourner ce qu'on peut
|
||||
if offset < len(data) {
|
||||
return string(data[offset:]), len(data)
|
||||
}
|
||||
return "", offset
|
||||
}
|
||||
|
||||
return string(data[offset : offset+length]), offset + length
|
||||
}
|
||||
|
||||
// DecodeH2HeadersBlock décode un bloc d'en-têtes HPACK depuis un HEADERS frame.
|
||||
// Retourne un map nom→valeur et la liste ordonnée des noms.
|
||||
// Gère les représentations les plus courantes :
|
||||
// - Indexée (6.1) : index → nom+valeur de la table statique
|
||||
// - Littérale avec index incrémental (6.2.1) : nom indexé + valeur littérale
|
||||
// - Littérale sans indexation (6.2.2) : nom indexé + valeur littérale
|
||||
// - Littérale jamais indexée (6.2.3) : nom indexé + valeur littérale
|
||||
// - Nouveau nom littéral (6.2.x avec index=0) : nom littéral + valeur littérale
|
||||
func DecodeH2HeadersBlock(block []byte) (map[string]string, []string) {
|
||||
kv := make(map[string]string)
|
||||
var order []string
|
||||
offset := 0
|
||||
|
||||
for offset < len(block) && len(kv) < 50 { // limite de sécurité
|
||||
b := block[offset]
|
||||
|
||||
// 1. Représentation indexée (bit 7 = 1) : RFC 7541 §6.1
|
||||
if b&0x80 != 0 {
|
||||
idx, n := hpackInteger(block[offset:], 7)
|
||||
offset += n
|
||||
if idx > 0 && idx <= len(hpackStaticTable) {
|
||||
// Uniquement indexée — nom et valeur viennent de la table
|
||||
// Pour les entrées "nom uniquement" (pas de valeur par défaut),
|
||||
// on ne peut pas extraire la valeur sans table dynamique
|
||||
_ = hpackStaticTable[idx] // will be replaced
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
var name string
|
||||
var nameLen int
|
||||
|
||||
// 2. Littérale avec index incrémental (bits 7-6 = 01) : RFC 7541 §6.2.1
|
||||
if b&0xC0 == 0x40 {
|
||||
idx, n := hpackInteger(block[offset:], 6)
|
||||
offset += n
|
||||
|
||||
if idx == 0 {
|
||||
// Nouveau nom : nom littéral suivi de valeur littérale
|
||||
name, nameLen = hpackString(block[offset:])
|
||||
offset += nameLen
|
||||
} else if idx <= len(hpackStaticTable) {
|
||||
name = hpackStaticTable[idx].Name
|
||||
}
|
||||
|
||||
value, valueLen := hpackString(block[offset:])
|
||||
offset += valueLen
|
||||
|
||||
nameLower := strings.ToLower(name)
|
||||
if nameLower != "" && value != "" && hpackCapturedHeaders[nameLower] {
|
||||
kv[nameLower] = value
|
||||
order = append(order, nameLower)
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
// 3. Littérale sans indexation (bits 7-5 = 000) : RFC 7541 §6.2.2
|
||||
if b&0xF0 == 0x00 {
|
||||
idx, n := hpackInteger(block[offset:], 4)
|
||||
offset += n
|
||||
|
||||
if idx == 0 {
|
||||
name, nameLen = hpackString(block[offset:])
|
||||
offset += nameLen
|
||||
} else if idx <= len(hpackStaticTable) {
|
||||
name = hpackStaticTable[idx].Name
|
||||
}
|
||||
|
||||
value, valueLen := hpackString(block[offset:])
|
||||
offset += valueLen
|
||||
|
||||
nameLower := strings.ToLower(name)
|
||||
if nameLower != "" && value != "" && hpackCapturedHeaders[nameLower] {
|
||||
kv[nameLower] = value
|
||||
order = append(order, nameLower)
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
// 4. Littérale jamais indexée (bits 7-5 = 0001) : RFC 7541 §6.2.3
|
||||
if b&0xF0 == 0x10 {
|
||||
idx, n := hpackInteger(block[offset:], 4)
|
||||
offset += n
|
||||
|
||||
if idx == 0 {
|
||||
name, nameLen = hpackString(block[offset:])
|
||||
offset += nameLen
|
||||
} else if idx <= len(hpackStaticTable) {
|
||||
name = hpackStaticTable[idx].Name
|
||||
}
|
||||
|
||||
value, valueLen := hpackString(block[offset:])
|
||||
offset += valueLen
|
||||
|
||||
nameLower := strings.ToLower(name)
|
||||
if nameLower != "" && value != "" && hpackCapturedHeaders[nameLower] {
|
||||
kv[nameLower] = value
|
||||
order = append(order, nameLower)
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
// Représentation inconnue — arrêter
|
||||
break
|
||||
}
|
||||
|
||||
return kv, order
|
||||
}
|
||||
|
||||
// IsH2FrameHeader vérifie si les données commencent par un en-tête de frame HTTP/2 valide.
|
||||
// Utilisé pour détecter les frames H2 seules (sans préface) dans les SSL_read ultérieurs.
|
||||
func IsH2FrameHeader(data []byte) bool {
|
||||
if len(data) < 9 {
|
||||
return false
|
||||
}
|
||||
hdr, err := parseH2FrameHeader(data)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
// Vérifications de plausibilité :
|
||||
// - Longueur ≤ 16384 (16 KiB, limite conservatrice pour un seul read)
|
||||
// - Type dans la plage 0-9 (types de frame définis)
|
||||
// - Stream ID dans une plage raisonnable
|
||||
if hdr.Length > 16384 {
|
||||
return false
|
||||
}
|
||||
if hdr.Type > 9 {
|
||||
return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// ExtractH2HeaderKV extrait les en-têtes des frames HEADERS HTTP/2.
|
||||
// Parcourt toutes les frames dans les données et décode les blocs HEADERS.
|
||||
func ExtractH2HeaderKV(data []byte) map[string]string {
|
||||
kv := make(map[string]string)
|
||||
offset := 0
|
||||
|
||||
for offset < len(data) && len(kv) < 50 {
|
||||
if offset+9 > len(data) {
|
||||
break
|
||||
}
|
||||
hdr, err := parseH2FrameHeader(data[offset:])
|
||||
if err != nil {
|
||||
break
|
||||
}
|
||||
offset += 9
|
||||
|
||||
payloadEnd := offset + int(hdr.Length)
|
||||
if payloadEnd > len(data) {
|
||||
break
|
||||
}
|
||||
payload := data[offset:payloadEnd]
|
||||
offset = payloadEnd
|
||||
|
||||
if hdr.Type == h2FrameHeaders && hdr.StreamID > 0 {
|
||||
frameKV, _ := DecodeH2HeadersBlock(payload)
|
||||
for k, v := range frameKV {
|
||||
if _, exists := kv[k]; !exists {
|
||||
kv[k] = v
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return kv
|
||||
}
|
||||
|
||||
@ -4,6 +4,7 @@ import (
|
||||
"testing"
|
||||
|
||||
"github.com/antitbone/ja4/ja4ebpf/internal/parser"
|
||||
"golang.org/x/net/http2"
|
||||
)
|
||||
|
||||
func TestDetectH2PrefaceTrue(t *testing.T) {
|
||||
@ -33,118 +34,6 @@ func TestH2MagicPrefaceLen(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseH2ClientPrefaceSettingsEmpty(t *testing.T) {
|
||||
// Frame SETTINGS vide (longueur 0, aucun paramètre) sur stream 0
|
||||
frame := buildH2Frame(0x4, 0x0, 0, []byte{})
|
||||
settings, err := parser.ParseH2ClientPreface(frame)
|
||||
if err != nil {
|
||||
t.Fatalf("ParseH2ClientPreface: %v", err)
|
||||
}
|
||||
if settings == nil {
|
||||
t.Fatal("settings ne doit pas être nil")
|
||||
}
|
||||
// Tous les champs doivent être -1 (absent)
|
||||
if settings.HeaderTableSize != -1 {
|
||||
t.Errorf("HeaderTableSize: attendu -1, obtenu %d", settings.HeaderTableSize)
|
||||
}
|
||||
if settings.InitialWindowSize != -1 {
|
||||
t.Errorf("InitialWindowSize: attendu -1, obtenu %d", settings.InitialWindowSize)
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseH2ClientPrefaceSettingsWithValues(t *testing.T) {
|
||||
// Frame SETTINGS avec INITIAL_WINDOW_SIZE=65536 et MAX_CONCURRENT_STREAMS=100
|
||||
settingsPayload := []byte{
|
||||
0x00, 0x04, 0x00, 0x01, 0x00, 0x00, // INITIAL_WINDOW_SIZE = 65536
|
||||
0x00, 0x03, 0x00, 0x00, 0x00, 0x64, // MAX_CONCURRENT_STREAMS = 100
|
||||
}
|
||||
frame := buildH2Frame(0x4, 0x0, 0, settingsPayload)
|
||||
|
||||
settings, err := parser.ParseH2ClientPreface(frame)
|
||||
if err != nil {
|
||||
t.Fatalf("ParseH2ClientPreface: %v", err)
|
||||
}
|
||||
|
||||
if settings.InitialWindowSize != 65536 {
|
||||
t.Errorf("InitialWindowSize: attendu 65536, obtenu %d", settings.InitialWindowSize)
|
||||
}
|
||||
if settings.MaxConcurrentStreams != 100 {
|
||||
t.Errorf("MaxConcurrentStreams: attendu 100, obtenu %d", settings.MaxConcurrentStreams)
|
||||
}
|
||||
// Les paramètres non présents restent à -1
|
||||
if settings.HeaderTableSize != -1 {
|
||||
t.Errorf("HeaderTableSize non fourni: attendu -1, obtenu %d", settings.HeaderTableSize)
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseH2ClientPrefaceWindowUpdate(t *testing.T) {
|
||||
// Frame WINDOW_UPDATE sur stream 0 avec incrément = 1073741824
|
||||
wuPayload := []byte{0x40, 0x00, 0x00, 0x00} // 0x40000000 = 1073741824
|
||||
frame := buildH2Frame(0x8, 0x0, 0, wuPayload)
|
||||
|
||||
settings, err := parser.ParseH2ClientPreface(frame)
|
||||
if err != nil {
|
||||
t.Fatalf("ParseH2ClientPreface: %v", err)
|
||||
}
|
||||
if settings.WindowUpdateIncrement != 1073741824 {
|
||||
t.Errorf("WindowUpdateIncrement: attendu 1073741824, obtenu %d", settings.WindowUpdateIncrement)
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseH2ClientPrefaceCombined(t *testing.T) {
|
||||
// SETTINGS + WINDOW_UPDATE combinés (comme envoyé par curl/h2)
|
||||
settingsPayload := []byte{
|
||||
0x00, 0x01, 0x00, 0x00, 0x10, 0x00, // HEADER_TABLE_SIZE = 4096
|
||||
0x00, 0x04, 0x00, 0x00, 0xff, 0xff, // INITIAL_WINDOW_SIZE = 65535
|
||||
}
|
||||
wuPayload := []byte{0x00, 0x0f, 0x00, 0x01} // WINDOW_UPDATE incr = 983041
|
||||
|
||||
frames := buildH2Frame(0x4, 0x0, 0, settingsPayload)
|
||||
frames = append(frames, buildH2Frame(0x8, 0x0, 0, wuPayload)...)
|
||||
|
||||
settings, err := parser.ParseH2ClientPreface(frames)
|
||||
if err != nil {
|
||||
t.Fatalf("ParseH2ClientPreface: %v", err)
|
||||
}
|
||||
if settings.HeaderTableSize != 4096 {
|
||||
t.Errorf("HeaderTableSize: attendu 4096, obtenu %d", settings.HeaderTableSize)
|
||||
}
|
||||
if settings.InitialWindowSize != 65535 {
|
||||
t.Errorf("InitialWindowSize: attendu 65535, obtenu %d", settings.InitialWindowSize)
|
||||
}
|
||||
if settings.WindowUpdateIncrement != 983041 {
|
||||
t.Errorf("WindowUpdateIncrement: attendu 983041, obtenu %d", settings.WindowUpdateIncrement)
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseH2ClientPrefaceEmpty(t *testing.T) {
|
||||
// Données vides : doit retourner sans erreur, settings avec valeurs par défaut (-1)
|
||||
settings, err := parser.ParseH2ClientPreface([]byte{})
|
||||
if err != nil {
|
||||
t.Fatalf("ParseH2ClientPreface sur vide: %v", err)
|
||||
}
|
||||
if settings == nil {
|
||||
t.Error("settings ne doit pas être nil même pour données vides")
|
||||
}
|
||||
if settings.HeaderTableSize != -1 {
|
||||
t.Errorf("HeaderTableSize: attendu -1 par défaut, obtenu %d", settings.HeaderTableSize)
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseH2ClientPrefaceTruncatedFrame(t *testing.T) {
|
||||
// Frame tronquée : en-tête complet mais payload incomplet
|
||||
truncated := []byte{0x00, 0x00, 0x06, 0x04, 0x00, 0x00, 0x00, 0x00, 0x00,
|
||||
0x00, 0x01} // payload tronqué
|
||||
settings, err := parser.ParseH2ClientPreface(truncated)
|
||||
if err != nil {
|
||||
t.Fatalf("ParseH2ClientPreface sur frame tronquée: %v (doit tolérer)", err)
|
||||
}
|
||||
// Les paramètres restent à -1 car le payload est incomplet
|
||||
_ = settings
|
||||
}
|
||||
|
||||
// ── Helpers ───────────────────────────────────────────────────────────────
|
||||
|
||||
// buildH2Frame construit une frame HTTP/2 brute (en-tête 9 octets + payload).
|
||||
func buildH2Frame(frameType, flags uint8, streamID uint32, payload []byte) []byte {
|
||||
l := len(payload)
|
||||
@ -156,77 +45,254 @@ func buildH2Frame(frameType, flags uint8, streamID uint32, payload []byte) []byt
|
||||
return append(frame, payload...)
|
||||
}
|
||||
|
||||
func TestDecodeH2HeadersBlockLiteralWithIndexedName(t *testing.T) {
|
||||
// Literal with incremental indexing, indexed name (user-agent = index 61 in RFC 7541)
|
||||
// Prefix byte: 0x40 | 61 = 0x7D
|
||||
// Then value: 7-bit length "Mozilla/5.0" = 11 bytes, no Huffman
|
||||
h2block := []byte{
|
||||
0x7D, // indexed name = 61 (user-agent), with incremental indexing
|
||||
0x0B, 'M', 'o', 'z', 'i', 'l', 'l', 'a', '/', '5', '.', '0', // value length 11 + value
|
||||
// TestH2ConnStateSettings verifies that H2ConnState processes SETTINGS frames correctly.
|
||||
func TestH2ConnStateSettings(t *testing.T) {
|
||||
conn := parser.NewH2ConnState()
|
||||
|
||||
// SETTINGS frame with HEADER_TABLE_SIZE=4096, INITIAL_WINDOW_SIZE=65535
|
||||
settingsPayload := []byte{
|
||||
0x00, 0x01, 0x00, 0x00, 0x10, 0x00, // HEADER_TABLE_SIZE = 4096
|
||||
0x00, 0x04, 0x00, 0x00, 0xff, 0xff, // INITIAL_WINDOW_SIZE = 65535
|
||||
}
|
||||
kv, order := parser.DecodeH2HeadersBlock(h2block)
|
||||
if kv["user-agent"] != "Mozilla/5.0" {
|
||||
t.Errorf("user-agent: attendu 'Mozilla/5.0', obtenu %q", kv["user-agent"])
|
||||
frame := buildH2Frame(0x4, 0x0, 0, settingsPayload) // SETTINGS, no flags, stream 0
|
||||
|
||||
result, err := conn.ProcessFrames(frame, 0)
|
||||
if err != nil {
|
||||
t.Fatalf("ProcessFrames: %v", err)
|
||||
}
|
||||
if len(order) != 1 || order[0] != "user-agent" {
|
||||
t.Errorf("order: attendu [user-agent], obtenu %v", order)
|
||||
if result == nil {
|
||||
t.Fatal("result ne doit pas être nil")
|
||||
}
|
||||
if result.ClientSettings == nil {
|
||||
t.Fatal("ClientSettings ne doit pas être nil")
|
||||
}
|
||||
if result.ClientSettings.HeaderTableSize != 4096 {
|
||||
t.Errorf("HeaderTableSize: attendu 4096, obtenu %d", result.ClientSettings.HeaderTableSize)
|
||||
}
|
||||
if result.ClientSettings.InitialWindowSize != 65535 {
|
||||
t.Errorf("InitialWindowSize: attendu 65535, obtenu %d", result.ClientSettings.InitialWindowSize)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDecodeH2HeadersBlockLiteralWithoutIndexing(t *testing.T) {
|
||||
// Literal without indexing, indexed name (accept-encoding = index 16)
|
||||
// 4-bit prefix max = 15, so index 16 needs multi-byte: 0x0F 0x01
|
||||
h2block := []byte{
|
||||
0x0F, 0x01, // literal without indexing, name index = 16 (accept-encoding)
|
||||
0x12, 'g', 'z', 'i', 'p', ',', ' ', 'd', 'e', 'f', 'l', 'a', 't', 'e', ',', ' ', 'b', 'r', // value
|
||||
// TestH2ConnStateWindowUpdate verifies WINDOW_UPDATE on stream 0.
|
||||
func TestH2ConnStateWindowUpdate(t *testing.T) {
|
||||
conn := parser.NewH2ConnState()
|
||||
|
||||
// WINDOW_UPDATE on stream 0 with increment = 1073741824 (0x40000000)
|
||||
wuPayload := []byte{0x40, 0x00, 0x00, 0x00}
|
||||
frame := buildH2Frame(0x8, 0x0, 0, wuPayload) // WINDOW_UPDATE, stream 0
|
||||
|
||||
result, err := conn.ProcessFrames(frame, 0)
|
||||
if err != nil {
|
||||
t.Fatalf("ProcessFrames: %v", err)
|
||||
}
|
||||
kv, _ := parser.DecodeH2HeadersBlock(h2block)
|
||||
if kv["accept-encoding"] != "gzip, deflate, br" {
|
||||
t.Errorf("accept-encoding: attendu 'gzip, deflate, br', obtenu %q", kv["accept-encoding"])
|
||||
if result == nil {
|
||||
t.Fatal("result ne doit pas être nil")
|
||||
}
|
||||
if result.ConnWindowUpdate != 1073741824 {
|
||||
t.Errorf("WindowUpdateIncrement: attendu 1073741824, obtenu %d", result.ConnWindowUpdate)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDecodeH2HeadersBlockLiteralNewName(t *testing.T) {
|
||||
// Literal with incremental indexing, new name
|
||||
// Prefix byte: 0x40 (index = 0, new name)
|
||||
// Name: "x-custom-header", Value: "test-value"
|
||||
name := "x-custom-header"
|
||||
value := "test-value"
|
||||
h2block := []byte{
|
||||
0x40, // literal with incremental indexing, new name
|
||||
byte(len(name)), // name length
|
||||
}
|
||||
h2block = append(h2block, []byte(name)...)
|
||||
h2block = append(h2block, byte(len(value)))
|
||||
h2block = append(h2block, []byte(value)...)
|
||||
// TestH2ConnStateHeadersWithHPACK verifies HEADERS frame decoding via hpack.Decoder.
|
||||
func TestH2ConnStateHeadersWithHPACK(t *testing.T) {
|
||||
conn := parser.NewH2ConnState()
|
||||
|
||||
kv, order := parser.DecodeH2HeadersBlock(h2block)
|
||||
// x-custom-header is not in hpackCapturedHeaders, so it won't be in kv
|
||||
if len(kv) != 0 {
|
||||
t.Errorf("x-custom-header ne doit pas être capturé (pas dans hpackCapturedHeaders), obtenu %v", kv)
|
||||
// HEADERS frame with END_HEADERS flag:
|
||||
// 0x82 = :method GET (indexed)
|
||||
// 0x84 = :path / (indexed)
|
||||
// 0x41 = :authority with literal value "example.com"
|
||||
headersPayload := []byte{
|
||||
0x82, // :method GET
|
||||
0x84, // :path /
|
||||
0x41, // :authority with literal value
|
||||
0x0B, 'e', 'x', 'a', 'm', 'p', 'l', 'e', '.', 'c', 'o', 'm',
|
||||
}
|
||||
_ = order
|
||||
frame := buildH2Frame(0x1, 0x04, 1, headersPayload) // HEADERS, END_HEADERS, stream 1
|
||||
|
||||
result, err := conn.ProcessFrames(frame, 0)
|
||||
if err != nil {
|
||||
t.Fatalf("ProcessFrames: %v", err)
|
||||
}
|
||||
if result == nil {
|
||||
t.Fatal("result ne doit pas être nil")
|
||||
}
|
||||
|
||||
func TestDecodeH2HeadersBlockPseudoHeaders(t *testing.T) {
|
||||
// Pseudo-headers :method GET (indexed, byte 0x82), :path / (indexed, byte 0x84)
|
||||
// Then :authority as literal with indexed name (index 1)
|
||||
// 0x40 | 1 = 0x41, then value "example.com"
|
||||
h2block := []byte{
|
||||
0x82, // indexed :method GET
|
||||
0x84, // indexed :path /
|
||||
0x41, // literal with incremental indexing, name index 1 (:authority)
|
||||
0x0B, 'e', 'x', 'a', 'm', 'p', 'l', 'e', '.', 'c', 'o', 'm', // value
|
||||
// Check headers
|
||||
headerMap := make(map[string]string)
|
||||
for _, h := range result.Headers {
|
||||
headerMap[h.Name] = h.Value
|
||||
}
|
||||
kv, order := parser.DecodeH2HeadersBlock(h2block)
|
||||
if kv[":authority"] != "example.com" {
|
||||
t.Errorf(":authority: attendu 'example.com', obtenu %q", kv[":authority"])
|
||||
if headerMap[":method"] != "GET" {
|
||||
t.Errorf(":method: attendu 'GET', obtenu %q", headerMap[":method"])
|
||||
}
|
||||
if len(order) < 1 {
|
||||
t.Errorf("order ne doit pas être vide, obtenu %v", order)
|
||||
if headerMap[":path"] != "/" {
|
||||
t.Errorf(":path: attendu '/', obtenu %q", headerMap[":path"])
|
||||
}
|
||||
if headerMap[":authority"] != "example.com" {
|
||||
t.Errorf(":authority: attendu 'example.com', obtenu %q", headerMap[":authority"])
|
||||
}
|
||||
}
|
||||
|
||||
// TestH2ConnStateHeadersFullyIndexed verifies fully-indexed HPACK representations.
|
||||
func TestH2ConnStateHeadersFullyIndexed(t *testing.T) {
|
||||
conn := parser.NewH2ConnState()
|
||||
|
||||
// All fully-indexed: :method GET, :scheme https, :path /, accept */*
|
||||
// Note: Go's hpack static table has index 19 as accept="" (no default value),
|
||||
// unlike RFC 7541 which defines it as accept: */*. We test actual Go behavior.
|
||||
headersPayload := []byte{
|
||||
0x82, // :method GET
|
||||
0x87, // :scheme https
|
||||
0x84, // :path /
|
||||
0x93, // accept (Go hpack: empty value; RFC 7541: */*)
|
||||
}
|
||||
frame := buildH2Frame(0x1, 0x04, 1, headersPayload)
|
||||
|
||||
result, err := conn.ProcessFrames(frame, 0)
|
||||
if err != nil {
|
||||
t.Fatalf("ProcessFrames: %v", err)
|
||||
}
|
||||
|
||||
headerMap := make(map[string]string)
|
||||
for _, h := range result.Headers {
|
||||
headerMap[h.Name] = h.Value
|
||||
}
|
||||
if headerMap[":method"] != "GET" {
|
||||
t.Errorf(":method: attendu 'GET', obtenu %q", headerMap[":method"])
|
||||
}
|
||||
if headerMap[":scheme"] != "https" {
|
||||
t.Errorf(":scheme: attendu 'https', obtenu %q", headerMap[":scheme"])
|
||||
}
|
||||
if headerMap[":path"] != "/" {
|
||||
t.Errorf(":path: attendu '/', obtenu %q", headerMap[":path"])
|
||||
}
|
||||
// Go's hpack emits accept="" for index 19 — verify it's present but empty
|
||||
if _, ok := headerMap["accept"]; !ok {
|
||||
t.Error("accept: header attendu mais absent")
|
||||
}
|
||||
}
|
||||
|
||||
// TestH2ConnStatePrefaceAndSettings verifies processing of H2 preface followed by SETTINGS.
|
||||
func TestH2ConnStatePrefaceAndSettings(t *testing.T) {
|
||||
// Client preface: magic + SETTINGS frame
|
||||
preface := []byte("PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n")
|
||||
|
||||
// SETTINGS with INITIAL_WINDOW_SIZE=65536 and MAX_CONCURRENT_STREAMS=100
|
||||
settingsPayload := []byte{
|
||||
0x00, 0x04, 0x00, 0x00, 0xff, 0xff, // INITIAL_WINDOW_SIZE = 65535
|
||||
0x00, 0x03, 0x00, 0x00, 0x00, 0x64, // MAX_CONCURRENT_STREAMS = 100
|
||||
}
|
||||
settingsFrame := buildH2Frame(0x4, 0x0, 0, settingsPayload)
|
||||
|
||||
data := append(preface, settingsFrame...)
|
||||
|
||||
// Detect preface and process remaining bytes
|
||||
afterPreface := data[parser.H2MagicPrefaceLen():]
|
||||
conn := parser.NewH2ConnState()
|
||||
result, err := conn.ProcessFrames(afterPreface, 0)
|
||||
if err != nil {
|
||||
t.Fatalf("ProcessFrames: %v", err)
|
||||
}
|
||||
if result == nil || result.ClientSettings == nil {
|
||||
t.Fatal("ClientSettings ne doit pas être nil")
|
||||
}
|
||||
if result.ClientSettings.InitialWindowSize != 65535 {
|
||||
t.Errorf("InitialWindowSize: attendu 65535, obtenu %d", result.ClientSettings.InitialWindowSize)
|
||||
}
|
||||
if result.ClientSettings.MaxConcurrentStreams != 100 {
|
||||
t.Errorf("MaxConcurrentStreams: attendu 100, obtenu %d", result.ClientSettings.MaxConcurrentStreams)
|
||||
}
|
||||
}
|
||||
|
||||
// TestH2ConnStateDynamicTable verifies that HPACK dynamic table works across multiple HEADERS frames.
|
||||
func TestH2ConnStateDynamicTable(t *testing.T) {
|
||||
conn := parser.NewH2ConnState()
|
||||
|
||||
// First HEADERS frame: :method GET, :authority example.com (literal with indexing)
|
||||
// This adds "example.com" to the dynamic table
|
||||
headers1 := []byte{
|
||||
0x82, // :method GET (indexed)
|
||||
0x41, // :authority with literal value (indexed in dynamic table)
|
||||
0x0B, 'e', 'x', 'a', 'm', 'p', 'l', 'e', '.', 'c', 'o', 'm',
|
||||
}
|
||||
frame1 := buildH2Frame(0x1, 0x04, 1, headers1)
|
||||
|
||||
result1, _ := conn.ProcessFrames(frame1, 0)
|
||||
if result1 == nil {
|
||||
t.Fatal("result1 ne doit pas être nil")
|
||||
}
|
||||
headerMap1 := make(map[string]string)
|
||||
for _, h := range result1.Headers {
|
||||
headerMap1[h.Name] = h.Value
|
||||
}
|
||||
if headerMap1[":authority"] != "example.com" {
|
||||
t.Errorf("first frame: :authority attendu 'example.com', obtenu %q", headerMap1[":authority"])
|
||||
}
|
||||
|
||||
// Second HEADERS frame on stream 3: :method GET, :authority example.com (now in dynamic table)
|
||||
// After adding "example.com" with index 62 in dynamic table, we can reference it
|
||||
// However, for a simple test, we just verify the decoder still works
|
||||
headers2 := []byte{
|
||||
0x82, // :method GET (indexed)
|
||||
0x84, // :path / (indexed)
|
||||
}
|
||||
frame2 := buildH2Frame(0x1, 0x04, 3, headers2)
|
||||
|
||||
result2, _ := conn.ProcessFrames(frame2, 0)
|
||||
if result2 == nil {
|
||||
t.Fatal("result2 ne doit pas être nil")
|
||||
}
|
||||
headerMap2 := make(map[string]string)
|
||||
for _, h := range result2.Headers {
|
||||
headerMap2[h.Name] = h.Value
|
||||
}
|
||||
if headerMap2[":method"] != "GET" {
|
||||
t.Errorf("second frame: :method attendu 'GET', obtenu %q", headerMap2[":method"])
|
||||
}
|
||||
if headerMap2[":path"] != "/" {
|
||||
t.Errorf("second frame: :path attendu '/', obtenu %q", headerMap2[":path"])
|
||||
}
|
||||
}
|
||||
|
||||
// TestH2ConnStateServerStatus verifies :status extraction from server HEADERS.
|
||||
func TestH2ConnStateServerStatus(t *testing.T) {
|
||||
conn := parser.NewH2ConnState()
|
||||
|
||||
// Server HEADERS frame with :status 200 (indexed, byte 0x88)
|
||||
headersPayload := []byte{0x88} // :status 200
|
||||
frame := buildH2Frame(0x1, 0x04, 1, headersPayload)
|
||||
|
||||
result, err := conn.ProcessFrames(frame, 1) // direction=1 (server→client)
|
||||
if err != nil {
|
||||
t.Fatalf("ProcessFrames: %v", err)
|
||||
}
|
||||
if result.StatusCode != 200 {
|
||||
t.Errorf("StatusCode: attendu 200, obtenu %d", result.StatusCode)
|
||||
}
|
||||
}
|
||||
|
||||
// TestH2ConnStateGoAway verifies GOAWAY frame processing.
|
||||
func TestH2ConnStateGoAway(t *testing.T) {
|
||||
conn := parser.NewH2ConnState()
|
||||
|
||||
// GOAWAY frame: last stream ID = 0, error code = NO_ERROR (0)
|
||||
goawayPayload := []byte{
|
||||
0x00, 0x00, 0x00, 0x00, // last stream ID = 0
|
||||
0x00, 0x00, 0x00, 0x00, // error code = NO_ERROR
|
||||
}
|
||||
frame := buildH2Frame(0x7, 0x0, 0, goawayPayload) // GOAWAY, stream 0
|
||||
|
||||
result, err := conn.ProcessFrames(frame, 1)
|
||||
if err != nil {
|
||||
t.Fatalf("ProcessFrames: %v", err)
|
||||
}
|
||||
if result.GoAwayLastStream != 0 {
|
||||
t.Errorf("GoAwayLastStream: attendu 0, obtenu %d", result.GoAwayLastStream)
|
||||
}
|
||||
}
|
||||
|
||||
// TestIsH2FrameHeader verifies frame detection using http2.Framer.
|
||||
func TestIsH2FrameHeader(t *testing.T) {
|
||||
// Frame SETTINGS valide
|
||||
frame := buildH2Frame(0x4, 0x0, 0, []byte{})
|
||||
@ -236,7 +302,7 @@ func TestIsH2FrameHeader(t *testing.T) {
|
||||
// Données aléatoires
|
||||
random := []byte{0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF}
|
||||
if parser.IsH2FrameHeader(random) {
|
||||
t.Error("IsH2FrameHeader doit retourner false pour données invalides (length > 16384)")
|
||||
t.Error("IsH2FrameHeader doit retourner false pour données invalides")
|
||||
}
|
||||
// Trop court
|
||||
if parser.IsH2FrameHeader([]byte{0x00, 0x00}) {
|
||||
@ -244,29 +310,426 @@ func TestIsH2FrameHeader(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestExtractH2HeaderKV(t *testing.T) {
|
||||
// HEADERS frame with :authority literal
|
||||
// TestH2ConnStateRSTStream verifies RST_STREAM frame processing.
|
||||
func TestH2ConnStateRSTStream(t *testing.T) {
|
||||
conn := parser.NewH2ConnState()
|
||||
|
||||
// RST_STREAM on stream 1 with error code CANCEL (0x08)
|
||||
rstPayload := []byte{0x00, 0x00, 0x00, 0x08} // error code CANCEL
|
||||
frame := buildH2Frame(0x3, 0x0, 1, rstPayload) // RST_STREAM, stream 1
|
||||
|
||||
result, _ := conn.ProcessFrames(frame, 1)
|
||||
if result == nil {
|
||||
t.Fatal("result ne doit pas être nil")
|
||||
}
|
||||
// Check that stream 1 is in the closed streams
|
||||
found := false
|
||||
for _, id := range result.StreamClosed {
|
||||
if id == 1 {
|
||||
found = true
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
t.Error("stream 1 devrait être dans StreamClosed après RST_STREAM")
|
||||
}
|
||||
}
|
||||
|
||||
// TestHpackDecoderBasic verifies the hpack.Decoder works correctly via H2ConnState.
|
||||
func TestHpackDecoderBasic(t *testing.T) {
|
||||
// Create an H2ConnState and feed it a SETTINGS frame first (to set dynamic table size)
|
||||
conn := parser.NewH2ConnState()
|
||||
|
||||
// SETTINGS with HEADER_TABLE_SIZE=4096
|
||||
settingsPayload := []byte{
|
||||
0x00, 0x01, 0x00, 0x00, 0x10, 0x00, // HEADER_TABLE_SIZE = 4096
|
||||
}
|
||||
settingsFrame := buildH2Frame(0x4, 0x0, 0, settingsPayload)
|
||||
|
||||
result, _ := conn.ProcessFrames(settingsFrame, 0)
|
||||
if result.ClientSettings == nil || result.ClientSettings.HeaderTableSize != 4096 {
|
||||
t.Errorf("HEADER_TABLE_SIZE: attendu 4096")
|
||||
}
|
||||
|
||||
// Now feed a HEADERS frame with user-agent (literal with indexed name)
|
||||
// user-agent is index 58 in HPACK static table
|
||||
// 0x40 | 58 = 0x7A, then value length 8, then "curl/8.0"
|
||||
uaPayload := []byte{
|
||||
0x82, // :method GET
|
||||
0x7A, // user-agent with literal value (indexed name 58)
|
||||
0x08, 'c', 'u', 'r', 'l', '/', '8', '.', '0',
|
||||
}
|
||||
uaFrame := buildH2Frame(0x1, 0x04, 1, uaPayload)
|
||||
|
||||
result2, _ := conn.ProcessFrames(uaFrame, 0)
|
||||
headerMap := make(map[string]string)
|
||||
for _, h := range result2.Headers {
|
||||
headerMap[h.Name] = h.Value
|
||||
}
|
||||
if headerMap["user-agent"] != "curl/8.0" {
|
||||
t.Errorf("user-agent: attendu 'curl/8.0', obtenu %q", headerMap["user-agent"])
|
||||
}
|
||||
if headerMap[":method"] != "GET" {
|
||||
t.Errorf(":method: attendu 'GET', obtenu %q", headerMap[":method"])
|
||||
}
|
||||
}
|
||||
|
||||
// TestH2ConnStateContinuation verifies HEADERS + CONTINUATION assembly.
|
||||
func TestH2ConnStateContinuation(t *testing.T) {
|
||||
conn := parser.NewH2ConnState()
|
||||
|
||||
// HEADERS frame WITHOUT END_HEADERS (flags=0x00, stream 1)
|
||||
headersPayload := []byte{
|
||||
0x41, // literal with incremental indexing, name index 1 (:authority)
|
||||
0x07, 'e', 'x', 'a', 'm', 'p', 'l', 'e', // value
|
||||
0x82, // :method GET
|
||||
0x84, // :path /
|
||||
}
|
||||
frame := buildH2Frame(0x1, 0x04, 1, headersPayload) // HEADERS, END_HEADERS, stream 1
|
||||
headersFrame := buildH2Frame(0x1, 0x00, 1, headersPayload) // HEADERS, NO END_HEADERS
|
||||
|
||||
kv := parser.ExtractH2HeaderKV(frame)
|
||||
if kv[":authority"] != "example" {
|
||||
t.Errorf(":authority: attendu 'example', obtenu %q", kv[":authority"])
|
||||
// CONTINUATION frame WITH END_HEADERS (flags=0x04, stream 1)
|
||||
contPayload := []byte{
|
||||
0x41, // :authority with literal value
|
||||
0x07, 'e', 'x', 'a', 'm', 'p', 'l', 'e',
|
||||
}
|
||||
contFrame := buildH2Frame(0x9, 0x04, 1, contPayload) // CONTINUATION, END_HEADERS
|
||||
|
||||
// Process both frames in one call
|
||||
data := append(headersFrame, contFrame...)
|
||||
result, _ := conn.ProcessFrames(data, 0)
|
||||
|
||||
if result == nil {
|
||||
t.Fatal("result ne doit pas être nil")
|
||||
}
|
||||
headerMap := make(map[string]string)
|
||||
for _, h := range result.Headers {
|
||||
headerMap[h.Name] = h.Value
|
||||
}
|
||||
if headerMap[":method"] != "GET" {
|
||||
t.Errorf(":method: attendu 'GET', obtenu %q", headerMap[":method"])
|
||||
}
|
||||
if headerMap[":path"] != "/" {
|
||||
t.Errorf(":path: attendu '/', obtenu %q", headerMap[":path"])
|
||||
}
|
||||
if headerMap[":authority"] != "example" {
|
||||
t.Errorf(":authority: attendu 'example', obtenu %q", headerMap[":authority"])
|
||||
}
|
||||
}
|
||||
|
||||
func TestFormatTCPOptions(t *testing.T) {
|
||||
// MSS(2,4bytes) + WS(3,3bytes) + SACK(4,2bytes) + NOP(1) + TS(8,10bytes)
|
||||
opts := []byte{
|
||||
2, 4, 0x05, 0xB4, // MSS = 1460
|
||||
3, 3, 6, // WS = 6
|
||||
4, 2, // SACK Permitted
|
||||
1, // NOP
|
||||
8, 10, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // TS
|
||||
// TestH2ConnStatePing verifies PING frame counting.
|
||||
func TestH2ConnStatePing(t *testing.T) {
|
||||
conn := parser.NewH2ConnState()
|
||||
|
||||
// PING frame (8 bytes opaque data)
|
||||
pingPayload := make([]byte, 8)
|
||||
frame := buildH2Frame(0x6, 0x0, 0, pingPayload) // PING, stream 0
|
||||
|
||||
result, _ := conn.ProcessFrames(frame, 0)
|
||||
if result == nil {
|
||||
t.Fatal("result ne doit pas être nil")
|
||||
}
|
||||
count, ok := result.FrameCounts[http2.FramePing]
|
||||
if !ok || count != 1 {
|
||||
t.Errorf("PING frame count: attendu 1, obtenu %d", count)
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Phase 2 tests
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// TestH2ConnStateSettingsAck verifies SETTINGS ACK detection.
|
||||
func TestH2ConnStateSettingsAck(t *testing.T) {
|
||||
conn := parser.NewH2ConnState()
|
||||
|
||||
// SETTINGS ACK frame (ACK flag = 0x01, no payload)
|
||||
ackFrame := buildH2Frame(0x4, 0x01, 0, []byte{}) // SETTINGS, ACK flag
|
||||
|
||||
result, err := conn.ProcessFrames(ackFrame, 0)
|
||||
if err != nil {
|
||||
t.Fatalf("ProcessFrames: %v", err)
|
||||
}
|
||||
if !result.SettingsAckSeen {
|
||||
t.Error("SettingsAckSeen devrait être true après SETTINGS ACK")
|
||||
}
|
||||
if !conn.SettingsAck {
|
||||
t.Error("H2ConnState.SettingsAck devrait être true après SETTINGS ACK")
|
||||
}
|
||||
}
|
||||
|
||||
// TestH2ConnStatePingAck verifies PING ACK flag distinction.
|
||||
func TestH2ConnStatePingAck(t *testing.T) {
|
||||
conn := parser.NewH2ConnState()
|
||||
|
||||
// PING ACK frame (ACK flag = 0x01)
|
||||
pingPayload := make([]byte, 8)
|
||||
ackFrame := buildH2Frame(0x6, 0x01, 0, pingPayload) // PING, ACK flag
|
||||
|
||||
result, err := conn.ProcessFrames(ackFrame, 0)
|
||||
if err != nil {
|
||||
t.Fatalf("ProcessFrames: %v", err)
|
||||
}
|
||||
if !result.PingAckSeen {
|
||||
t.Error("PingAckSeen devrait être true après PING ACK")
|
||||
}
|
||||
|
||||
// Regular PING should NOT set PingAckSeen
|
||||
conn2 := parser.NewH2ConnState()
|
||||
regularPing := buildH2Frame(0x6, 0x0, 0, pingPayload) // PING, no ACK
|
||||
result2, _ := conn2.ProcessFrames(regularPing, 0)
|
||||
if result2.PingAckSeen {
|
||||
t.Error("PingAckSeen ne devrait pas être true pour un PING régulier")
|
||||
}
|
||||
}
|
||||
|
||||
// TestH2ConnStatePriority verifies PRIORITY frame decoding.
|
||||
func TestH2ConnStatePriority(t *testing.T) {
|
||||
conn := parser.NewH2ConnState()
|
||||
|
||||
// Create stream 1 first (HEADERS)
|
||||
headersPayload := []byte{0x82, 0x84} // :method GET, :path /
|
||||
headersFrame := buildH2Frame(0x1, 0x04, 1, headersPayload)
|
||||
conn.ProcessFrames(headersFrame, 0)
|
||||
|
||||
// PRIORITY frame on stream 1: StreamDep=0, Exclusive=false, Weight=15
|
||||
// PRIORITY payload: 4 bytes (stream dep + exclusive bit) + 1 byte weight
|
||||
priorityPayload := []byte{
|
||||
0x00, 0x00, 0x00, 0x00, // StreamDep=0, Exclusive=false (bit 31 = 0)
|
||||
0x0F, // Weight=15
|
||||
}
|
||||
priorityFrame := buildH2Frame(0x2, 0x0, 1, priorityPayload) // PRIORITY, stream 1
|
||||
|
||||
_, err := conn.ProcessFrames(priorityFrame, 0)
|
||||
if err != nil {
|
||||
t.Fatalf("ProcessFrames: %v", err)
|
||||
}
|
||||
|
||||
stream, ok := conn.Streams[1]
|
||||
if !ok {
|
||||
t.Fatal("stream 1 devrait exister")
|
||||
}
|
||||
if stream.Priority == nil {
|
||||
t.Fatal("stream.Priority ne devrait pas être nil après PRIORITY frame")
|
||||
}
|
||||
if stream.Priority.Weight != 15 {
|
||||
t.Errorf("Weight: attendu 15, obtenu %d", stream.Priority.Weight)
|
||||
}
|
||||
if stream.Priority.StreamDep != 0 {
|
||||
t.Errorf("StreamDep: attendu 0, obtenu %d", stream.Priority.StreamDep)
|
||||
}
|
||||
if stream.Priority.Exclusive {
|
||||
t.Error("Exclusive devrait être false")
|
||||
}
|
||||
// Verify frame type history
|
||||
found := false
|
||||
for _, ft := range stream.FrameTypes {
|
||||
if ft == http2.FramePriority {
|
||||
found = true
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
t.Error("PRIORITY devrait être dans FrameTypes du stream")
|
||||
}
|
||||
}
|
||||
|
||||
// TestH2ConnStatePerStreamWindowUpdate verifies per-stream WINDOW_UPDATE.
|
||||
func TestH2ConnStatePerStreamWindowUpdate(t *testing.T) {
|
||||
conn := parser.NewH2ConnState()
|
||||
|
||||
// Create stream 3 (client-initiated, odd)
|
||||
headersPayload := []byte{0x82, 0x84} // :method GET, :path /
|
||||
headersFrame := buildH2Frame(0x1, 0x04, 3, headersPayload)
|
||||
conn.ProcessFrames(headersFrame, 0)
|
||||
|
||||
// WINDOW_UPDATE on stream 3 with increment = 32768
|
||||
wuPayload := []byte{0x00, 0x00, 0x80, 0x00} // 32768
|
||||
wuFrame := buildH2Frame(0x8, 0x0, 3, wuPayload) // WINDOW_UPDATE, stream 3
|
||||
|
||||
result, err := conn.ProcessFrames(wuFrame, 0)
|
||||
if err != nil {
|
||||
t.Fatalf("ProcessFrames: %v", err)
|
||||
}
|
||||
if result == nil {
|
||||
t.Fatal("result ne doit pas être nil")
|
||||
}
|
||||
|
||||
stream, ok := conn.Streams[3]
|
||||
if !ok {
|
||||
t.Fatal("stream 3 devrait exister")
|
||||
}
|
||||
if stream.WindowIncr != 32768 {
|
||||
t.Errorf("WindowIncr: attendu 32768, obtenu %d", stream.WindowIncr)
|
||||
}
|
||||
}
|
||||
|
||||
// TestH2ConnStateFrameChronology verifies H2FrameRecord in results.
|
||||
func TestH2ConnStateFrameChronology(t *testing.T) {
|
||||
conn := parser.NewH2ConnState()
|
||||
|
||||
// SETTINGS frame
|
||||
settingsPayload := []byte{
|
||||
0x00, 0x01, 0x00, 0x00, 0x10, 0x00, // HEADER_TABLE_SIZE = 4096
|
||||
}
|
||||
settingsFrame := buildH2Frame(0x4, 0x0, 0, settingsPayload)
|
||||
|
||||
// HEADERS frame
|
||||
headersPayload := []byte{0x82, 0x84} // :method GET, :path /
|
||||
headersFrame := buildH2Frame(0x1, 0x04, 1, headersPayload)
|
||||
|
||||
// Process both frames in one call
|
||||
data := append(settingsFrame, headersFrame...)
|
||||
result, err := conn.ProcessFrames(data, 0)
|
||||
if err != nil {
|
||||
t.Fatalf("ProcessFrames: %v", err)
|
||||
}
|
||||
|
||||
if len(result.Frames) != 2 {
|
||||
t.Fatalf("Frames: attendu 2, obtenu %d", len(result.Frames))
|
||||
}
|
||||
|
||||
// First frame: SETTINGS
|
||||
f0 := result.Frames[0]
|
||||
if f0.Index != 1 {
|
||||
t.Errorf("Frame[0].Index: attendu 1, obtenu %d", f0.Index)
|
||||
}
|
||||
if f0.Direction != 0 {
|
||||
t.Errorf("Frame[0].Direction: attendu 0, obtenu %d", f0.Direction)
|
||||
}
|
||||
if f0.Type != http2.FrameSettings {
|
||||
t.Errorf("Frame[0].Type: attendu SETTINGS, obtenu %v", f0.Type)
|
||||
}
|
||||
if f0.StreamID != 0 {
|
||||
t.Errorf("Frame[0].StreamID: attendu 0, obtenu %d", f0.StreamID)
|
||||
}
|
||||
|
||||
// Second frame: HEADERS
|
||||
f1 := result.Frames[1]
|
||||
if f1.Index != 2 {
|
||||
t.Errorf("Frame[1].Index: attendu 2, obtenu %d", f1.Index)
|
||||
}
|
||||
if f1.Type != http2.FrameHeaders {
|
||||
t.Errorf("Frame[1].Type: attendu HEADERS, obtenu %v", f1.Type)
|
||||
}
|
||||
if f1.StreamID != 1 {
|
||||
t.Errorf("Frame[1].StreamID: attendu 1, obtenu %d", f1.StreamID)
|
||||
}
|
||||
}
|
||||
|
||||
// TestH2ConnStateStreamInitiator verifies stream initiator tracking.
|
||||
func TestH2ConnStateStreamInitiator(t *testing.T) {
|
||||
conn := parser.NewH2ConnState()
|
||||
|
||||
// Stream 1 (client, odd)
|
||||
h1 := []byte{0x82, 0x84} // :method GET, :path /
|
||||
frame1 := buildH2Frame(0x1, 0x04, 1, h1)
|
||||
conn.ProcessFrames(frame1, 0)
|
||||
|
||||
// Stream 2 (server, even) — server-initiated push promise
|
||||
h2 := []byte{0x88} // :status 200
|
||||
frame2 := buildH2Frame(0x1, 0x04, 2, h2)
|
||||
conn.ProcessFrames(frame2, 1)
|
||||
|
||||
stream1, ok1 := conn.Streams[1]
|
||||
if !ok1 {
|
||||
t.Fatal("stream 1 devrait exister")
|
||||
}
|
||||
if stream1.Initiator != 0 {
|
||||
t.Errorf("stream 1 Initiator: attendu 0 (client), obtenu %d", stream1.Initiator)
|
||||
}
|
||||
|
||||
stream2, ok2 := conn.Streams[2]
|
||||
if !ok2 {
|
||||
t.Fatal("stream 2 devrait exister")
|
||||
}
|
||||
if stream2.Initiator != 1 {
|
||||
t.Errorf("stream 2 Initiator: attendu 1 (serveur), obtenu %d", stream2.Initiator)
|
||||
}
|
||||
}
|
||||
|
||||
// TestH2ConnStateStreamStateMachine verifies open → half-closed → closed transitions.
|
||||
func TestH2ConnStateStreamStateMachine(t *testing.T) {
|
||||
conn := parser.NewH2ConnState()
|
||||
|
||||
// Stream 1: HEADERS with END_STREAM (client sends request + END_STREAM)
|
||||
h1 := []byte{0x82, 0x84} // :method GET, :path /
|
||||
frame1 := buildH2Frame(0x1, 0x05, 1, h1) // HEADERS, END_STREAM + END_HEADERS
|
||||
conn.ProcessFrames(frame1, 0)
|
||||
|
||||
stream1, ok := conn.Streams[1]
|
||||
if !ok {
|
||||
t.Fatal("stream 1 devrait exister")
|
||||
}
|
||||
if stream1.State != "half-closed-remote" {
|
||||
t.Errorf("après END_STREAM client: état attendu 'half-closed-remote', obtenu %q", stream1.State)
|
||||
}
|
||||
|
||||
// Server responds with END_STREAM → closed
|
||||
h2 := []byte{0x88} // :status 200
|
||||
frame2 := buildH2Frame(0x1, 0x05, 1, h2) // HEADERS, END_STREAM + END_HEADERS
|
||||
conn.ProcessFrames(frame2, 1)
|
||||
|
||||
if stream1.State != "closed" {
|
||||
t.Errorf("après END_STREAM serveur: état attendu 'closed', obtenu %q", stream1.State)
|
||||
}
|
||||
}
|
||||
|
||||
// TestH2ConnStateStreamFrameHistory verifies FrameTypes accumulation per stream.
|
||||
func TestH2ConnStateStreamFrameHistory(t *testing.T) {
|
||||
conn := parser.NewH2ConnState()
|
||||
|
||||
// HEADERS on stream 1
|
||||
h1 := []byte{0x82, 0x84}
|
||||
frame1 := buildH2Frame(0x1, 0x04, 1, h1) // HEADERS, END_HEADERS
|
||||
conn.ProcessFrames(frame1, 0)
|
||||
|
||||
// DATA on stream 1
|
||||
dataPayload := []byte("hello")
|
||||
dataFrame := buildH2Frame(0x0, 0x01, 1, dataPayload) // DATA, END_STREAM
|
||||
conn.ProcessFrames(dataFrame, 0)
|
||||
|
||||
stream, ok := conn.Streams[1]
|
||||
if !ok {
|
||||
t.Fatal("stream 1 devrait exister")
|
||||
}
|
||||
if len(stream.FrameTypes) != 2 {
|
||||
t.Fatalf("FrameTypes: attendu 2, obtenu %d", len(stream.FrameTypes))
|
||||
}
|
||||
if stream.FrameTypes[0] != http2.FrameHeaders {
|
||||
t.Errorf("FrameTypes[0]: attendu HEADERS, obtenu %v", stream.FrameTypes[0])
|
||||
}
|
||||
if stream.FrameTypes[1] != http2.FrameData {
|
||||
t.Errorf("FrameTypes[1]: attendu DATA, obtenu %v", stream.FrameTypes[1])
|
||||
}
|
||||
}
|
||||
|
||||
// TestH2ConnStateMultipleFramesInBatch verifies frame index persistence across calls.
|
||||
func TestH2ConnStateMultipleFramesInBatch(t *testing.T) {
|
||||
conn := parser.NewH2ConnState()
|
||||
|
||||
// First call: SETTINGS + HEADERS
|
||||
settingsPayload := []byte{
|
||||
0x00, 0x01, 0x00, 0x00, 0x10, 0x00, // HEADER_TABLE_SIZE = 4096
|
||||
}
|
||||
settingsFrame := buildH2Frame(0x4, 0x0, 0, settingsPayload)
|
||||
|
||||
h1 := []byte{0x82, 0x84}
|
||||
headersFrame := buildH2Frame(0x1, 0x04, 1, h1)
|
||||
|
||||
data1 := append(settingsFrame, headersFrame...)
|
||||
result1, _ := conn.ProcessFrames(data1, 0)
|
||||
if len(result1.Frames) != 2 {
|
||||
t.Fatalf("Batch 1: attendu 2 frames, obtenu %d", len(result1.Frames))
|
||||
}
|
||||
if result1.Frames[0].Index != 1 || result1.Frames[1].Index != 2 {
|
||||
t.Errorf("Batch 1 indices: attendu [1,2], obtenu [%d,%d]", result1.Frames[0].Index, result1.Frames[1].Index)
|
||||
}
|
||||
|
||||
// Second call: PING → index should continue at 3
|
||||
pingPayload := make([]byte, 8)
|
||||
pingFrame := buildH2Frame(0x6, 0x0, 0, pingPayload)
|
||||
result2, _ := conn.ProcessFrames(pingFrame, 0)
|
||||
|
||||
if len(result2.Frames) != 1 {
|
||||
t.Fatalf("Batch 2: attendu 1 frame, obtenu %d", len(result2.Frames))
|
||||
}
|
||||
if result2.Frames[0].Index != 3 {
|
||||
t.Errorf("Batch 2 index: attendu 3, obtenu %d", result2.Frames[0].Index)
|
||||
}
|
||||
// This function is in the writer package, not parser - skip direct test here
|
||||
_ = opts
|
||||
}
|
||||
@ -48,7 +48,7 @@ func ParseClientHello(payload []byte) (*ClientHello, error) {
|
||||
recordVersion := binary.BigEndian.Uint16(payload[1:3])
|
||||
recordLength := int(binary.BigEndian.Uint16(payload[3:5]))
|
||||
|
||||
// Le XDP capture au maximum MAX_TLS_PAYLOAD (512) octets.
|
||||
// Le programme TC capture au maximum MAX_TLS_PAYLOAD (2048) octets.
|
||||
// Si la taille du record TLS dépasse les données disponibles, on travaille
|
||||
// avec ce qu'on a (le ClientHello est toujours en début de record).
|
||||
available := len(payload) - 5
|
||||
@ -69,7 +69,7 @@ func ParseClientHello(payload []byte) (*ClientHello, error) {
|
||||
|
||||
// Longueur du ClientHello (3 octets big-endian)
|
||||
chLen := int(uint32(hs[1])<<16 | uint32(hs[2])<<8 | uint32(hs[3]))
|
||||
// Tolérance à la troncature XDP : on travaille avec ce qu'on a
|
||||
// Tolérance à la troncature TC : on travaille avec ce qu'on a
|
||||
if chLen > len(hs)-4 {
|
||||
chLen = len(hs) - 4
|
||||
}
|
||||
|
||||
122
services/ja4ebpf/internal/procutil/proc_lookup_test.go
Normal file
122
services/ja4ebpf/internal/procutil/proc_lookup_test.go
Normal file
@ -0,0 +1,122 @@
|
||||
package procutil
|
||||
|
||||
import (
|
||||
"net"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestParseHexIPv4(t *testing.T) {
|
||||
tests := []struct {
|
||||
hex string
|
||||
want string
|
||||
isErr bool
|
||||
}{
|
||||
// "0201010A" → 10.1.1.2 (little-endian kernel encoding)
|
||||
{"0201010A", "10.1.1.2", false},
|
||||
// "0100007F" → 127.0.0.1
|
||||
{"0100007F", "127.0.0.1", false},
|
||||
// "00000000" → 0.0.0.0
|
||||
{"00000000", "0.0.0.0", false},
|
||||
// Invalid: wrong length
|
||||
{"0101", "", true},
|
||||
{"", "", true},
|
||||
// Invalid: non-hex
|
||||
{"ZZZZZZZZ", "", true},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
ip, err := parseHexIPv4(tt.hex)
|
||||
if tt.isErr {
|
||||
if err == nil {
|
||||
t.Errorf("parseHexIPv4(%q): expected error, got ip=%v", tt.hex, ip)
|
||||
}
|
||||
continue
|
||||
}
|
||||
if err != nil {
|
||||
t.Errorf("parseHexIPv4(%q): unexpected error: %v", tt.hex, err)
|
||||
continue
|
||||
}
|
||||
got := ip.String()
|
||||
if got != tt.want {
|
||||
t.Errorf("parseHexIPv4(%q) = %v, want %v", tt.hex, got, tt.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseHexIPv6(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
hex string
|
||||
want string
|
||||
isErr bool
|
||||
}{
|
||||
// IPv4-mapped ::ffff:127.0.0.1
|
||||
// In /proc/net/tcp6: 4 x 32-bit LE words
|
||||
// word0=00000000 word1=00000000 word2=0xffff0000 (LE for 00 00 ff ff) word3=0x0100007f (LE for 7f 00 00 01)
|
||||
{
|
||||
"ipv4-mapped loopback",
|
||||
"0000000000000000FFFF00000100007F",
|
||||
"127.0.0.1",
|
||||
false,
|
||||
},
|
||||
// Invalid: wrong length
|
||||
{"too short", "0000", "", true},
|
||||
// Invalid: non-hex
|
||||
{"bad hex", "ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ", "", true},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
ip, err := parseHexIPv6(tt.hex)
|
||||
if tt.isErr {
|
||||
if err == nil {
|
||||
t.Errorf("parseHexIPv6(%q): expected error, got ip=%v", tt.hex, ip)
|
||||
}
|
||||
return
|
||||
}
|
||||
if err != nil {
|
||||
t.Errorf("parseHexIPv6(%q): unexpected error: %v", tt.hex, err)
|
||||
return
|
||||
}
|
||||
got := ip.String()
|
||||
if got != tt.want {
|
||||
t.Errorf("parseHexIPv6(%q) = %v, want %v", tt.hex, got, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseHexIPv6_Native(t *testing.T) {
|
||||
// Full IPv6: 2001:db8::1
|
||||
// Little-endian chunks: 0000:0000:0000:0000:0000:0000:0000:0001 → but in /proc format
|
||||
// "00000000000000000000000001000000" → maps to a real IPv6
|
||||
ip, err := parseHexIPv6("01000000000000000000000000000000")
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
// Should be native IPv6, not IPv4-mapped
|
||||
if ip.To4() != nil && !isIPv4MappedIPv6(ip.To16()) {
|
||||
t.Errorf("expected native IPv6, got IPv4: %v", ip)
|
||||
}
|
||||
}
|
||||
|
||||
func TestIsIPv4MappedIPv6(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
ip net.IP
|
||||
want bool
|
||||
}{
|
||||
{"nil", nil, false},
|
||||
{"4-byte", net.IP{10, 0, 0, 1}, false},
|
||||
{"loopback mapped", net.ParseIP("::ffff:127.0.0.1"), true},
|
||||
{"mapped 10.0.0.1", net.ParseIP("::ffff:10.0.0.1"), true},
|
||||
{"not mapped", net.ParseIP("2001:db8::1"), false},
|
||||
{"all zeros", net.IPv6zero, false},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
got := isIPv4MappedIPv6(tt.ip)
|
||||
if got != tt.want {
|
||||
t.Errorf("isIPv4MappedIPv6(%v) = %v, want %v", tt.ip, got, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@ -90,6 +90,7 @@ type sessionRecord struct {
|
||||
H2WindowUpdate uint32 `json:"h2_window_update,omitempty"`
|
||||
H2PseudoOrder string `json:"h2_pseudo_order,omitempty"`
|
||||
H2HasPriority uint8 `json:"h2_has_priority,omitempty"`
|
||||
H2SettingsAck uint8 `json:"h2_settings_ack,omitempty"`
|
||||
H2HeaderTableSize *int32 `json:"h2_header_table_size,omitempty"`
|
||||
H2EnablePush *int32 `json:"h2_enable_push,omitempty"`
|
||||
H2MaxConcurrentStreams *int32 `json:"h2_max_concurrent_streams,omitempty"`
|
||||
@ -110,6 +111,11 @@ func NewClickHouseWriter(dsn string, batchSize int, flushInterval time.Duration)
|
||||
return nil, fmt.Errorf("analyse DSN ClickHouse: %w", err)
|
||||
}
|
||||
|
||||
// Désactiver l'insertion asynchrone pour les writes transactionnels
|
||||
opts.Settings = map[string]interface{}{
|
||||
"async_insert": 0,
|
||||
}
|
||||
|
||||
conn, err := clickhouse.Open(opts)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("connexion ClickHouse: %w", err)
|
||||
@ -201,6 +207,10 @@ func (w *ClickHouseWriter) flushBatch(ctx context.Context, batch []*correlation.
|
||||
}
|
||||
|
||||
for _, s := range batch {
|
||||
// Ignorer les sessions sans aucune donnée applicative (SYN-only)
|
||||
if len(s.Requests) == 0 && s.TLS == nil {
|
||||
continue
|
||||
}
|
||||
record := sessionToRecord(s)
|
||||
jsonBytes, err := json.Marshal(record)
|
||||
if err != nil {
|
||||
@ -371,12 +381,29 @@ func sessionToRecord(s *correlation.SessionState) sessionRecord {
|
||||
}
|
||||
}
|
||||
|
||||
// Champs HTTP/2 au niveau connexion (H2ConnState)
|
||||
if s.H2Conn != nil {
|
||||
if s.H2Conn.SettingsAck {
|
||||
rec.H2SettingsAck = 1
|
||||
}
|
||||
// Vérifier si un stream a reçu une frame PRIORITY
|
||||
for _, stream := range s.H2Conn.Streams {
|
||||
if stream.Priority != nil {
|
||||
rec.H2HasPriority = 1
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return rec
|
||||
}
|
||||
|
||||
// pseudoOrderToShort convertit la liste de pseudo-headers en notation abrégée.
|
||||
// Ex: [":method", ":authority", ":scheme", ":path"] → "m,a,s,p"
|
||||
func pseudoOrderToShort(headers []string) string {
|
||||
if len(headers) == 0 {
|
||||
return ""
|
||||
}
|
||||
short := make([]byte, 0, len(headers)*2-1)
|
||||
for i, h := range headers {
|
||||
if i > 0 {
|
||||
@ -391,6 +418,8 @@ func pseudoOrderToShort(headers []string) string {
|
||||
short = append(short, 's')
|
||||
case h == ":path":
|
||||
short = append(short, 'p')
|
||||
case h == ":status":
|
||||
short = append(short, 't')
|
||||
default:
|
||||
short = append(short, '?')
|
||||
}
|
||||
@ -506,7 +535,7 @@ func formatTCPOptions(opts []byte) string {
|
||||
kind := opts[i]
|
||||
switch kind {
|
||||
case 0: // End of Options List
|
||||
break
|
||||
return strings.Join(names, ",")
|
||||
case 1: // NOP
|
||||
names = append(names, "NOP")
|
||||
i++
|
||||
|
||||
179
services/ja4ebpf/internal/writer/clickhouse_test.go
Normal file
179
services/ja4ebpf/internal/writer/clickhouse_test.go
Normal file
@ -0,0 +1,179 @@
|
||||
package writer
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/antitbone/ja4/ja4ebpf/internal/correlation"
|
||||
)
|
||||
|
||||
func TestFormatTLSVersion(t *testing.T) {
|
||||
tests := []struct {
|
||||
input uint16
|
||||
want string
|
||||
}{
|
||||
{0x0301, "TLSv1.0"},
|
||||
{0x0302, "TLSv1.1"},
|
||||
{0x0303, "TLSv1.2"},
|
||||
{0x0304, "TLSv1.3"},
|
||||
{0x0000, ""},
|
||||
{0x0300, ""},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
got := formatTLSVersion(tt.input)
|
||||
if got != tt.want {
|
||||
t.Errorf("formatTLSVersion(0x%04x) = %q, want %q", tt.input, got, tt.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestHeaderVal(t *testing.T) {
|
||||
kv := map[string]string{
|
||||
"User-Agent": "Mozilla/5.0",
|
||||
"accept": "text/html",
|
||||
"Accept-Encoding": "gzip",
|
||||
"accept-encoding": "br",
|
||||
}
|
||||
tests := []struct {
|
||||
titleKey string
|
||||
lowerKey string
|
||||
want string
|
||||
}{
|
||||
{"User-Agent", "user-agent", "Mozilla/5.0"},
|
||||
{"Accept", "accept", "text/html"}, // lowercase key in map
|
||||
{"Accept-Encoding", "accept-encoding", "gzip"}, // title-case wins
|
||||
{"X-Missing", "x-missing", ""}, // not present
|
||||
}
|
||||
for _, tt := range tests {
|
||||
got := headerVal(kv, tt.titleKey, tt.lowerKey)
|
||||
if got != tt.want {
|
||||
t.Errorf("headerVal(kv, %q, %q) = %q, want %q", tt.titleKey, tt.lowerKey, got, tt.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildClientHeaders(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
kv map[string]string
|
||||
want string
|
||||
}{
|
||||
{"empty", nil, ""},
|
||||
{"empty map", map[string]string{}, ""},
|
||||
{"single header", map[string]string{"user-agent": "curl/8.0"}, `{"user-agent":"curl/8.0"}`},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
got := buildClientHeaders(tt.kv)
|
||||
if got != tt.want {
|
||||
t.Errorf("buildClientHeaders() = %q, want %q", got, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestPseudoOrderToShort(t *testing.T) {
|
||||
tests := []struct {
|
||||
input []string
|
||||
want string
|
||||
}{
|
||||
{[]string{":method", ":path", ":scheme", ":authority"}, "m,p,s,a"},
|
||||
{[]string{":method", ":authority", ":scheme", ":path"}, "m,a,s,p"},
|
||||
{[]string{":method"}, "m"},
|
||||
{[]string{":status"}, "t"},
|
||||
{[]string{":method", ":path", ":unknown"}, "m,p,?"},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
got := pseudoOrderToShort(tt.input)
|
||||
if got != tt.want {
|
||||
t.Errorf("pseudoOrderToShort(%v) = %q, want %q", tt.input, got, tt.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestFormatTCPOptions(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
opts []byte
|
||||
want string
|
||||
}{
|
||||
{"nil", nil, ""},
|
||||
{"empty", []byte{}, ""},
|
||||
{"MSS only", []byte{2, 4, 0x05, 0xB4}, "MSS"}, // MSS=1460
|
||||
{"WS only", []byte{3, 3, 6}, "WS"}, // WS=6
|
||||
{"SACK", []byte{4, 2}, "SACK"}, // SACK Permitted
|
||||
{"TS", []byte{8, 10, 0, 0, 0, 0, 0, 0, 0, 0}, "TS"}, // Timestamp
|
||||
{"NOP+MSS+WS+SACK+TS", []byte{1, 2, 4, 0x05, 0xB4, 3, 3, 6, 4, 2, 8, 10, 0, 0, 0, 0, 0, 0, 0, 0}, "NOP,MSS,WS,SACK,TS"},
|
||||
{"EOL", []byte{0}, ""},
|
||||
{"NOP", []byte{1, 1}, "NOP,NOP"},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
got := formatTCPOptions(tt.opts)
|
||||
if got != tt.want {
|
||||
t.Errorf("formatTCPOptions(%v) = %q, want %q", tt.opts, got, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildH2Fingerprint(t *testing.T) {
|
||||
h2 := &correlation.HTTP2Settings{
|
||||
HeaderTableSize: 4096,
|
||||
EnablePush: 0,
|
||||
MaxConcurrentStreams: 100,
|
||||
InitialWindowSize: 65535,
|
||||
MaxFrameSize: 16384,
|
||||
MaxHeaderListSize: 262144,
|
||||
WindowUpdateIncrement: 15663105,
|
||||
PseudoHeaderOrder: []string{":method", ":authority", ":scheme", ":path"},
|
||||
}
|
||||
got := buildH2Fingerprint(h2)
|
||||
// Expected: "1:4096,2:0,3:100,4:65535,5:16384,6:262144|15663105|0|m,a,s,p"
|
||||
want := "1:4096,2:0,3:100,4:65535,5:16384,6:262144|15663105|0|m,a,s,p"
|
||||
if got != want {
|
||||
t.Errorf("buildH2Fingerprint() = %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildH2Fingerprint_Minimal(t *testing.T) {
|
||||
h2 := &correlation.HTTP2Settings{
|
||||
HeaderTableSize: 4096,
|
||||
EnablePush: 0,
|
||||
InitialWindowSize: 65535,
|
||||
MaxConcurrentStreams: -1,
|
||||
MaxFrameSize: -1,
|
||||
MaxHeaderListSize: -1,
|
||||
WindowUpdateIncrement: 0,
|
||||
PseudoHeaderOrder: nil,
|
||||
}
|
||||
got := buildH2Fingerprint(h2)
|
||||
want := "1:4096,2:0,4:65535||0|"
|
||||
if got != want {
|
||||
t.Errorf("buildH2Fingerprint() = %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildH2SettingsFP(t *testing.T) {
|
||||
h2 := &correlation.HTTP2Settings{
|
||||
MaxConcurrentStreams: 100,
|
||||
InitialWindowSize: 65535,
|
||||
EnablePush: 0,
|
||||
}
|
||||
got := buildH2SettingsFP(h2)
|
||||
want := "3:100,4:65535,2:0"
|
||||
if got != want {
|
||||
t.Errorf("buildH2SettingsFP() = %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildH2SettingsFP_Empty(t *testing.T) {
|
||||
h2 := &correlation.HTTP2Settings{
|
||||
MaxConcurrentStreams: -1,
|
||||
InitialWindowSize: -1,
|
||||
EnablePush: -1,
|
||||
}
|
||||
got := buildH2SettingsFP(h2)
|
||||
if got != "" {
|
||||
t.Errorf("buildH2SettingsFP() = %q, want empty", got)
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user