code_public/ja4-platform
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(ja4ebpf): fix TLS capture, SYN offsets, TCP option parsing

Browse Source 
- Increase MAX_TLS_PAYLOAD from 512 to 2048 bytes to capture full
  TLS ClientHellos (modern browsers/curl send 1000-1543 byte ClientHellos)
- Fix ParseClientHello to tolerate XDP-truncated payloads: clamp
  recordLength and chLen to available data instead of returning error
- Fix cipher suites, compression, extensions truncation to use clamping
- Fix consumeSynEvents struct field offsets: dst_ip (4 bytes at offset 4)
  was not accounted for, causing all L3/L4 metadata to be read from
  wrong positions (TTL was actually dst_ip[0], windowSize was dst_port, etc.)
- Add parseTCPOptions() to extract MSS and Window Scale from raw TCP options
  (C code sets defaults of mss=0, window_scale=0xFF, expects Go to parse)
- Fix consumeAcceptEvents: skip zero-IP events to avoid phantom sessions
- Fix consumeSSLEvents: filter zero-IP/port events when proc fallback fails
- Add missing consumeHTTPPlainEvents goroutine (was defined but never called)
- Fix race condition: SYN consumer sets Correlated=true if TLS already present
- Update tls_hello_event struct offsets in Go consumer (payload_len now at
  offset 2054, was 518, due to payload array growing from 512 to 2048 bytes)
- Remove debug logging from consumers and GC

E2E verified: HTTP plain (port 80) and HTTPS (port 443) both produce
fully correlated sessions in ClickHouse with correct:
  - ip_meta_ttl=64, ip_meta_df=true, ip_meta_id
  - tcp_meta_window_size=64240, tcp_meta_window_scale=10, tcp_meta_mss=1460
  - ja4=t13i3010_1d37bd780c83_95d2a80e6515
  - tls_alpn=http/1.1
  - method=GET, path=/, header_order_signature=Host;User-Agent;Accept
  - correlated=1

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
toto
2026-04-12 04:16:44 +02:00
parent f85a10b012
commit b1218a2367
12 changed files with 715 additions and 248 deletions
Download Patch File Download Diff File Expand all files Collapse all files

25
services/ja4ebpf/internal/loader/loader.go
View File

@ -10,7 +10,6 @@ import (
"net"
"os"
"github.com/cilium/ebpf"
"github.com/cilium/ebpf/link"
"github.com/cilium/ebpf/ringbuf"
"github.com/cilium/ebpf/rlimit"
@ -122,23 +121,31 @@ func New() (*Loader, error) {
}, nil
}
// AttachTC attache le programme TC ingress sur l'interface réseau spécifiée.
// Utilise TCX (TC eXpress) disponible depuis le noyau 6.6+.
// AttachTC attache le programme XDP sur l'interface réseau spécifiée.
// Essaie le mode natif XDP (driver support) puis se replie sur le mode générique
// (SKB_MODE, compatible kernel ≥ 4.8, fonctionne dans les VMs).
func (l *Loader) AttachTC(iface string) error {
// Résoudre l'interface réseau par son nom
netIface, err := net.InterfaceByName(iface)
if err != nil {
return fmt.Errorf("interface réseau %q introuvable: %w", iface, err)
}
// Attacher le programme TC en ingress via TCX
lnk, err := link.AttachTCX(link.TCXOptions{
// Mode natif (meilleure performance sur serveurs avec NIC compatible XDP)
lnk, err := link.AttachXDP(link.XDPOptions{
Interface: netIface.Index,
Program: l.tcObjs.CaptureTcIngress,
Attach: ebpf.AttachTCXIngress,
Program: l.tcObjs.CaptureXdp,
Flags: link.XDPDriverMode,
})
if err != nil {
return fmt.Errorf("attachement TC ingress sur %q: %w", iface, err)
// Repli sur le mode générique (VMs, NICs sans driver XDP natif)
lnk, err = link.AttachXDP(link.XDPOptions{
Interface: netIface.Index,
Program: l.tcObjs.CaptureXdp,
Flags: link.XDPGenericMode,
})
if err != nil {
return fmt.Errorf("attachement XDP sur %q (natif et générique): %w", iface, err)
}
}
l.tcLink = lnk