feat(correlation): add include_dest_ports filter + README/arch update (v1.1.12)

- feat: new config directive include_dest_ports ([]int) in correlation section
- feat: if non-empty, only events with a matching dst_port are correlated
- feat: filtered events are silently ignored (not correlated, not emitted as orphan)
- feat: new metric failed_dest_port_filtered tracked in ProcessEvent
- feat: DEBUG log 'event excluded by dest port filter: source=A dst_port=22'
- test: TestCorrelationService_IncludeDestPorts_AllowedPort
- test: TestCorrelationService_IncludeDestPorts_FilteredPort
- test: TestCorrelationService_IncludeDestPorts_EmptyAllowsAll
- docs(readme): full rewrite to match current code (v1.1.12)
- docs(readme): add include_dest_ports section, fix version refs, clean outdated sections
- docs(arch): add dest_port_filtering section, failed_dest_port_filtered metric, debug log example
- fix(config.example): remove obsolete stdout.level field
- chore: bump version to 1.1.12

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
toto
2026-03-05 13:51:20 +01:00
parent ba9e0ab477
commit a8e024105d
9 changed files with 454 additions and 544 deletions

View File

@ -412,6 +412,21 @@ correlation:
description: >
Stratégie 1àN : un log réseau peut être utilisé pour plusieurs logs HTTP
successifs tant qu'il n'a pas expiré ni été évincé.
ip_filtering:
directive: exclude_source_ips
description: >
Liste d'IPs source (exactes ou plages CIDR) à ignorer silencieusement.
Événements non corrélés, non émis en orphelin. Métrique : failed_ip_excluded.
dest_port_filtering:
directive: include_dest_ports
description: >
Liste blanche de ports de destination. Si non vide, seuls les événements
dont le dst_port est dans la liste participent à la corrélation. Les autres
sont silencieusement ignorés (non corrélés, non émis en orphelin).
Liste vide = tous les ports autorisés (comportement par défaut).
Métrique : failed_dest_port_filtered.
example:
include_dest_ports: [80, 443, 8080, 8443]
schema:
description: >
@ -708,6 +723,8 @@ architecture:
responsibilities:
- Modèles NormalizedEvent et CorrelatedLog.
- CorrelationService (fenêtre, TTL, buffers bornés, one-to-many/Keep-Alive, orphelins).
- Filtrage par IP source (exclude_source_ips, CIDR).
- Filtrage par port destination (include_dest_ports, liste blanche).
- Custom JSON marshaling pour CorrelatedLog (structure plate).
- name: internal/ports
type: ports
@ -849,6 +866,7 @@ observability:
- "A event has no matching B key in buffer: key=..."
- "A event has same key as B but outside time window: key=... time_diff=5s window=10s"
- "event excluded by IP filter: source=A src_ip=10.0.0.1 src_port=8080"
- "event excluded by dest port filter: source=A dst_port=22"
- "TTL reset for B event (Keep-Alive): key=... new_ttl=120s"
- "[clickhouse] DEBUG batch sent: rows=42 table=correlated_logs_http_network"
info_logs:
@ -877,6 +895,7 @@ observability:
"failed_buffer_eviction": 5,
"failed_ttl_expired": 12,
"failed_ip_excluded": 7,
"failed_dest_port_filtered": 3,
"buffer_a_size": 23,
"buffer_b_size": 18,
"orphans_emitted_a": 92,
@ -900,6 +919,7 @@ observability:
- failed_buffer_eviction: Buffer plein, événement évincé
- failed_ttl_expired: TTL du événement B expiré
- failed_ip_excluded: Événement exclu par filtre IP (exclude_source_ips)
- failed_dest_port_filtered: Événement exclu par filtre port destination (include_dest_ports)
buffers:
- buffer_a_size: Taille actuelle du buffer HTTP
- buffer_b_size: Taille actuelle du buffer réseau
@ -929,6 +949,9 @@ observability:
- symptom: failed_ip_excluded élevé
cause: Traffic depuis des IPs configurées dans exclude_source_ips
solution: Vérifier la configuration, c'est normal si attendu
- symptom: failed_dest_port_filtered élevé
cause: Traffic sur des ports non listés dans include_dest_ports
solution: Vérifier la configuration include_dest_ports, ou vider la liste pour tout accepter
- symptom: orphans_emitted_a élevé
cause: Beaucoup de logs A sans correspondance B
solution: Vérifier que la source B envoie bien les événements attendus