feat(correlation): add include_dest_ports filter + README/arch update (v1.1.12)
- feat: new config directive include_dest_ports ([]int) in correlation section - feat: if non-empty, only events with a matching dst_port are correlated - feat: filtered events are silently ignored (not correlated, not emitted as orphan) - feat: new metric failed_dest_port_filtered tracked in ProcessEvent - feat: DEBUG log 'event excluded by dest port filter: source=A dst_port=22' - test: TestCorrelationService_IncludeDestPorts_AllowedPort - test: TestCorrelationService_IncludeDestPorts_FilteredPort - test: TestCorrelationService_IncludeDestPorts_EmptyAllowsAll - docs(readme): full rewrite to match current code (v1.1.12) - docs(readme): add include_dest_ports section, fix version refs, clean outdated sections - docs(arch): add dest_port_filtering section, failed_dest_port_filtered metric, debug log example - fix(config.example): remove obsolete stdout.level field - chore: bump version to 1.1.12 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
@ -412,6 +412,21 @@ correlation:
|
||||
description: >
|
||||
Stratégie 1‑à‑N : un log réseau peut être utilisé pour plusieurs logs HTTP
|
||||
successifs tant qu'il n'a pas expiré ni été évincé.
|
||||
ip_filtering:
|
||||
directive: exclude_source_ips
|
||||
description: >
|
||||
Liste d'IPs source (exactes ou plages CIDR) à ignorer silencieusement.
|
||||
Événements non corrélés, non émis en orphelin. Métrique : failed_ip_excluded.
|
||||
dest_port_filtering:
|
||||
directive: include_dest_ports
|
||||
description: >
|
||||
Liste blanche de ports de destination. Si non vide, seuls les événements
|
||||
dont le dst_port est dans la liste participent à la corrélation. Les autres
|
||||
sont silencieusement ignorés (non corrélés, non émis en orphelin).
|
||||
Liste vide = tous les ports autorisés (comportement par défaut).
|
||||
Métrique : failed_dest_port_filtered.
|
||||
example:
|
||||
include_dest_ports: [80, 443, 8080, 8443]
|
||||
|
||||
schema:
|
||||
description: >
|
||||
@ -708,6 +723,8 @@ architecture:
|
||||
responsibilities:
|
||||
- Modèles NormalizedEvent et CorrelatedLog.
|
||||
- CorrelationService (fenêtre, TTL, buffers bornés, one-to-many/Keep-Alive, orphelins).
|
||||
- Filtrage par IP source (exclude_source_ips, CIDR).
|
||||
- Filtrage par port destination (include_dest_ports, liste blanche).
|
||||
- Custom JSON marshaling pour CorrelatedLog (structure plate).
|
||||
- name: internal/ports
|
||||
type: ports
|
||||
@ -849,6 +866,7 @@ observability:
|
||||
- "A event has no matching B key in buffer: key=..."
|
||||
- "A event has same key as B but outside time window: key=... time_diff=5s window=10s"
|
||||
- "event excluded by IP filter: source=A src_ip=10.0.0.1 src_port=8080"
|
||||
- "event excluded by dest port filter: source=A dst_port=22"
|
||||
- "TTL reset for B event (Keep-Alive): key=... new_ttl=120s"
|
||||
- "[clickhouse] DEBUG batch sent: rows=42 table=correlated_logs_http_network"
|
||||
info_logs:
|
||||
@ -877,6 +895,7 @@ observability:
|
||||
"failed_buffer_eviction": 5,
|
||||
"failed_ttl_expired": 12,
|
||||
"failed_ip_excluded": 7,
|
||||
"failed_dest_port_filtered": 3,
|
||||
"buffer_a_size": 23,
|
||||
"buffer_b_size": 18,
|
||||
"orphans_emitted_a": 92,
|
||||
@ -900,6 +919,7 @@ observability:
|
||||
- failed_buffer_eviction: Buffer plein, événement évincé
|
||||
- failed_ttl_expired: TTL du événement B expiré
|
||||
- failed_ip_excluded: Événement exclu par filtre IP (exclude_source_ips)
|
||||
- failed_dest_port_filtered: Événement exclu par filtre port destination (include_dest_ports)
|
||||
buffers:
|
||||
- buffer_a_size: Taille actuelle du buffer HTTP
|
||||
- buffer_b_size: Taille actuelle du buffer réseau
|
||||
@ -929,6 +949,9 @@ observability:
|
||||
- symptom: failed_ip_excluded élevé
|
||||
cause: Traffic depuis des IPs configurées dans exclude_source_ips
|
||||
solution: Vérifier la configuration, c'est normal si attendu
|
||||
- symptom: failed_dest_port_filtered élevé
|
||||
cause: Traffic sur des ports non listés dans include_dest_ports
|
||||
solution: Vérifier la configuration include_dest_ports, ou vider la liste pour tout accepter
|
||||
- symptom: orphans_emitted_a élevé
|
||||
cause: Beaucoup de logs A sans correspondance B
|
||||
solution: Vérifier que la source B envoie bien les événements attendus
|
||||
|
||||
Reference in New Issue
Block a user