Ajoute un filtre d'entrée de connexion (AP_FTYPE_CONNECTION, APR_HOOK_LAST)
qui s'insère entre mod_ssl et mod_http2 pour lire de manière non-destructive
le preface HTTP/2 (RFC 9113 §3.4) et en extraire :
- h2_fingerprint : fingerprint Akamai complet
ex. '1:65536,2:0,4:6291456,6:262144|15663105|0|m,a,s,p'
- h2_settings_fp : entrées SETTINGS brutes (ex. '1:65536,4:6291456')
- h2_window_update : incrément WINDOW_UPDATE (ex. '15663105')
- h2_pseudo_order : ordre des pseudo-headers (ex. 'm,a,s,p' Chrome,
'm,p,s,a' Firefox)
Technique : lecture spéculative AP_MODE_SPECULATIVE (non-destructive)
de 512 octets — la donnée reste disponible pour mod_http2. Le filtre
se retire de la chaîne après la première invocation.
Stockage dans c->notes (H2_NOTE_*) puis émission JSON dans log_request().
ClickHouse : 4 nouvelles colonnes dans http_logs + JSONExtract dans mv_http_logs.
Migration pour déploiements existants : 04_http2_fields.sql.
14 tests unitaires (cmocka) couvrent Chrome/Firefox/HTTP1/troncature/HPACK.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add BuildRequires: systemd-rpm-macros to sentinel and correlator specs
- Replace manual systemctl calls with %systemd_post, %systemd_preun,
%systemd_postun_with_restart macros (handles daemon-reload, stop/disable,
try-restart on upgrade correctly and is a no-op in containers)
- ja4sentinel.spec: use %{_unitdir} macro instead of hardcoded path
(/usr/lib/systemd/system); remove cross-service /var/run/logcorrelator
from %files and %post (owned by logcorrelator package, not sentinel)
- logcorrelator.spec: move unit from /etc/systemd/system (admin namespace)
to %{_unitdir} (/usr/lib/systemd/system) — correct packaging location;
move user/group creation from %post to %pre so file ownership is valid
during RPM install phase; add Requires(pre): shadow-utils; fix bare
directory entries in %files with %dir macro; add version fallback macro
so spec is buildable without --define version
- test-rpm.sh: auto-build RPM via Dockerfile.package if dist/rpm/ is
empty; update service file path check to /usr/lib/systemd/system/
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
deploy_views.sql (v13 → v14):
- CRITICAL: ml_detected_anomalies ORDER BY (src_ip) → (src_ip, ja4, host, model_name)
ReplacingMergeTree was collapsing all detections to 1 row per IP on merge
- Add PARTITION BY toDate + ttl_only_drop_parts on all 4 data tables
- ml_all_scores TTL 3d → 7d; ml_detected_anomalies TTL 30d → 7d
- agg_host_ip_ja4_1h + agg_header_fingerprint_1h: add partition + TTL 7d
- view_ip_recurrence: add WHERE detected_at >= now() - 7 DAY (was full scan)
- Remove dead views: summary/timeseries/threat_dist/variability
- Add view_dashboard_entities (fixes HTTP 500 in clustering/incidents/fingerprints)
- Add view_dashboard_user_agents (fixes HTTP 500 in fingerprints/metrics)
- Add view_ai_features_24h (enables ENABLE_MULTIWINDOW in bot_detector)
- Mark max_requests_per_sec as DEPRECATED (always 0)
New files:
- correlator/sql/migrations/01_ttl_adjustments.sql: ALTER TABLE migration
- tests/integration/verify_mvs.py: MV pipeline verification assertions
- docs/THESIS_HTTP_Traffic_Detection.md: detection techniques thesis
All DB references use ja4_processing/ja4_logs (no mabase_prod).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace golang:1.24 (Debian) builder with rockylinux:9 + dnf golang.
All three RPM packages (sentinel, correlator, mod-reqin-log) now build
entirely on Rocky Linux Docker images, ensuring native ABI compatibility.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
