Feat: Détection menaces HTTP via vues ClickHouse + simplification shutdown
Some checks failed
Build and Test / test (push) Has been cancelled
Build and Test / build (push) Has been cancelled
Build and Test / docker (push) Has been cancelled

Nouvelles vues de détection (sql/views.sql) :
- Identification hosts par IP/JA4 (view_host_identification, view_host_ja4_anomalies)
- Détection brute force POST et query params variables
- Header fingerprinting (ordre, headers modernes manquants, Sec-CH-UA)
- ALPN mismatch detection (h2 déclaré mais HTTP/1.1 parlé)
- Rate limiting & burst detection (50 req/min, 20 req/10s)
- Path enumeration/scanning (paths sensibles)
- Payload attacks (SQLi, XSS, path traversal)
- JA4 botnet detection (même fingerprint sur 20+ IPs)
- Correlation quality (orphan ratio >80%)

ClickHouse (sql/init.sql) :
- Compression ZSTD(3) sur champs texte (path, query, headers, ja3/ja4)
- TTL automatique : 1 jour (raw) + 7 jours (http_logs)
- Paramètre ttl_only_drop_parts = 1

Shutdown simplifié (internal/app/orchestrator.go) :
- Suppression ShutdownTimeout et logique de flush/attente
- Stop() = cancel() + Close() uniquement
- systemd TimeoutStopSec gère l'arrêt forcé si besoin

File output toggle (internal/config/*.go) :
- Ajout champ Enabled dans FileOutputConfig
- Le sink fichier n'est créé que si enabled && path != ''
- Tests : TestValidate_FileOutputDisabled, TestLoadConfig_FileOutputDisabled

RPM packaging (packaging/rpm/logcorrelator.spec) :
- Changelog 1.1.18 → 1.1.22
- Suppression logcorrelator-tmpfiles.conf (redondant RuntimeDirectory=)

Nettoyage :
- idees.txt → idees/ (dossier)
- Suppression 91.224.92.185.txt (logs exemple)

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
This commit is contained in:
toto
2026-03-11 18:28:07 +01:00
parent 5df2fd965b
commit 20ebe7240e
17 changed files with 1089 additions and 6598 deletions

View File

@ -13,8 +13,6 @@ import (
const (
// DefaultEventChannelBufferSize is the default size for event channels
DefaultEventChannelBufferSize = 1000
// ShutdownTimeout is the maximum time to wait for graceful shutdown
ShutdownTimeout = 30 * time.Second
// OrphanTickInterval is how often the orchestrator drains pending orphans.
// Set to half the default emit delay (500ms/2) so orphans are emitted promptly
// even when no new events arrive.
@ -143,46 +141,17 @@ func (o *Orchestrator) processEvents(eventChan <-chan *domain.NormalizedEvent) {
}
// Stop gracefully stops the orchestrator.
// It stops all sources first, then flushes remaining events, then closes sinks.
// It stops all sources and closes sinks immediately without waiting for queue drainage.
// systemd TimeoutStopSec handles forced termination if needed.
func (o *Orchestrator) Stop() error {
if !o.running.CompareAndSwap(true, false) {
return nil // Not running
}
// Create shutdown context with timeout
shutdownCtx, shutdownCancel := context.WithTimeout(context.Background(), ShutdownTimeout)
defer shutdownCancel()
// First, cancel the main context to stop accepting new events
// Cancel context to stop accepting new events immediately
o.cancel()
// Wait for source goroutines to finish
// Use a separate goroutine with timeout to prevent deadlock
done := make(chan struct{})
go func() {
o.wg.Wait()
close(done)
}()
select {
case <-done:
// Sources stopped cleanly
case <-shutdownCtx.Done():
// Timeout waiting for sources
}
// Flush remaining events from correlation service
flushedLogs := o.correlationSvc.Flush()
for _, log := range flushedLogs {
if err := o.config.Sink.Write(shutdownCtx, log); err != nil {
// Log error but continue
}
}
// Flush and close sink with timeout
if err := o.config.Sink.Flush(shutdownCtx); err != nil {
// Log error
}
// Close sink (flush skipped - in-flight events are dropped)
if err := o.config.Sink.Close(); err != nil {
// Log error
}