Commit Graph

15 Commits

Author SHA1 Message Date
e9dcd8ea51 feat: observability, IP filtering, stdout/clickhouse fixes (v1.1.11)
- feat(observability): metrics server with /metrics and /health endpoints
- feat(observability): correlation metrics (events, success/failed, reasons, buffers)
- feat(correlation): IP exclusion filter (exact IPs and CIDR ranges)
- feat(correlation): pending orphan delay for late-arriving B events
- fix(stdout): sink is now a no-op for data; JSON must never appear on stdout
- fix(clickhouse): all flush errors were silently discarded, now properly logged
- fix(clickhouse): buffer overflow with DropOnOverflow now logged at WARN
- fix(clickhouse): retry attempts logged at WARN with attempt/delay/error context
- feat(clickhouse): connection success logged at INFO, batch sends at DEBUG
- feat(clickhouse): SetLogger() for external logger injection
- test(stdout): assert stdout remains empty for correlated and orphan logs
- chore(rpm): bump version to 1.1.11, update changelog
- docs: README and architecture.yml updated

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-05 11:40:54 +01:00
e0c622f635 feat(correlation): add configurable delay before emitting orphan A events
New feature: Apache events can now wait for B events before being emitted as orphans.

Changes:
- Add ApacheEmitDelayMs config (default: 500ms)
- Add pendingOrphans structure for delayed emission
- processSourceA(): add A to pending orphans instead of immediate emission
- processSourceB(): check pending orphans before buffer A
- emitPendingOrphans(): emit orphans after delay expires
- Flush(): emit all pending orphans immediately (shutdown)

Configuration:
correlation:
  orphan_policy:
    apache_always_emit: true
    apache_emit_delay_ms: 500  # Wait 500ms before emitting as orphan

Backward compatibility:
- apache_emit_delay_ms: 0 → immediate emission (legacy mode)
- apache_emit_delay_ms < 0 → default 500ms

Tests added (5 new tests):
- TestCorrelationService_ApacheEmitDelay_BArrivesDuringDelay
- TestCorrelationService_ApacheEmitDelay_NoBArrives
- TestCorrelationService_ApacheEmitDelay_ZeroDelay
- TestCorrelationService_ApacheEmitDelay_MultipleA_SameKey
- TestCorrelationService_ApacheEmitDelay_Flush

All 30 tests pass. Coverage: 75.1%

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-03 22:03:31 +00:00
9db6848757 fix: critical Keep-Alive correlation bug - network events evicted prematurely
Some checks failed
Build and Test / test (push) Has been cancelled
Build and Test / build (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
- Fix cleanExpired() to use TTL map instead of event timestamp for B events
- Increase default correlation time window from 1s to 10s
- Increase default network TTL from 30s to 120s for long sessions
- Use payload timestamp for network events when available (fallback to now)
- Add comprehensive Keep-Alive tests (TTL reset, long session scenarios)
- Bump version to 1.1.7

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-03 16:32:48 +01:00
c352e06b88 fix: create socket parent directory on startup
Some checks failed
Build and Test / test (push) Has been cancelled
Build and Test / build (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
- Create /var/run/logcorrelator/ if missing before binding sockets
- Fixes issue with tmpfs /var/run being cleared on reboot
- Add filepath import for directory handling

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-03 00:17:01 +01:00
24aa84bd9c test: add comprehensive tests to improve coverage
Some checks failed
Build and Test / test (push) Has been cancelled
Build and Test / build (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
- observability: added tests for LogLevel.String(), Warn(), Warnf(), Infof(),
  Debugf(), Error(), WithFields(), and concurrent access patterns
- file: added tests for Reopen(), Close(), empty/whitespace paths,
  validateFilePath allowed/rejected paths, concurrent writes, Flush(),
  and marshal errors
- config: added tests for TimeWindowConfig.GetDuration(),
  CorrelationConfig getters, validation scenarios (no inputs, no outputs,
  duplicate sockets, ClickHouse validation), and LogConfig.GetLevel()

Coverage improvements:
- observability: 57.7% → 79.5%
- file: 68.6% → 78.6%
- config: 69.8% → 97.7%
- total: 68.6% → 74.4%

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-02 22:52:09 +01:00
324b0042f8 fix(rpm): example config in /etc/logcorrelator + socket permissions 0666
Some checks failed
Build and Test / test (push) Has been cancelled
Build and Test / build (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
- Install logcorrelator.yml.example to /etc/logcorrelator/ instead of /usr/share/logcorrelator/
- Change default socket permissions from 0660 to 0666 (world read/write)
- Bump version to 1.1.2
- Remove CHANGELOG.md

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-02 22:07:50 +01:00
33e19b4f52 feat: Keep-Alive correlation, TTL management, SIGHUP handling, logrotate support
Some checks failed
Build and Test / test (push) Has been cancelled
Build and Test / build (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
Major features:
- One-to-many correlation mode (Keep-Alive) for HTTP connections
- Dynamic TTL for network events with reset on each correlation
- Separate configurable buffer sizes for HTTP and network events
- SIGHUP signal handling for log rotation without service restart
- FileSink.Reopen() method for log file rotation
- logrotate configuration included in RPM
- ExecReload added to systemd service

Configuration changes:
- New YAML structure with nested sections (time_window, orphan_policy, matching, buffers, ttl)
- Backward compatibility maintained for deprecated fields

Packaging:
- RPM version 1.1.0 with logrotate config
- Updated spec file and changelog
- All distributions: el8, el9, el10

Tests:
- New tests for Keep-Alive mode and TTL reset
- Updated mocks with Reopen() interface method

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-02 20:32:59 +01:00
27c7659397 fix: renforcer corrélation A/B et sorties stdout/fichier
Some checks failed
Build and Test / test (push) Has been cancelled
Build and Test / build (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
Co-authored-by: aider (openrouter/openai/gpt-5.3-codex) <aider@aider.chat>
2026-03-01 12:10:17 +01:00
a3ae5421cf chore: version 1.0.7 - add log levels
Some checks failed
Build and Test / test (push) Has been cancelled
Build and Test / build (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
- Add configurable log levels: DEBUG, INFO, WARN, ERROR
- Replace debug.enabled with log.level in configuration
- Add Warn/Warnf methods for warning messages
- Log orphan events and buffer overflow as WARN
- Log parse errors as WARN
- Log raw events and correlations as DEBUG

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-01 02:33:04 +01:00
56c2923121 chore: version 1.0.6 - simplify YAML configuration
Some checks failed
Build and Test / test (push) Has been cancelled
Build and Test / build (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
- Remove service.name and service.language (unused)
- Remove enabled flags on outputs (presence = enabled)
- Simplify correlation config: time_window_s (integer) instead of nested object
- Simplify orphan_policy to emit_orphans boolean
- Rename apache socket to http.socket
- Add socket_permissions option for unix sockets (default: 0660)
- Update tests for new configuration format

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-03-01 01:59:59 +01:00
180c57c35b chore: release v1.0.2 with critical fixes and test improvements
- fix: add missing ClickHouse driver dependency
- fix: resolve race condition in orchestrator (single goroutine per source)
- feat: add explicit source_type config for Unix socket sources
- test: improve coverage from 50.6% to 62.0%
- docs: add CHANGELOG.md with release notes
- build: update version to 1.0.2 in build scripts and Dockerfiles

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-02-28 21:45:00 +01:00
7e9535122e fix: durcir la validation et fiabiliser flush/arrêt idempotents
Co-authored-by: aider (openrouter/openai/gpt-5.3-codex) <aider@aider.chat>
2026-02-28 20:10:28 +01:00
f4d95eed41 test: update config tests for YAML format
- Replace custom directive tests with YAML-based tests
- Test valid YAML config loading
- Test invalid YAML handling
- Test default values with partial YAML config

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-02-27 15:54:13 +01:00
37f9c21672 feat: migrate configuration from custom format to YAML
- Replace custom directive-based config parser with YAML using gopkg.in/yaml.v3
- Rename config.example.conf to config.example.yml with YAML syntax
- Update default config path to /etc/logcorrelator/logcorrelator.yml
- Update Dockerfile.package to copy YAML config files
- Update packaging scripts to install logcorrelator.yml
- Update architecture.yml to document YAML configuration
- Add yaml.v3 dependency to go.mod

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-02-27 15:51:25 +01:00
8fc14c1e94 Initial commit: logcorrelator with unified packaging (DEB + RPM using fpm)
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-02-27 15:31:46 +01:00